by LINE Engineer on 2017.2.10
Hello, Kushii here.
In this blog post, I’d like to talk about the unveiling of the LINE Engineering site and what it means for the future of the LINE Engineers’ Blog.
The LINE Engineering site is a constantly updated hub for everything from LINE development culture, to open-source projects, job openings, and development documentation straight from the LINE developers site. The site is available in English, Japanese, and Korean. Support for Traditional Chinese is coming soon.
by LINE Engineer on 2017.2.8
About the LINE Security Bug Bounty Program
Hello, my name is Lee Myoung Jae (MJ) and I’m in charge of security at LINE. In this post I’m going to talk about the LINE Security Bug Bounty Program and the results of the program in 2016. The LINE Security Bug Bounty Program is an ongoing program to make the LINE app more secure for our users by letting external engineers submit bug (vulnerability) reports which we would then immediately fix.
We first conducted a trial run of the program called the LINE Bug Bounty Program from August 24 to September 23 in 2015. Based on our experience from that trial, we made various changes to create an improved bug bounty program for 2016. And on June 2, 2016, we launched the new and improved LINE Security Bug Bounty Program.
LINE Security Bug Bounty Program website: https://bugbounty.linecorp.com/
by LINE Engineer on 2017.1.31
Greetings! I’m Jun, a frontend engineer at LINE.
We had a five-day event for internal engineers called the “LINE Haskell Boot Camp” starting on October 24th, 2016 at the LINE Shibuya office. I will share what the event was about, and what Haskell is in this post.
What is Haskell?
Haskell is a modern application programming language that helps you write high-performance software with a great deal of flexibility, composability, and safety. With recent successful cases of adopting Haskell by companies like Facebook or Standard Chartered, this innovative tool has been gaining attention as it has inspired many libraries and concepts in other languages.
LINE is not an exception in this regard. With growing interest in Haskell, there is an increasing number of developers having study group meetings on Haskell or developing internal services written in Haskell. The LINE Haskell Boot Camp was a five-day gathering of engineers attempting to enter the world of safer and happier programming. Instead of diving into the depths of Haskell’s theoretical background, we chose to have a hands-on experience to effectively spur further enthusiasm for Haskell. The event was organized by Han, a LINE Plus Corporation engineer who recently released an internal web service in Haskell.
Here are some examples of making a high-performance program while keeping the code safe and simple at the same time with Haskell.
import Data.List (isPrefixOf) appendIfNeeded xs ys = if xs `isPrefixOf` ys then ys else xs ++ ys
In the definition of the
appendIfNeeded function above, none of the types are stated. However, thanks to Haskell’s type inference, the parameters of the function,
ys, are guaranteed to be comparable lists. If the value is not a list or if the elements of the lists are incomparable, the compiler will raise an error. Haskell’s type inference keeps the code simple while reducing the risk of having wrong code executed at run time.
The safety guaranteed by Haskell is not only limited to value types. Let’s say that there is a function type as shown below.
toString :: Integer -> String
toString function shown above is an ordinary function that is applied to an integer and returns a string. This function can be implemented in any language. However, Haskell guarantees that if the function is applied to the same integer, the same exact string will be returned, regardless of where and when the function is executed. In other words, for Haskell functions, if the input value is the same, the output value is also always the same. In other languages, the result of a function call is uncertain without actually looking into the code. Calling a function could lead to the unintentional transmission of specific files from a file system to a server, or even to the firing of intercontinental ballistic missiles. However in Haskell, you can be sure that there won’t be any such unexpected incidents.
readFile :: FilePath -> IO ByteString
On the other hand, the
readFile above is a function that is applied to a certain file path and then reads its content. This function needs to access a file system, and the file system can change at anytime. In these cases, Haskell can make a function an action by using a type called
IO. An action can interact with the real world and can make meaningful side effects like file I/O or network communication. Moreover, the important thing is that only the actions designated by the
IO type can make these side effects. You don’t have to look into any other code other than the actions when an exception occurs for the side effect. This separation of the side effect not only makes the code structure healthy but also reduces the cost of maintenance.
As mentioned above, Haskell is a language that guarantees referential transparency by managing side effects at the type level. The Haskell compiler, at compile time, can tell which type a function has as its parameters and return value, if a function returns the same value regardless of time and location, and if it can cause side effects. This naturally makes it easier to implement parallelism and concurrency. Parallelism and concurrency are the two most important, and also difficult, subjects of modern programming. In most languages, programmers solve these difficulties by paying more attention. However, as the Haskell compiler already has sufficient information, it can manage concurrency on its own.
f x + g y
This expression expresses the summation of the results of applying a function
x and a function
y. As Haskell’s + is not an
IO action, the compiler knows that neither
g is an action, even without a type notation. This means that
f x and
g y will return constant values regardless of where and when they are executed. To put it differently, even if
f x and
g y are executed simultaneously or executed on different CPU cores, the return values will always be the same. Therefore, by providing a simple hint about parallelism to the Haskell compiler, you can make a program which can run in parallel.
f x `par` g y `pseq` f x + g y
In the code above, the
par function makes
f x and
g y run in parallel, while the
pseq function makes it so that the latter expression is evaluated after the previous part is done. Now, with proper compile and runtime options,
f x and
g y will run in parallel automatically. This means that you don’t have the hassle of dealing with processes, threads or thread pools to implement concurrency and parallelism.
As the examples above have shown, Haskell has an effective and innovative structure which enables safety, referential transparency and concurrency with simple code. Furthermore, Haskell also has plenty of advantages including performance, composability, community and ecosystem.
Haskell Boot Camp
The five-day boot camp involved two-hour sessions every afternoon and was a demanding time commitment for busy people. Nevertheless, we had so many participants that the venue was overloaded and we had to bring in extra chairs and put them by the door. It was encouraging to see such a large number of fellow LINE engineers eager to experience a better technology.
Each session began with Han’s presentation slides which went through basic concepts and conventions and was followed by a hands-on live coding session. Haskell has a concise syntax, but it may look totally alien for newcomers. Many beginners would give up and break off at this point but to prevent this from happening, the boot camp was focused on getting everyone familiar with actually coding in Haskell.
An interesting thing was that the engineers came from a variety of backgrounds from client, server, to the front end. Haskell, under the de facto motto of “avoid success at all cost”, has pushed forward an avant garde advancement of the Haskell programming language and has resulted in a positive influence over many other languages. This means that learning Haskell will help in understanding and mastering other languages as well. I felt that the introduction of these concepts and their counterparts in other languages was beneficial for both understanding Haskell and understanding Haskell’s value in the world of programming.
One thing I liked about the boot camp was that, while Haskell itself can be infinitely profound, the sessions tried to stay as practical as possible. People with prior knowledge of Haskell associate it with words like lambda calculus, category theory, type class, and monad, all of which sound very academic and terrifying. On the other hand, Han described Haskell in terms of its pragmatic advantages like flexibility, composability, and safety. Of course you don’t need a doctorate in category theory to use Haskell. Learning Haskell shouldn’t be too different from learning other languages; once you get to know how to code in it, you can try building stuff with it and proceed to more sophisticated notions. The LINE Haskell Boot Camp seemed to accomplish this purpose pretty well.
The boot camp followed a curriculum slightly different from the traditional approach taken by most Haskell gatherings. The first day began with an overview of Stack, a development tool, and Stackage, the repository of Haskell packages and documentation. It then went on with the goal of building something useful and covered required tools such as Text and Bytestring which resolves the inefficiency of strings, Template Haskell which enables metaprogramming in Haskell, and WAI which is the web application interface of Haskell. Haskell meetups tend to be academic, so I found it impressive that this boot camp was filled with ready-to-use knowledge for developers from various fields.
We weren’t completely immersed in code or libraries for the whole five days. One important purpose of the boot camp was to garner interest in and to encourage everyone to use Haskell in the long term. Han’s idea was to ask “Why Haskell at LINE?”. We came up with real-world answers to that question. At LINE, a large number of engineers work together on a service that operates on an even greater number of virtual or physical machines, which is then used by a far greater number of users. In this context, service reliability is vital. To achieve high levels of service reliability in mainstream languages, developers are forced to pay attention to following strict conventions or using outside devices such as static code analyzers or testing frameworks. On the other hand, Haskell guarantees safer programs at the language level, making it a viable option when reliability is paramount. Another gift of Haskell is that it appeals to programmers who are passionate about the act of programming itself. Using Haskell increases the chance of working with motivated developers who pursue excellence. If you are willing to recruit talented developers like LINE does, Haskell is surely worth a try.
The Monad Fear, which was the name of the final day’s presentation, was also quite remarkable. The monad is one of those concepts that makes Haskell seem baffling and frustrates budding Haskellers. The concept of monads actually comes from category theory and fully understanding it can require a high level of academic knowledge. However, Han cited a quote from a fellow Haskeller:
Attempting to learn how to use monads by understanding what they are is like asking “What is a musical instrument?” and then assuming once you know the answer to that, you’ll be able to play all of the musical instruments. — kqr, The “What are monads?” fallacy
Haskell hobbyists may find it surprising that LINE uses Haskell at all or that it held an internal Haskell boot camp. So did I as a long-time personal follower of this language who never expected to be able to use it at the workplace. The LINE Haskell Boot Camp was a reminder for us that LINE, as a technology company, values the importance of engineering and its underlying backbone. It was also a personally delightful occasion that made me realize I actually work together with colleagues who are motivated and willing to learn.
The greatest achievement of this boot camp, I think, is that it demonstrated that a future tool is now mature enough to be chosen even in a commercial context. Despite its academic origin, commercially successful deployment of Haskell is now a reality. The practical aspects of Haskell covered in the boot camp attracted my attention as well, especially when it comes to software transactional memory which looked promising enough to be something I would actually want to use in production. Although we couldn’t cover all of the details of these appealing aspects in the given time, the boot camp did its job in sparking my interest in Haskell, allowing me to have a better grasp of the tools, and encouraging me to venture into using Haskell as a real-world weapon.
I hope my efforts in using Haskell will eventually bear fruit so that I will have some more positive news to share with you. For instance, I hope to, one day, deliver a technically detailed introduction of software written in Haskell that sits within LINE’s server architecture. Below is the presentation slides of the final day, The Monad Fear. Please take a look.
by LINE Engineer on 2016.11.30
Hi, my name is Hasebe. I’m in charge of the development of LINE Notify.
In a previous post, we introduced how you can use LINE Notify to send messages to LINE from the command-line. Today, I’m going to introduce two features newly implemented in the LINE Notify API. One is sending stickers and the other is uploading images.
The use case demonstrated in the previous post was sending a build result from Jenkins to LINE by using LINE Notify. In that use case, an image of Moon laughing was used to show how it can notify us of a build failure. It was during that time that we came to wonder, “why not make LINE Notify send stickers as well as images?”
We thought that sending stickers would make LINE Notify more “LINE-ish.” That’s why we decided to develop this feature.
Sending stickers from the command-line
Let’s try sending a sticker by using the curl command.
$ curl -X POST https://notify-api.line.me/api/notify -H 'Authorization: Bearer YOUR_PERSONAL_ACCESS_TOKEN' -F 'message=test' -F 'stickerPackageId=1' -F 'stickerId=113'
You can now send stickers in this way.
by LINE Engineer on 2016.11.24
This post introduces how to run tests with the ad client module provided for the LINE Platform. The LINE ad client module runs on both mobile and the web. This post will focus on testing with a mobile client.
LINE Ads Platform overview
The structure of the LINE Platform is quite simple as shown below. Various types of protocols can be used for server-client communications. This post will focus on testing with the HTTP protocol.
by LINE Engineer on 2016.11.14
In this blog post, I’d like to explain how the LINE TODAY service was developed using the Agile development method. LINE TODAY is a mobile news service that was released in Taiwan, Thailand, Indonesia, Myanmar, and the United States in early 2016. As of July 30, 2016, the service recorded nearly 30M daily PV (page views). In Japan, a service similar to LINE TODAY is available under the name of LINE NEWS.
The LINE TODAY development project is a transnational project with users, customers, developers and planners from various countries. Plenty of members participated in the project including developers, planners, QA engineers, UIT (User Interface Technology) engineers, designers, and business owners located across Taiwan, Dalian, and Korea. Moreover, it was a newly created team starting everything from scratch. Most of the developers were rookies unfamiliar to the LINE development environments and skills. No one had the experience of handling a new development project and no one knew the overall process of building a global product.
The LINE TODAY service was planned from the beginning to be released in several countries such as Taiwan, Thailand, and Indonesia. Various requirements had to be taken into account. For example, we had to consider that the content providers (CP) might prefer different types of content feeding mechanisms; Taiwan CPs preferred FTP whereas Thailand and Indonesia CPs preferred RSS.
LINE TODAY was a project with very tight development schedules. The LINE TODAY service was released twice as a FastTrack version and RegularTrack version. The FastTrack version was a proof of concept for evaluating business potential, which was then leveraged to the RegularTrack version aimed at providing the service in the long term. The FastTrack version had to be designed, developed, tested and released in 6 weeks and the RegularTrack version within 3 months after the FastTrack version was released.
by LINE Engineer on 2016.11.9
Hello, my name is Inami (@inamiy), a member of the LINE development team. In this post, I’ll be sharing my experience from being a part of the panel discussion at GitHub Universe 2016.
About GitHub Universe 2016
GitHub Universe 2016 took place during September 13-15 at Pier 70 in San Francisco.
Inside the refurbished warehouse, now conference hall, gathered over 1500 developers, technology and business leaders. Beginning with a keynote presentation from GitHub cofounder and CEO Chris Wanstrath, over 40 speakers from all over the globe gave talks on various open source projects and business activities.
I was invited as the sole Japanese member at the panel discussion.