One of Facebook’s most important weapons in the fight against abuse is a system called Sigma. Sigma’s job is to proactively identify malicious content such as spam, phishing attacks, and links to malware. All of the bad content detected by Sigma is removed automatically so it doesn’t show up in your News Feed.
We recently completed a two-year-long major redesign of Sigma, which involved replacing the in-house FXL language previously used to program Sigma with Haskell. The Haskell-powered Sigma now runs in production, serving more than one million requests per second. Our codebase includes hundreds of thousands of lines of Haskell, and the spam-fighting team at Facebook, which runs to dozens of engineers, is now using Haskell day-to-day.
This was a long and complex migration, with many ups and downs. In the talk I’d like to explain what led us to choose Haskell, and share some of the experiences and lessons we’ve learned along the way.
Simon Marlow is a Software Engineer on Facebook’s Site Integrity Infrastructure team in London. He is working on Haxl, a Haskell-based domain-specific language that is used by the teams fighting spam and malware. Simon is a co-author of the Glasgow Haskell Compiler, author of the book “Parallel and Concurrent Programming in Haskell”, and has a string of research publications in functional programming, language design, compilers, and language implementation.