- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
Network World - Netflix's Open Source Software strategy started on June 23, 2011. One of the company's senior software engineers had an idea: "At some point, I think it would be valuable to open source the Zookeeper library I've written," Jordan Zimmerman wrote to his bosses, talking about a piece of customized code he helped develop. "Does Netflix have a policy on that?"
The response he got: “Go for it. Our policy is no policies ;-)”
During the past two years Netflix has pulled back the curtains to provide a behind-the-scenes glimpse into how it runs one of the most popular video streaming websites on the Internet, almost entirely in the public cloud. The company has open sourced dozens of tools it’s developed internally. In doing so, some argue that Netflix is turning into one of the most important cloud computing companies in the industry, not only by proving that a company making $3.7 billion annually can run some of its most critical workloads in the public cloud, but also by sharing with developers how it’s being done and providing others with a path to follow.
VIDEO STREAMING TIPS: The 10 best options for online video streaming
Netflix Open Source (OSS) is a collection of Apache code bundles that the company has created and open sourced. As one of the biggest users of Amazon Web Service’s public cloud, many are related to plugins for using AWS resources; others are add-ons for other open source projects like Apache Hadoop, Cassandra and Pig. But mostly they focus on deploying public cloud computing resources, creating tools for automating and managing tasks, ensuring high availability and analyzing use.
Perhaps the most notable of Netflix’s OSS tools is the Simian Army – a series of tools that test for the tolerance of your cloud deployment by randomly shutting down certain systems. Chaos Monkey automatically selects individual virtual machines to collapse, while Chaos Gorilla does the same thing on a larger scale by replicating an entire Availability Zone in AWS’s cloud to shut down. Other projects like Asgard provide a cloud management dashboard to manage resources, while ICE tracks cloud spending by usage. Revealing the inner secrets of how it manages the tens of thousands of instances it uses in Amazon’s cloud at any given time isn’t all altruistic for Netflix though.
“There’s this massive realization in the industry that if you’re benefitting from these projects, then why not pay it forward and get the benefit of community input,” says Michael Skok, an industry watcher and venture capitalist at North Bridge Venture Partners who advises early stage cloud startups. “Ultimately you’re reducing your costs and increasing your value when you’re contributing to a movement. Everybody wins.”
The chief architect behind Netflix’s cloud and OSS strategy is Adrian Cockcroft, a former distinguished engineer at eBay and Sun, who says Netflix has many agendas in developing OSS. For one, it’s working to establish Netflix’s process as a best practice way of operating in the public cloud. Doing so allows the company to benefit from the knowledge of the broader open source community who recommend improvements. Furthermore, it helps Netflix hire and retain top engineering talent all while building up the company’s technology brand.