Promise and peril in the journey to DevOps

Like many organizations today, Nationwide must regularly churn out new software applications to stay competitive in a crowded industry.

But the insurance and financial services company wasn't always able to keep up with demand. For years, Nationwide relied on 23 business unit IT shops, each of which used its own hodgepodge of tools and methodologies, to create new products and services. That is, until DevOps technology director Carmen DeArdo realized that Nationwide's "monolithic" approach to developing apps was resulting in bloated project teams, sprawling design plans and painfully slow software development life cycles.

Rather than throw more staff at the problem, DeArdo opted to migrate from the stilted world of waterfall development to a DevOps culture of collaboration.

Definitions vary, but DevOps is generally recognized as a software development movement that encourages automation, integration and greater collaboration between software developers and operations people. By bridging the gap between those two sometimes warring factions, DevOps establishes a consistent, repeatable way for IT to manage its production environment, resulting in faster time to market, higher productivity levels and smoother server and app deployment.

Since embracing DevOps in 2009, Nationwide has improved software quality by 50% and reduced user downtime by 70%. Today, more than 100 agile teams, growing at a rate of 35% a year, handle a whopping 60% of the company's development work and new projects. Code is continually integrated and deployed into a development environment several times a day, resulting in "higher quality, higher productivity and more on-time delivery" of applications, according to DeArdo.

Nationwide is one of a growing number of companies embracing DevOps to contend with the consumerization of enterprise software. Long gone are the days of quarterly releases. These days, organizations are expected to produce code and launch new software tools with the speed and agility of an Amazon or a Google. At the same time, the push for frequent releases gives rise to complex and unruly production environments that are tough to manage. Many IT leaders are hoping to find an answer in DevOps, despite some drawbacks, including the risk of cultural warfare and the possibility of heightened peer criticism.

At best, DevOps promises to help businesses keep pace with today's frenetic software delivery life cycle. The goal is for developers to issue software releases more frequently without adding layers of complexity to the infrastructure, thereby lightening the load for operations.

Researchers are even discovering a correlation between IT agility and improvements in business performance. In its "2014 State of DevOps Report," which was based on a survey of 9,200 IT professionals from around the world, software automation vendor Puppet Labs states that "high-performing IT organizations are more agile and reliable: They deploy code 30 times more frequently than their lower-performing peers, with 50% fewer failures. . . . [And companies] with high IT performance are twice as likely to exceed their profitability, market share and productivity goals."

Research firm Gartner predicts that about 25% of Global 2000 companies will adopt the DevOps philosophy by 2016. And the market for DevOps tools is expected to grow to $2.3 billion by the end of this year, driven in part by sales of new products from vendors such as Chef, Puppet, Ansible, IBM, Microsoft and Atlassian.

Don't go chasing waterfall

This isn't IT's first crack at simplifying software development. The Information Technology Infrastructure Library (ITIL) is a decades-old approach to aligning IT services with business needs. But with its best practices detailed in five voluminous tomes, rivaling the size and scope of Winston Churchill's memoirs, it's no wonder many companies are abandoning ITIL's formal prescriptions.

"DevOps is more of a common-sense approach, a ground-up cultural change that encourages creativity and collaboration," says Gartner analyst Laurie Wurster.

But persuading developers and operations people to join forces to accelerate software delivery is no easy feat. Embracing DevOps takes strong leadership, reliable metrics, compelling incentives, effective tools and something akin to a cultural revolution.

Indeed, in a Saugatuck Research survey of more than 300 development and IT operations professionals and managers, only 37% of the respondents said they have formal DevOps strategies in place. And more than half of those polled said that "overcoming cultural habits inside my organization/company" was an obstacle to fully adopting DevOps principles.

Culture vultures

Just ask Ryan Kelley. A systems engineer at the National Renewable Energy Laboratory (NREL), Kelley first started using DevOps for a project that involved gaining regulatory approval for adding new cloud capabilities to the lab's IT infrastructure. "We were really siloed," he recalls. "There were certain teams that did their own thing without much communication in between. What I liked about DevOps was the notion of breaking down those silos."

However, introducing continuous delivery principles proved more difficult than Kelley anticipated. "When you have an established culture, it's hard to break down the barriers of trying to do things differently," he says. "For any large enterprise, it's the biggest stumbling block -- getting past the change culturally."

Jez Humble agrees.

"The problem is you have one department -- operations -- that is measured by stability. And you've got another department -- development -- which is measured by throughput, such as how fast can we get code written," says Humble, who is co-author of Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation and engineering vice president at Chef, an IT automation software provider that counts NREL among its customers. "When those two departments are measured that way, they immediately come into conflict."

In other words, operations people keep the boat afloat while developers love to rock it. Complicating matters further is the fact that the two groups usually communicate via ticketing systems rather than face-to-face encounters.

It's a hurdle Nationwide was able to clear by establishing what it calls an application development center. Originally an "experiment" to centralize six development teams with about 50 people in all, the ADC has grown into 180 cross-functional teams, each dedicated to agile development, says DeArdo.

Another common barrier to DevOps adoption is a steep learning curve. "DevOps requires different skills, so you need a robust training capability if you're going to implement it," says DeArdo. For that reason, Nationwide now holds "Teaching Thursday" sessions twice a month -- informal meetings where people can swap tips and pool their expertise on continuous software delivery.

The NREL discovered a similar knowledge gap when it first introduced DevOps. Kelley says that while the lab is accustomed to managing a traditional data center, its IT talent had a limited understanding of day-to-day software development practices. As a result, he says, "DevOps was a giant learning curve and it still is. We still have a long way to go. We spend a lot of time just trying to understand what Chef can do, these new methodologies and how to implement them."

Offering gains for pains

From culture shock to intense training, Wurster says it's critical that companies offer their developers and operations people something in exchange for their trouble. "DevOps includes culture change, and it builds on taking responsibility, being creative and being attentive," she says, adding that it won't work unless employees are offered an incentive to try it.

Promotions, raises, bonuses -- Wurster says those are all rewards that companies can use to drive greater adoption of DevOps. Being able to see the fruits of your labor is also a powerful incentive. In the past, it wasn't uncommon for a developer to spend months creating an application, only to throw it over the wall to operations and never hear about it again.

Humble knows what that's like. A veteran engineer, he says he once worked tirelessly on a software product, only to find out several years later that, "although it had come in on time, on budget and on schedule, the company that we built it for actually had to cancel it because no one wanted it. That was very sobering for me and was one of the epiphanies I had about DevOps continuous delivery."