A tour of DreamWorks Studios' data center

One computer-generated film is made up of as many as 120,000 frames and requires 200TB of disk storage

DreamWorks Studios is busy producing eight to 10 films at any given time, and each film produces 120,000 frames of film and uses 200TB of storage. In recent years, 150 engineers have been working on creating software that can take advantage of 25,000 CPUs in a parallel fashion.

About 15% of DreamWorks' servers are virtualized, about 20% of their CG rendering is performed using cloud services, and the company has yet to find a need for solid state drives (SSDs), according to Mike Cutler, global director of Infrastructure Operations at DreamWorks Animation.

DreamWorks took a few dozen attendees of the SNW trade show here on a tour of its Redwood City, Calif., studio and data center. Among the amenities the company offers its employees is free food and medical care -- the doctor's office resides directly across the hall from the data center.

In the data center are 3.8 petabytes of disk storage capacity and 4,000 servers with 25,000 CPU cores.

Dreamworks

While the studio may not use a lot of hypervisor technology in its servers, it invests heavily in state-of-the-art server blades, storage arrays and networks to keep its artists efficient and satisfied.

"The number one thing we invest in here to make sure our artists and engineers stay happy is our technology, because that's what they spend their day in. If we don't stay a couple steps ahead of state-of-the-art, they'll try to go find it somewhere else," said Kate Swanborg, head of enterprise marketing at DreamWorks.

DreamWorks, which has two studios in the U.S. and one in Bangalore, has a goal of releasing three films a year, but each feature film takes up to three years to create. The company always has multiple films in production, anywhere from eight to 10 at a time. With each film release, audience expectations of the quality of CG imagery grows, according to Swanborg.

The amount of processing power and storage capacity required during the production of a CG 3D film can be tremendous. There are more than 300 high-end workstations used for creating the animation. One film can take more than 60 million hours of rendering work, using 17,000 cores simultaneously.

Each animated character has up to 2,000 control points or features that can be manipulated by an animator. And, each character takes six months to craft, according to Swanborg. "Each pixel in every film is controllable," she said, adding that a competed film has more than 250 billion pixels in it.

At 24 frames a second, and 120,000 frames per film, just one movie production can consume 200TB of data storage capacity and contain half a billion data files. "That's the same number of components as a Boeing jet," Swanborg said.

There are about 400,000 processing jobs per day on the studio's servers. DreamWorks uses Red Hat Enterprise MRG for messaging and scheduling those jobs in the proper sequence. "And, most of it is done in parallel," Cutler said.

Not including developers working on actual film productions, DreamWorks has 150 software engineers who keep applications running smoothly, according to Jeff Wike, director of R&D at the Redwood City Studio. Wike, who said he gave up a "promising career" in the aerospace industry, said about 20% of the company's software engineers have Ph.Ds.

"With a movie coming out every three months, it's all like brand new. So things never get stale," Wike said.

For the past three years, DreamWorks software engineering has been "parallelizing its software" to take advantage of the latest Intel 16-core Sandy Bridge processors in its servers.

"We don't write all of our software, but we do write a lot of it. We buy where we can and build where we must," Wike said. "We have the advantage that we're driven by a CEO who really, really loves technology and doesn't understand it at all. It's a great thing because he doesn't know what's hard and he doesn't know what's impossible. He'll just say we should have a tool that can just do this. So we just scratch our heads and say, 'OK, I guess we can do that.' "

The amount of work that goes into a film is staggering, but the cost can certainly raise eyebrows as well. One 90-minute feature film, such as Shrek 4, costs about $130 million to produce. An additional $130 million to $150 million is then spent on distribution and marketing.

And, while CG imagery may be the final product, creating an animated movie takes hand crafting. Each DreamWorks film is story boarded, meaning that each scene is hand drawn by an artist. Films can require anywhere from 70,000 to 100,000 story boards.

DreamWorks has standardized much of its IT infrasctucture on Hewlett-Packard HP BladeSystem c-Class server blades, 3,000 of them, which consist of preconfigured compute, storage and network architecture. It also uses HP NAS and HP's 3Par storage arrays. The only exception to the HP hardware is a small amount of storage from Hitachi Data Systems and NetApp NAS.

The studio's server racks back up into ventilation "chimneys" where the hot exhaust is drawn up and out of the room. "Cold air and hot air never mix," Cutler said. "So that's 30 degrees (F) of hot air that we don't have to cool."

Just as with its first use of 3D imaging in 2009 for its Monsters vs. Aliens film, CG is also pushing the technology requirements at DreamWorks. 3D required an additional 100TB of storage capacity for productions because it doubled many of the images.

This holiday season, the release of The Hobbit by MGM will mark the first time a motion picture has been made using 48 frames per second technology.

"If that's an experience consumers appreciate, that will have a huge impact on storage and rendering," Wike said.

"And it may have a price tag attached to it, in the same way stereoscopic films did, but if it provides a premium experience people are willing to pay for that's OK,' Swanborg said. "That's a great tradeoff from the content creation standpoint."

covers storage, disaster recovery and business continuity, financial services infrastructure and health care IT for Computerworld. Follow Lucas on Twitter at @lucasmearian or subscribe to Lucas's RSS feed . His e-mail address is lmearian@computerworld.com.

This story, "A tour of DreamWorks Studios' data center" was originally published by Computerworld.

From CSO: 7 security mistakes people make with their mobile device
Join the discussion
Be the first to comment on this article. Our Commenting Policies