Among Ryan Nelson’s nightmare scenarios is when a major league baseball team pinch-hits for the lead-off batter in an inning. That simple act has mighty repercussions when you're the director of operations for Major League Baseball Advanced Media (MLBAM), which runs the MLB.com Web sites where millions of fans track games pitch by pitch.
“After the third out in an inning, everyone goes away, then we have 60 seconds for commercials,” Nelson says. “God forbid they pinch-hit at the top of the next inning. Then half a million people request the same JPEG within a 10-second span. You go from zero to 600 miles per hour really fast.”
These days, however, MLB.com is able to weather such pinch-hitter storms with relative aplomb. As Nelson explained during a presentation at the Network IT Roadmap event in Chicago in June, a combination of Citrix NetScaler appliances and Akamai content distribution services is helping the company ensure nobody misses a pitch.
About 65 Web servers support MLB.com, which provides Web services for all 30 major league and some 100 minor league teams. Additionally, MLBAM runs sites for Major League Soccer and the annual NCAA men’s basketball tournament, for which it broadcasts all games live via streaming video.
Nelson arrived at MLBAM in May 2001, during the organization’s first baseball season. That season “didn’t go so well,” he says, beginning with opening day in April when some fans found they couldn’t access audio broadcasts of games. “We rearchitected a lot of stuff on the fly in the first couple of months of that season. I was brought in as part of that.”
At that time, MLB.com was uniformly balanced across about 40 Web servers. That meant if any component had a glitch that caused it to use too many system resources, the problem would eventually spread across all the servers, creating a drag on the entire infrastructure.
Compounding the problem was the nature of MLB.com’s Web pages. Many pages, such as the GameDay pages where visitors can follow games pitch by pitch, are dynamic in nature, with content that changes with every pitch. Many pages also have lots of small components, including box scores, team logos that vary by site, schedules and statistics.
A typical page may have 100 such components, each of which amounts to a separate image. Downloading that page means a typical Web browser must connect and disconnect to the site 100 times, Nelson says. “That’s the kind of thing that really slows down the way a Web site feels. It loads in a lot of latency,” he says.
Nelson had experience with an early version of the NetScaler appliance (before NetScaler was acquired by Citrix) from a previous employer. He originally introduced the boxes to MLB.com solely as SSL gateways, enabling the firm to save money on SSL costs. MLB.com was paying $60,000 per year in support contracts for SSL hardware, plus $20 to $800 for each server certificate. Using a pair of NetScaler appliances as SSL gateways meant it needed certificates only on those two appliances. The cost savings in the first year paid for the devices, Nelson says.