Keynote to benchmark streaming media performance

SAN MATEO, CALIF. - Keynote Systems last week launched a new index designed to measure the performance of streaming media on the Internet.

Dubbed the "The Streaming Media 20," the index aggregates weekly performance information of 20 streaming media sites in four categories: audio e-commerce, financial audio, broadcast radio and cable television. Keynote measures the audio and video delivery quality each day to build the index, which can be used as the basis for service-level agreements with streaming media service providers.

"The industry needs to know how well streaming media is doing," says Matt Parks, senior product manager at Keynote. "If it's a Web page with a 4-second download time, everyone can understand that. But with video, if you say there's 14% packet loss, no one will understand what that means."

Keynote takes into account three factors when building the scale: How long it takes the clip to start from the time someone clicks play; how well the audio performs; and how well the video performs. Each is ranked on a score of 1 (lowest) to 10 (highest).

Audio and video quality are measured by the intended viewing experience, based on how the clip was encoded, as well as the actual quality delivered to the user.

Today, scoring a 10 is nearly impossible because one would have to deliver DVD-quality video, which requires many megabytes of bandwidth. Naturally, audio-only sites will score lower because they do not have the video component to boost their scores.

"Most people will get 5s and 6s, tops," Parks says. "We set the bar high because that's what end users are expecting to see - TV-quality video."

To balance lower scores, Keynote also gives a rendering score that measures if the end user received the intended quality set by the broadcaster. For instance, if an audio stream is encoded for only 7K bit/sec and the end user received 7K bit/sec, the site would get 100% in the rendering category but perhaps only a 2 or 3 on overall quality.

Keynote uses remote measuring computers located on multiple backbones in the 10 largest U.S. cities. The company only supports the RealMedia format now, but the firm plans to support Windows Media soon. Sites that make up the index included Amazon.com, The Weather Channel, MTV.com, Bloomberg, Chicago's WGN, the Motley Fool and CNN.

For those sites that want to have their streaming media performance measured, Keynote charges between $395 and $9,995, depending on the size of the clips and sampling frequency.

The index is available on Keynote's site at www.keynote.com.