A new mobile market research project now measures and ranks mobile Web sites, trying to identify how well they perform for handheld device users.

Intended to be an ongoing weekly measure of mobile Web performance, the mobile Web performance index, from Keynote Competitive Research, uses a trio of metrics to evaluate 10 popular online destinations. The index doesn’t compare the wireless network metrics to wired broadband metrics, but it does show wide variance in how well the mobile sites perform.

The three metrics are:
- Response time in seconds – the average time it took for the mobile Web site to download to the smartphone.
- Download speed in byte/second – the average speed in byte/sec for the site to download.
- Success rate – the percentage of time the mobile site successfully downloaded the complete content to the mobile phone.

The nine sites surveyed were a mix of types and content, likely to be popular with mobile users: CNN, ESPN, Facebook, Fandango, Google, MapQuest, MSN, Schwab (online brokerage firm), The Weather Channel and Yahoo.

The initial data covers the period from Nov. 19-26.

Response times varied from just less than 7 seconds, for Google and Facebook, to just more than 15 seconds for ESPN. To get an idea how long that is, count “one thousand one, one thousand two” and so on till you reach 15. You probably won’t bother. Part of the variances was related to the size of the main page download: Google downloaded just 6,684 bytes, but ESPN downloaded 26,635.

But the byte/second metric showed a very different ranking, often reversing the previous ranking. In many cases, the sites with the largest Web downloads in bytes were also had the highest bytes per second. Facebook was last with an average of 693 byte/sec for a 4,663-byte download. But CNN was No.1 with 2,368 byte/sec, for a 24,211-byte download, followed by ESPN with 1,756 byte/sec, for a 26,635-byte download. Google was mid-way in the pack: 1,009 byte/sec, for a 6,684-byte download.

The sites were much closer to each other in their success rates, again the percentage of time each site successfully downloaded the full content to the mobile phone. Facebook and Google tied for first with a success rate of 99.23%. CNN and Yahoo were nearly tied for second, with 98.99% and 98.97% respectively. But even the last place site, MSN, had a 97% success rate.