In
the Web world, much is made out of average response times. But
they can't be the only measure of a site's performance.
When
a system is under serious stress, you will get weird behaviors,"
says Alberto Savoia, chief technologist in Keynote Systems KeyReadiness
Services group. "Averages can hide those problems."
Savoia
equates the situation to sticking one foot in boiling water and
the other in freezing water. If you take the average, the feet
appear fine. But looking at the individual units tells an entirely
different story. With the Web, an average response time of 5 seconds
could mean one user experiences a 1-second response time while
another has a 10-second response time, an unacceptable delay on
the Web.
Savoia
characterizes Internet performance problems as:
-
Those you can do something about (add more servers and bigger
pipes).
-
Those you can do nothing about (the Internet as a whole is slow).
-
Those you can reduce, but can't entirely control (cache
popular content at the edges of the network).
United
Parcel Service uses Keynote's monitoring service to keep
tabs on itself from the outside world. "It measures us from
15 different locations, and we react," says John Nallin,
vice president of IS at UPS.
To
test for the problems you can control, Keynote looks at user logs
to find out how customers use a Web site. Only some site visitors
complete transactions. Others are just looking for information,
so they generate less load.
Taking
user abandonment into account, Keynote then tests loads based
on the average number of people that will make a transaction,
browse part way through a site and go to the home page. For companies
building test routines, it's important to pound on the site
like normal users would, not like 100,000 people going through
the entire process.
Savoia says this gives a more accurate view, so problems will be more easily detectable.
Contact Multimedia Editor Jason Meserve at jmeserve@nww.com.
