How we did it

We used approximately 100 PC's to drive the tests. These were Compaq Deskpro EN systems with 450-MHz processors and 256M bytes of RAM. All the machines ran FreeBSD-3.3 as the underlying operating system and IRCache's Web Polygraph 2.2.9 test software (polygraph.ircache.net) to generate HTTP requests and responses.

Each machine was identically configured, except for its IP address. One pair of these machines generated up to 400 HTTP requests per second. All the Polygraph PC's used dual-mode 10/100 network interface cards and all were running at 100 M bit/sec across the network. Other network gear, such as hubs and switches, varies from bench to bench as each vendor was responsible for bringing its their own network gear to tie the network together.

Each product underwent three separate tests: Maximum Segment Lifetime (MSL), PolyMix-2 and downtime.

The MSL test ensures that the product keeps recently-closed connections in the TIME_WAIT state for at least 60 seconds. Why is this important? A product may be able to achieve higher performance with a lower setting because resources (TCP control blocks, for example) would be released more quickly.

PolyMix-2 was the most important test in this suite because we used it to collect all performance data. Before running a PolyMix-2 test, we filled each cache. Reporting the performance of a cache that starts out empty is worthless because production caches are full during normal operation. To fill the cache we first flush it, and then run the Polygraph traffic generator so that the amount of cacheable miss traffic is at least twice the cache size. All the measurements reported in this article are taken from the second half of the 14 hour PolyMix-2 test (See time chart, page xx).

With the downtime test, we found out how quickly a product recovers from a loss of power or a hardware/software reset. While Polygraph requests objects at the rate of three per second, we turned off power to all of the vendor's equipment, including the networking gear. After five seconds, power is restored and we started a timer. We measured how long it took each cache to serve its first miss and its first hit.

The authors are employed by the University of California, San Diego, and together they comprise the IRCache group, which finds its roots in National Laboratory for Applied Network Research. They work in Boulder, Colo.

Rousskov is the author of Web Polygraph, and he can be reached at rousskov@ircache.net . Wessels is the creator of the IRCache group and the author of Squid caching proxy, and he can be reached at wessels@ircache.net . Chisholm works on various IRCache projects, including Polygraph tools and Squid performance optimization, and he can be reached at chisolm@ircache.net .

Back to the main review

Cache panache
A well-placed caching device can boost Web site performance and cut WAN costs.

The bake-off workload
Details about our model

Interactive buyers guide

Law firm defends move from software to hardware caching
Better performance drives Perkins Coie from software proxy servers to CacheFlow's Internet Caching Appliances.