How we conducted Wi-Fi scalability test

The "enterprise" in "enterprise Wi-Fi scalability" implies a larger system than a handful of access points, each supporting a few clients. Accordingly, we asked each participating vendor to supply 25 access points, at least two controllers and/or switches with enough ports for all the access points, plus power over Ethernet for the access points.

We assessed enterprise Wi-Fi scalability using five sets of tests: throughput, latency, VoIP call capacity, and single- and multiclient roaming. The test instrument for this project was the VeriWave WT-90, a new chassis-based system that can concurrently emulate hundreds of clients associated with dozens of access points. VeriWave custom-developed a new suite of tests especially for this project.

Aruba 6000

For the test bed, we used three WT-90 chassis, collectively equipped with one Gigabit Ethernet port and 25 802.11b/g test ports (the WT-90s also support 802.11a, which we did not use in this test).

To reduce interference and ensure greater repeatability, we placed each of the 25 access points in its own RF-shielded enclosure to reduce interference and also connected access points to the test instruments using cable instead of sending signals over the air. We also used attenuators to keep signal levels in the 30- to 40-dBm range.

To measure throughput, a VeriWave script emulated 20 clients associating with one access point, authenticating using 802.1X from a RADIUS server; and obtaining IP addresses from a DHCP server. The script then offered test traffic from the Gigabit Ethernet port destined for all 20 clients. Using a binary search algorithm, the script determined the highest offered load with no more than 0.1% frame loss. (To understand why we used 0.1% acceptable loss instead of the standard 0% loss, see "Breaking the standards.") The script determined the throughput rate for 88-, 512- and 1,518-byte frames (as measured on the Ethernet side).

A complete version of the test methodology

We then repeated the throughput tests for 25 access points, each with 20 clients associated and authenticated. Again, we measured aggregate throughput for three differing frame lengths. The test duration for both the single- and 25-access point tests was 60 seconds.

We also used single- and 25-access point test-bed configurations to measure latency. For each frame length, we measured latency twice: once at the throughput rate and again at 10% of theoretical line rate.

In the call-capacity tests, the goal was to determine the maximum number of VoIP calls the system could sustain in the presence of background data traffic. We encouraged vendors to use 802.11e/WMM prioritization if supported, but this was not a mandatory requirement. We ran the call-capacity tests twice: in single- and 24-access point configurations.

In the tests of single access points, the script offered two classes of test traffic: real-time protocol (RTP) packets over 802.11b between wireless clients emulating G.711-encoded voice traffic, and 1,518-byte Ethernet frames over 802.11g emulating background traffic at a rate of 15Mbps. The test script used a binary search algorithm to determine the maximum number of calls the system under test could handle without dropping more than 5% of voice frames. The test duration was 300 seconds.

We then repeated the same tests using 24 access points and emulated wireless handsets calling between access points. In both the single- and 24-access-point cases, we measured maximum calls supported, latency and jitter for voice traffic.

To measure single-client roaming, we measured the time needed for one client to roam across all 25 access points while receiving a steady stream of data from the Ethernet side of the test bed. A script associated and authenticated one client with one access point and then offered 128-byte frames from the Ethernet to the wireless client at a rate of 1,000 frames per second.

Every 250 seconds, the client would associate to a new access point. Roaming delay is measured starting from the point where the client makes the roaming decision (such as moves away from the current access point) to when the first data packet is received from the new access point. We measured roaming time and failed roams, if any.

To measure roaming in the 25-access-point test - in what colloquially became known as the "merry-go-round of death" test - we configured the VeriWave instrument to move 250 clients across all 25 access points, repeating this cycle 50 times until all clients had roamed twice across all 25 access points.

Initially, the script brought up 500 wireless clients. Once these clients had associated and authenticated, the script then offered each client 128-byte frames from the Ethernet. Every 250 seconds, the script instructed half the wireless clients - 250 in all - to roam to the next access point. The 250 clients roamed across all 25 access points twice to show the benefit, if any, of pairwise master key caching on the second round of roaming events. We measured roaming time and failed roams, if any.

< Return to main Wi-Fi test

Learn more about this topic

Wireless Buyer's Guide

Q&A: Wi-Fi Alliance exec defends about-face


802.11T puts WLANs to the test


Wi-Fi test: How we did it 11/07/05


Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2006 IDG Communications, Inc.