In 3-stream vs 2-stream tests, 7 MIMO-based access points deliver impressive performance increases
The rapid adoption of 802.11n has become a significant milestone in the history of wireless LANs. The MIMO-based technologies used in most 802.11n systems provide enough throughput, reliability, and rate vs. range performance to effectively remove the last major barriers to the broad adoption of WLANs in the enterprise.
But there is a broad range of options specified in 802.11n, and, consequently, many products with highly varying performance are on the market. A given 802.11n product is usually categorized by the number of spatial streams, with nominally 150Mbps of throughput possible per stream, assuming a 40 MHz. channel and a short guard interval.
Today, 600Mbps, via four streams, is the upper bound of the 802.11n standard, with two-stream implementations at nominally 300Mbps the effective norm. But we’re starting to see a significant number of three-stream access points promising up to 450Mbps coming into the market.
We’d heard, anecdotally, about a 10% to 15% effective performance increase in three-stream products, which, while not bad, wouldn’t set the world on fire. For this test we set out to do a level-playing-field analysis of three-stream products, using three-stream access points from seven different vendors and a two-stream access point as a baseline. Both enterprise-class and consumer/SMB-class products were tested in order to explore a broad range of options.
Is your 802.11n installation already obsolete?
The seven, three-stream access points are: Cisco Linksys E4200, D-Link DIR-665, Belkin N750, Netgear WNDR4500, Meraki MR24, Xirrus XR4830 and Cisco 3600. Our baseline AP was a Netgear WNDR 3800.
Of course, 450Mbps for a three-stream AP represents a maximum over-the-air PHY signaling rate, or, in other words, the speed that the marketing department of any given wireless LAN vendor claims that their product will deliver. Still, one might be led to expect a significant performance increase for three-stream over two-stream implementations, especially using a three-stream client (although a nominal increase with two-stream clients could also be expected) and all other conditions being equal. (See A note on nomenclature.)
There’s also an appeal on the economic front: given that prices for three-stream access points are often about the same as those of two-stream products, price/performance should be noticeably improved. And, finally, demand for three-stream products should be significant, given growing requirements for capacity from an onslaught of bandwidth-hungry users wielding a growing arsenal of wireless devices, from handsets and tablets to good old notebooks running ever-more bandwidth-intensive applications. (Watch a slideshow version of this story.)
Results exceeded our expectations
When we average the performance of all access points in this series of tests, we find that the improvement of three-stream over two-stream in the “near case scenario” (when the access point and client are about 4 meters apart in the same room) is an amazing 93.04%.
And in the “far case scenario” (when the distance is about 7 meters but the signal travels between the basement and second floor of our test location) the improvement is a still-remarkable 39.18% – a good deal more than we expected.
Click to see: Results of the far wireless access point test
In fact, we were very impressed by these results overall and thus would encourage anyone in the market for WLAN equipment to consider three-stream products.
Here’s a short summary of each product’s performance:
Cisco Linksys: Price $159.99
In the two-stream baseline test, the Cisco Linksys came in next to last, at 74Mbps in the near-distance test; and dead last at 62Mbps in the far-distance test. But when we turned on three-stream MIMO, the Linksys delivered 190Mbps and 125Mbps respectively, which puts it in second place overall. Not only that, the Linksys delivered improvement rates for three-stream over two-stream of 158% and 100% respectively.
D-Link: Price $99.
The D-Link DIR-665 is a low-cost consumer device that finished in the middle of the pack. In the near-distance test, D-Link improved from 91M to 123Mbps (up 35%) and in the far-distance test, D-Link went from 68M to 91Mbps (up 34%.)
Click to see: Results of near wireless access point test
Belkin: Price $99
The Belkin N750 is the other under-$100 device in our test. Belkin’s far-distance test results were at the bottom of the pack – 64Mbps for two-stream results and 72Mbps for three stream. Its near-distance results were much better – 87Mbps to 157Mbps, for an improvement of 80%.
Netgear: Price $179.99
Netgear was our top performer in terms of total throughput, delivering 223Mbps in the near-distance test, and 128Mbps in the far-distance test.
Meraki: Price $1,199
Meraki scored well in our near-field test, coming in third with three-stream throughput of 184Mbps. In the far-distance test, Meraki was in the middle of the pack, with 89Mbps, a 35% increase over Meraki’s two-stream performance.
Xirrus: Price: $6,450
The Xirrus price is a bit deceiving, since the Xirrus Array comes with eight separate radios. In terms of performance, Xirrus trailed the pack in the near-distance test, with a three-stream throughput of 115Mbps, but Xirrus performed near the top in the far-distance test with 93Mbps.
Cisco: Price: $1,495
The Cisco 3600 AP delivered 77Mbps in the near-distance, two-stream test and 179Mbps in the near-distance, three-stream test, an improvement of 133%. In the far-distance tests, Cisco went from 65M to 97Mbps, a 50% improvement.
Interestingly, the test of the two-stream baseline access point (a Netgear WNDR3800) showed a modest improvement in performance when using the three-stream client, leading to the conclusion that, just as 802.11n clients used backwards-compatibly with 802.11g infrastructure to yield a “better g than g” effect, similar “better than n” performance should result when using a three-stream client with two-stream access points.
The WNDR3800 also, remarkably, had the best two-stream near-distance and second-best two-stream far-distance performance, perhaps more than anything suggesting maturity in firmware and overall two-stream system implementation.
How we tested three-stream 802.11n access points
Testing was conducted in freespace, but in a residential environment so as to provide a greater degree of control over the airwaves by minimizing the impact of any potential interference. It’s almost impossible today to find a commercial facility free of a good deal of arbitrary Wi-Fi traffic, evidence of the status that Wi-Fi has achieved.
All testing was monitored with a spectrum analyzer (in this case, AirMagnet’s Spectrum XT and Fluke Networks’ AirCheck WiFi Tester) to assure a level spectral playing field. The server end of this test was an Ixia VeriWave WaveTest90 chassis with a WBE1604 Ethernet Board, providing a consistent and highly instrumented environment with respect to load and the recording of results.
The client software was Ixia’s VeriWave WaveInsite and WaveTest running on a Console PC connected to the back of the chassis, and the company’s WaveAgent on the Test notebook PCs equipped with wireless client adapters.
The only exceptions were in the case of Cisco, in which a Cisco Catalyst 3750-X PoE switch and 5500 WLAN controller were also used, and Meraki, which requires a connection to the Internet for control and management plane implementation. This was accomplished via reconfiguring IP addresses and connecting to the Web via a switch and our office router.
Each series of tests with each subject access point was conducted with both two- and three-stream client devices so as to fully evaluate the performance of the access point. The two-stream client was Linksys’ popular WUSB600N, a USB 2.0 device using the Ralink RT2870/2850 chipset, and the three-stream client was a TRENDnet TEW-680MB media bridge, connected to the Test PC’s gigabit Ethernet port, and based on the Ralink RT3883F chip.
While three-stream USB adapters are available, it was felt that the 450Mbps peak of the wireless device was too close to the 480Mbps limit of USB 2.0, and that gigabit Ethernet would provide more headroom just in case. All device settings were left at their defaults other than being configured to automatically connect to the test SSID.
All of the access points under test, except the Xirrus Array, were configured to use a single 40-MHz. channel (149/153) in the 5GHz band. By virtue of its need to manage eight separate radios, the Xirrus instead auto-configured channels across all of its radios. Apart from the radio channel, all that was changed from the default in all access point configurations was the IP address, SSID, and WPA2 key. The 2.4GHz radio, if present, was disabled if possible and otherwise left unconfigured, so the only variables for each test run were the particular client adapter used and the distance involved. The real variable under study, then, was the performance of the access point under test. WaveInsite was configured to generate both upstream and downstream HTTP traffic at a nominal 1Gbps rate, thus saturating the link.
Each 30-second test run was repeated three times, with the results averaged so as to factor out any otherwise-undetected anomalies, over both “Near” (both endpoints in the same room with nominally four meters of distance between them) and “Far” (straight up from the basement of the structure to the second floor, a linear distance of about seven meters but through numerous solid objects) geometries. The Near test was designed to evaluate performance that might be seen in a typical dense-access point deployment, while Far is more characteristic of a typical access point deployment. We thus had a pretty good look at the contribution of three-stream MIMO across all of the access points. Just to be sure, we also baselined the test using a direct wired gigabit Ethernet connection, which showed that gigabit speeds were indeed possible.
Your results will vary
As tempting as it may be, any use of these results to directly compare the performance of the enterprise-class access points tested here should be avoided. Note that we used only a single workload on a single client, a highly unrealistic real-world enterprise scenario, and many other considerations, including system architecture, scalability, capacity, security management features and much more preclude the application of any single figure of merit in this case.
Comparing residential-class and enterprise-class products based on these results would be similarly unrealistic and misleading given wildly diverging sets of capabilities and core missions. Again, our objective was to learn what contribution, if any, might be made by three-stream capability – and we were more than surprised by the results.
Mathias is a principal at Farpoint Group, a wireless advisory firm in Ashland, Mass. He can be reached at craig@farpointgroup.com.
