Interpreting throughput testing – read the fine print!

Ah, yes, another switch throughput performance test. And look – this vendor, too, says its product also performs at wire speed. As I thumb through the vendor’s marketing collateral, I carefully read the test methodology to gain a better understanding of the conditions that the vendor meets to achieve wire-speed performance. You should, too.

My cause for alarm is that all throughput tests are not created equal, a common misconception. Configurations for throughput tests come in all shapes and sizes: unidirectional, bidirectional, full-mesh, many-to-one port pairing and one-to-many port pairing. Within that collection of possibilities, one must take into consideration the type of system being tested. Is it a stackable or a chassis-based system? Has the test been designed to show the capabilities of the line card (in a chassis-based system), or is it demonstrating the switching fabric of the backplane or a hybrid of both?

Each test is valid – yet each one has a different meaning in the results it delivers. Take unidirectional tests. We are living in a world where everything is full-duplex (bidirectional) in nature. This is one reason why we have moved away from single collision domain hubs to switches. Consequently, unidirectional tests today have limited value.

Full-mesh, many-to-one port pairing, one-to-many port pairing . . . what do they all mean? Am I comparing apples to apples when comparing these test results? The answer is, it depends. It depends on your intended application. A port-pairing test on 10G Ethernet interfaces is acceptable in an enterprise product where the 10G Ethernet uplink ports will be used to connect buildings. However, in a service provider-class device, a full mesh of 10G Ethernet ports might be more appropriate to demonstrate the nonblocking architecture of the device.

Workgroup switches that contain mixed topologies (Fast Ethernet with Gigabit Ethernet uplinks) are more appropriately tested using a many-to-one or one-to-many port-pairing configuration. The collection of workstations (Fast Ethernet ports) will not suffer any degradation when exiting the workgroup switch on the Gigabit Ethernet uplinks in a nonoversubscribed scenario. By this I mean that the configuration is 10 Fast Ethernet ports to a single Gigabit Ethernet port and vice versa, with any remaining Fast Ethernet ports thrown into a full mesh among themselves. This type of test configuration also adequately exercises the switching fabric because every packet must hit the switching fabric to traverse the different topologies. This, of course, depends on the vendor’s switch design.

Aggregation switches can be a little tricky. These switches tend to be chassis-based, where one needs to examine the specific needs of the speeds and feeds. We need to look for intramodule performance (where switching happens within a single line card) and intermodule tests that exercise the backplane. So, it is not unreasonable to see a configuration of an eight-port line card, two of which are used for intermodule tests and six others for intramodule full-mesh tests.

Like workgroup switches, look for core switches tested in a full-mesh configuration that fully exercise the switch fabric to reveal a nonblocking total capacity.

In all switch tests, there is value to be had. Read the marketing collateral fine print and seek out the exact conditions that allow for the vendors wire-speed claims. Only then will you really understand if the product truly fits your needs or whether the marketing collateral contains more marketing spin than reliable product performance data.

Tolly is a senior engineer with The Tolly Group, a strategic consulting and independent testing company in Manasquan, N.J. He can be reached at btolly@tolly.com.