How we did it

We used a combination of Spirent's WebAvalanche and WebReflector tools with Shunra's Virtual Enterprise WAN emulation tool to performance test the Symantec Mail Security 8160 appliance.

We used the WebAvalanche system to generate SMTP traffic through the Mail Security 8160 at gigabit levels. On the other end, we used the WebReflector to accept those messages and provide some statistics on actual throughput. Before installing the 8160, we verified that with gigabit interfaces, our WebAvalanche could generate about 2,300 messages a second. We developed a testing profile based on our anti-spam performance testing with message sizes ranging from 2.5K to 10K bytes, representing 90% of the traffic (including spam) received by a typical corporate e-mail system.

Next, we installed the Mail Security 8160 and ran our testing profile against the system, sending 100% non-spam messages. We gradually ramped up the messages per second level until the Mail Security 8160 hit 100% CPU utilization and until the performance as seen by the WebReflector SMTP receiver began to become erratic. That turned out to be between 860 and 900 messages a second. We picked 850 messages a second as a highly stressing load that the Mail Security 8160 seemed to be able to handle, but without a lot of breathing room.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

We used a combination of Spirent's WebAvalanche and WebReflector tools with Shunra's Virtual Enterprise WAN emulation tool to performance test the Symantec Mail Security 8160 appliance.

We used the WebAvalanche system to generate SMTP traffic through the Mail Security 8160 at gigabit levels. On the other end, we used the WebReflector to accept those messages and provide some statistics on actual throughput. Before installing the 8160, we verified that with gigabit interfaces, our WebAvalanche could generate about 2,300 messages a second. We developed a testing profile based on our anti-spam performance testing with message sizes ranging from 2.5K to 10K bytes, representing 90% of the traffic (including spam) received by a typical corporate e-mail system.

Next, we installed the Mail Security 8160 and ran our testing profile against the system, sending 100% non-spam messages. We gradually ramped up the messages per second level until the Mail Security 8160 hit 100% CPU utilization and until the performance as seen by the WebReflector SMTP receiver began to become erratic. That turned out to be between 860 and 900 messages a second. We picked 850 messages a second as a highly stressing load that the Mail Security 8160 seemed to be able to handle, but without a lot of breathing room.

To see how bandwidth was managed, we next installed the Shunra Virtual Enterprise device. All systems were connected with Gigabit Ethernet interfaces. Shunra's role was to simulate the Internet and we introduced a 30 millisec delay along with a loss rate of 1 packet in 10,000. We confirmed that the Mail Security 8160 could still handle messages at a rate of 850 per second and then moved onto our next test.

To check how well the Mail Security 8160 throttled mail, we divided the WebAvalanche senders into three groups based on what percentage of their mail was spam. One had no spam, one had 50% spam and the third had 100% spam. We pre-loaded the Mail Security 8160 by letting it see the mail and calculate its own spam percentages based on Brightmail's analysis of the spam. After we confirmed that the Mail Security 8160 had detected and classified our IP addresses, we ran the same load up against the Mail Security 8160 for a 5-minute test and watched how much mail came out the other end. For the 0% spam sender, almost all messages came out at the desired rate. A few were blocked, probably because we were pushing the Mail Security 8160 to its limits. For the 100% spam sender, we observed that the Mail Security 8160 only let through about a dozen messages during our 5-minute test. The 50% spam sender was less predictable. We tried to use bandwidth management to limit the amount of mail this sender got through. While the Mail Security 8160 did block about 60% of the messages each time, the actual bandwidth used on the Internet side was far higher than the allocated bandwidth. On the "inside" of the Mail Security 8160, the bandwidth was very well controlled.

Back to Clear Choice Test: Symantec's Mail Security 8100 Series appliance

Read more about security in Network World's Security section.