How we tested the cloud services

In first-ever test of enterprise cloud services, vendors score high marks on speed, security, performance and management. Cost benefits are less clear.

We used our facility in our hosted network operations center at n|Frame in Indianapolis to connect with each competitor's cloud. We used a Vyatta router/VPN virtual machine appliance to connect with each vendor to test their skills in connecting non-Cisco equipment.

We used our facility in our hosted network operations center at n|Frame in Indianapolis to connect with each competitor's cloud. We used a Vyatta router/VPN virtual machine appliance to connect with each vendor to test their skills in connecting non-Cisco equipment. 

We connected each VPN via IPSec protocol running an AES-256 key. In each enterprise cloud, we ran a regimen of tests. Each instance was built to have LAMP/WAMP (Linux or Windows, Apache/MySQL/PhP), then we loaded an Apache 'ab' benchmark with a known outcome to see how the event registered on the provider's monitoring apps, and to gauge costs. It turns out that each vendor's test results were pretty much the same, with only a small deviation among them.

We also tested upload/download speed using scp from our n|Frame NOC through the Vyatta VPN appliance to servers inside each private cloud. We were surprised to find little difference among them.

We used several hosts to control each virtual private cloud, including an Apple Xserve, an HP 580 G5, and HP 585 G5, and a Dell 1970. Connection speed provided by n|Frame for these tests was 10Mbps. We used various HP and Apple notebooks to control the NOC resources, which in turn, connected to the virtual private cloud under test.

We examined each participant's administrative portal, and tried to use each portal to check costs. In each case, costs where reflected in a portal didn't give the overall costs — but Terremark's came closest to accuracy and offered the most information.

Return to main test.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Related:
Now read: Getting grounded in IoT