Dell, LeftHand, NetApp score highest in performance testing

There is no messier can of worms to open than one containing a disk I/O performance benchmark. Performance is difficult to measure, because there are few trustworthy tools. Performance is also difficult to characterize, because every application (and even versions of the same application) uses the file system differently. And small configuration changes within either the test tool or the iSCSI server can lead to substantial changes in performance.

Rather than try and identify the fastest iSCSI subsystem, we focused on four sets of performance-related questions:

• First, how does iSCSI compare to locally attached disk storage? Is iSCSI fast enough to replace internal disks, or does the network act as a bottleneck?

• Second, how does iSCSI over Gigabit Ethernet perform? Is there a requirement for multiple connections to each iSCSI initiator? Is Ethernet a bottleneck? Does iSCSI need to be shelved until 10G Ethernet is widely available?

• Third, is there reason to pay extra money for Serial-Attached SCSI (SAS) drives compared with Serial ATA (SATA) drives?

• Finally, are there some general observations we can make about which iSCSI servers are faster than others?

To get a baseline, we ran basic benchmarks on locally connected SCSI drives. Each network and application server in our test bed had two internal 10K RPM SCSI drives, and we ran the same benchmarks across four servers with local drives that we would later run across the LAN to the iSCSI servers under test (see How we tested).

Results show that a single server talking to local disks would generally see lower levels of performance than the same single server running the same benchmark to an iSCSI SAN device. If you simply replaced internal disks on a single server with a iSCSI SAN server, even the slowest device tested would be faster than the local disks.

Click to see: Tracking performance of iSCSI SAN servers with SAS drives

Of course, these iSCSI servers are designed to serve many servers at once, and you wouldn't buy a whole iSCSI server for a single application server. Because our test bed had four servers, we ran the same test with four servers hitting the iSCSI array at once. In that case, the sum of all eight local drives in all four servers turned in a more respectable performance when compared with iSCSI over Gigabit Ethernet, because the local disk performance scaled linearly. When four servers with local drives were compared with the iSCSI servers we tested, they placed in the bottom third of our results: nine of the iSCSI SAN servers were still faster, but five were slower.

Click to see: Tracking performance of iSCSI SAN servers with SATA drives

It's impossible to guess exactly what the limits of the iSCSI SANs are without having a lot more servers (and time) to test, but a simple linear extrapolation indicates that for the test loads we used, you'd have to have about 15 servers, all running flat out with disk I/O, before the top three performing iSCSI SANs we tested (from Dell, LeftHand Networks, and NetApp) would be slower than locally attached disks. And, at that point, you'd have spent about $45,000 on disks and RAID controllers for the 15 servers, compared with $55,000 to $96,000 for one of the iSCSI servers we tested. That doesn't necessarily make the iSCSI SAN servers we tested a bargain, but the costs are not too far off — especially if you factor in the other value prospects these virtualized storage systems bring.