How we did it

By John Bass
Network World, 08/06/01

Our test bed consists of 13 clients with a minimum configuration of two 400-MHz Pentium II processors with 128M bytes of RAM. Each client has one 100M bit/sec Ethernet network interface card for connection to a Cisco 2948G Ethernet switch. The server under test is connected to a Gigabit Ethernet Port of the switch.

We used Quest's Benchmark Factory software to coordinate the test development, client load, result gathering and archiving for all the tests. We ran a series of file, network and database tests against Windows NT 4.0, Windows 2000 and Novell NetWare 5.1.

Using the Benchmark Factory software, we defined several tests to look at the performance of the file subsystem.

For the small file transfer tests, we used a three-dimensional test matrix of transfer direction (read/write), block size (1K/8K byte), and transaction type (random/_sequential). This test matrix resulted in eight tests. We separated all combinations into individual tests to see how each server would react in each situation. The small file transfer tests used a file size mix of 80% 1K-byte file, 10% 10K-byte files and 10% 50K-byte files.

For the large file transfer tests, we combined the reads and writes together in the same tests. We then created a set of tests that covered all combinations of the transfer type (random/sequential), and block size (1K/8K byte). This resulted in four tests. The reads and writes were combined in order to emulate large file service behavior for services such as FTP and home space services. The reads and writes were distributed as 90% reads and 10% writes. For each of the large file transfer tests, the file size distribution was 80% 500K-byte files and 20% 1M-byte files. Ninety percent of each of the file sizes were reads and 10% were writes.

Each of the files needed for each virtual user was created at the beginning of each test. Each test ran five iterations of increasing load. The number of virtual users started on each client controlled the load. The number of virtual users for each step was determined by running each of the tests against each network operating system to find where the knee of the performance curve lay. From there, we determined a standard set of load parameters to run the tests.

The CPU database test used Microsoft SQL Server 7.0 with NT and Win 2000. We increased the number of _virtual users from two to 30. The number of virtual users in no way implies a limitation the database server. Each virtual user in our test is atypical of a 'real world' user in that the load generated by these virtual users is much larger. Each virtual user calls a computationally heavy SQL statement, thus reducing the number of network transactions per unit time, to concentrate the load on the processors. The virtual users then waited 9 seconds before executing the SQL statement again.

We used another database test employing a much less intensive transaction to test the network I/O performance of the server. This was achieved by generating a large amount of transactions against the server. Like the CPU database test, the number of virtual users is increased from two to 30 to obtain a performance curve.

To test this server's performance in a NetWare environment, we ran similar CPU and network tests against an Oracle8i database running on NetWare 5.1.