How we did it: Remote access servers

Network World, 3/23/98

Test bed

XXCAL Testing Laboratories, Inc. used 60 Dell Computer Corp. OptiPlex PCs with 166MHz Pentium processors and 32M bytes of RAM. Each PC was connected via standard serial cables to a 33.6K bit/sec modem in a US Robotics MP/16 V.34 Total Control Modem Pool. Each modem pool had two high-density modem cards with eight modems each. The modems dialed into an AT&T/Lucent Definity G3 PBX, which used three ISDN PRI lines to feed data to the remote access server (RAS) being tested.

During testing, all of the RAS units were configured to authenticate users using CHAP and auto-assign IP addresses. All clients used the same user name and password for dial-up authentication. Each PC client used the dial up networking utility in Microsoft Windows 95 OSR2 when dialing into the RAS units.

Each client executed an automated test script that entered the commands needed to perform a series of file transfers. XXCAL's automation script opened a DOS window, loaded the Windows 95 ftp.exe utility and performed uploads and downloads with a LAN server. The server used in testing was an AST Research, Inc. Manhattan D with a 200MHz Pentium Pro processor and 320M bytes of RAM running Windows NT Server 4.0 with service pack 3 and Internet Information Server 4.0. Both the server and the RAS unit being tested were connected to a Bay Networks, Inc. 350T Ethernet switch.

All 60 clients established a dial up link with the RAS unit being tested. Once all 60 clients had a connection, file transfers began with a single client. After that first client was finished, another was added to the test so that two clients were transferring files at once. Clients were added one at a time until all 60 were transferring files at once or the RAS unit started dropping calls. If the ftp program on any of the clients timed out for any reason, the client-to-server connection would be broken and the call would drop. When the call got dropped, the test script stopped running.

Test objectives

1. The test was designed to produce consistent, repeatable results.
2. The test was also designed to reflect realistic usage of the products. Any use of exotic client simulation in which one PC emulated multiple PCs or measurement tools was avoided, as they require technical interpretation of results to be meaningful.
3. TCP/IP and ftp were used because both are commonly found in enterprise networks.
4. Test results provide a meaningful measure of RAS server performance over varying client load. This shows how consistently a RAS unit will perform over all conditions.

The test scenario had the following characteristics:

• Throughput was measured at each client. This provides a measure of performance that individual end users can expect to see as opposed to how much data can be pumped into and out of the RAS unit at once.
• Transfer rates were based on statistics compiled by the Windows 95 ftp program, which does not include ftp command latency.
• The use of an automation script enabled clients to synchronize the transfer of each individual file so that multiple tests could produce similar results. In other words, once one batch of clients was done transferring one file, they synchronized with each other so that all clients started the next transfer at the same time.
• Short-form EIA TSB-38 test files were used. These industry-standard 32K byte files are used for evaluating modems. Clients perform transfers in a synchronized sequence. Starting with a single machine that downloads five files and uploads five files and ending with sixty machines that perform the same operations all at once. Machines are added to the test bed one at a time.

The first was a number-only file signature that is typical of a vector drawing file such as a CAD file. The next was a WordPerfect document. After that was a portion of an executable file for an 80x86 processor. Fourth was a 64-bit random data file that is typical of a .ZIP or other compressed file. Lastly, there was a file with mixed content, the type of file that is typically used to evaluate v.42bis compression in modems.

File transfers were summarized into two results; one for upload and one for download speeds. Transfer times for all five file types are averaged to produce a single aggregate result.

This test methodology requires RAS units to be stable - i.e. they cannot drop calls at any point during testing. Since the telephone system used constitutes an ideal environment (no line noise) this should not have been an issue. However, multiple vendors failed to complete the test script due to connections being dropped repeatedly by the RAS unit being tested.

We ran each test script three times and averaged the results. If the RAS unit kept dropping calls, we took the three best runs from all the ones we did.