King of the NOS hill
NetWare holds the performance reins, but Windows 2000 reigns supreme for features overall.
It all boils down to what you're looking for in a network operating system (NOS).
Do you want it lean and flexible so you can install it any way you please? Perhaps administration bells and management whistles are what you need so you can deploy several hundred servers. Or maybe you want an operating system that's robust enough so that you sleep like a baby at night?
The good news is that there is a NOS waiting just for you. After the rash of recent software revisions, we took an in-depth look at four of the major NOSes on the market: Microsoft's Windows 2000 Advanced Server, Novell's NetWare 5.1, Red Hat Software's Linux 6.1 and The Santa Cruz Operation's (SCO) UnixWare 7.1.1. Sun declined our invitation to submit Solaris because the company says it's working on a new version.
Which NOS is best?
Argue it out with the vendors in our Face-off.
Scorecard and NetResults
Find out the tuning parameters
The config files
Ganymede Chariot test files: Short | Long
Detailed performance numbers
A NOS-only search engine.
Microsoft's Windows 2000 edges out NetWare for the Network World Blue Ribbon Award. Windows 2000 tops the field with its management interface, server monitoring tools, storage management facilities and security measures.
However, if it's performance you're after, no product came close to Novell's NetWare 5.1's numbers in our exhaustive file service and network benchmarks. With its lightning-fast engine and Novell's directory-based administration, NetWare offers a great base for an enterprise network.
We found the latest release of Red Hat's commercial Linux bundle led the list for flexibility because its modular design lets you pare down the operating system to suit the task at hand. Additionally, you can create scripts out of multiple Linux commands to automate tasks across a distributed environment.
While SCO's UnixWare seemed to lag behind the pack in terms of file service performance and NOS-based administration features, its scalability features make it a strong candidate for running enterprise applications.
The numbers are inRegardless of the job you saddle your server with, it has to perform well at reading and writing files and sending them across the network. We designed two benchmark suites to measure each NOS in these two categories. To reflect the real world, our benchmark tests consider a wide range of server conditions (see "how we did it).
NetWare was the hands-down leader in our performance benchmarking, taking first place in two-thirds of the file tests and earning top billing in the network tests.
Red Hat Linux followed NetWare in file performance overall and even outpaced the leader in file tests where the read/write loads were small. However, Linux did not perform well handling large loads - those tests in which there were more than 100 users. Under heavier user loads, Linux had a tendency to stop servicing file requests for a short period and then start up again.
Windows 2000 demonstrated poor write performance across all our file tests. In fact, we found that its write performance was about 10% of its read performance. After consulting with both Microsoft and Client/Server Solutions, the author of the Benchmark Factory testing tool we used, we determined that the poor write performance could be due to two factors. One, which we were unable to verify, might be a possible performance problem with the SCSI driver for the hardware we used.
More significant, though, was an issue with our test software. Benchmark Factory sends a write-through flag in each of its write requests that is supposed to cause the server to update cache, if appropriate, and then force a write to disk. When the write to disk occurs, the write call is released and the next request can be sent.
At first glance, it appeared as if Windows 2000 was the only operating system to honor this write-through flag because its write performance was so poor. Therefore, we ran a second round of write tests with the flag turned off.
With the flag turned off, NetWare's write performance increased by 30%. This test proved that Novell does indeed honor the write through flag and will write to disk for each write request when that flag is set. But when the write through flag is disabled, NetWare writes to disk in a more efficient manner by batching together contiguous blocks of data on the cache and writing all those blocks to disk at once.
Likewise, Red Hat Linux's performance increased by 10% to 15% when the write through flag was turned off. When we examined the Samba file system code, we found that it too honors the write through flag. The Samba code then finds an optimum time during the read/write sequence to write to disk.
This second round of file testing proves that Windows 2000 is dependent on its file system cache to optimize write performance. The results of the testing with the write through flag off were much higher - as much as 20 times faster. However, Windows 2000 still fell behind both NetWare and RedHat Linux in the file write tests when the write through flag was off.
SCO honors the write through flag by default, since its journaling file system is constructed to maximize data integrity by writing to disk for all write requests. The results in the write tests with the write through flag on were very similar to the test results with the write through flag turned off.
For the network benchmark, we developed two tests. Our long TCP transaction test measured the bandwidth each server can sustain, while our short TCP transaction test measured each server's ability to handle large numbers of network sessions with small file transactions.
Despite a poor showing in the file benchmark, Windows 2000 came out on top in the long TCP transaction test. Both Windows 2000 and UnixWare have multithreaded IP stacks which allow those NOSes handle network requests with multiple processors. Novell and Red Hat say they are working on integrating this capability into their products.
NetWare and UnixWare also registered strong long TCP test results, coming in second and third, respectively.
In the short TCP transaction test, NetWare came out the clear winner. Linux earned second place in spite of its lack of support for abortive TCP closes, a method by which an operating system can quickly tear down TCP connections. Our testing software, Ganymede Software's Chariot, uses abortive closes in its TCP tests.
Moving into managementAs enterprise networks grow to require more servers and support more end users, NOS management tools become crucial elements in keeping networks under control. We looked at the management interfaces of each product and drilled down into how each handled server monitoring, client administration, file and print management, and storage management.
We found Windows 2000 and NetWare provide equally useful management interfaces.
Microsoft Management Console (MMC) is the glue that holds most of the Windows 2000 management functionality together. This configurable graphical user interface (GUI) lets you snap in Microsoft and third-party applets that customize its functionality. It's a two-paned interface, much like Windows Explorer, with a nested list on the left and selection details on the right. The console is easy to use and lets you configure many local server elements, including users, disks, and system settings such as time and date.
MMC also lets you implement management policies for groups of users and computers using Active Directory, Microsoft's new directory service. From the Active Directory management tool inside MMC, you can configure users and change policies.
The network configuration tools are found in a separate application that opens when you click on the Network Places icon on the desktop. Each network interface is listed inside this window. You can add and change protocols and configure, enable and disable interfaces from here without rebooting.
NetWare offers several interfaces for server configuration and management. These tools offer duplicate functionality, but each is useful depending from where you are trying to manage the system. The System Console offers a number of tools for server configuration. One of the most useful is NWConfig, which lets you change start-up files, install system modules and configure the storage subsystem. NWConfig is simple, intuitive and predictable.
ConsoleOne is a Java-based interface with a few graphical tools for managing and configuring NetWare. Third-party administration tools can plug into ConsoleOne and let you manage multiple services. We think ConsoleOne's interface is a bit unsophisticated, but it works well enough for those who must have a Windows- based manager.
Novell also offers a Web-accessible management application called NetWare Management Portal, which lets you manage NetWare servers remotely from a browser, and NWAdmin32, a relatively simple client-side tool for administering Novell Directory Services (NDS) from a Windows 95, 98 or NT client.
Red Hat's overall systems management interface is called LinuxConf and can run as a graphical or text-based application. The graphical interface, which resembles that of MMC, works well but has some layout issues that make it difficult to use at times. For example, when you run a setup application that takes up a lot of the screen, the system resizes the application larger than the desktop size.
Still, you can manage pretty much anything on the server from LinuxConf, and you can use it locally or remotely over the Web or via telnet. You can configure system parameters such as network addresses; file system settings and user accounts; and set up add-on services such as Samba - which is a service that lets Windows clients get to files residing on a Linux server - and FTP and Web servers. You can apply changes without rebooting the system.
Overall, Red Hat's interface is useful and the underlying tools are powerful and flexible, but LinuxConf lacks the polish of the other vendors' tools.
SCO Admin is a GUI-based front end for about 50 SCO UnixWare configuration and management tools in one window. When you click on a tool, it brings up the application to manage that item in a separate window.
Some of SCO's tools are GUI-based while others are text-based. The server required a reboot to apply many of the changes. On the plus side, you can manage multiple UnixWare servers from SCOAdmin.
SCO also offers a useful Java-based remote administration tool called WebTop that works from your browser.
An eye on the servers and clientsOne important administration task is monitoring the server itself. Microsoft leads the pack in how well you can keep an eye on your server's internals.
The Windows 2000 System Monitor lets you view a real-time, running graph of system operations, such as CPU and network utilization, and memory and disk usage. We used these tools extensively to determine the effect of our benchmark tests on the operating system. Another tool called Network Monitor has a basic network packet analyzer that lets you see the types of packets coming into the server. Together, these Microsoft utilities can be used to compare performance and capacity across multiple Windows 2000 servers.
NetWare's Monitor utility displays processor utilization, memory usage and buffer utilization on a local server. If you know what to look for, it can be a powerful tool for diagnosing bottlenecks in the system. Learning the meaning of each of the monitored parameters is a bit of a challenge, though.
If you want to look at performance statistics across multiple servers, you can tap into Novell's Web Management Portal.
Red Hat offers the standard Linux command-line tools for monitoring the server, such as iostat and vmstat.
As with any Unix operating system, you can write scripts to automate these tools across Linux servers. However, these tools are typically cryptic and require a high level of proficiency to use effectively. A suite of graphical monitoring tools would be a great addition to Red Hat's Linux distribution.
UnixWare also offers a number of monitoring tools. System Monitor is UnixWare's simple but limited GUI for monitoring processor and memory utilization. The sar and rtpm command-line tools together list real-time system utilization of buffer, CPUs and disks. Together, these tools give you a good overall idea of the load on the server.
Client administrationAlong with managing the server, you must manage its users. It's no surprise that the two NOSes that ship with an integrated directory service topped the field in client administration tools.
We were able to configure user permissions via Microsoft's Active Directory and the directory administration tool in MMC. You can group users and computers into organizational units and apply policies to them.
You can manage Novell's NDS and NetWare clients with ConsoleOne, NWAdmin or NetWare Management Portal. Each can create users, manage file space, and set permissions and rights. Additionally, NetWare ships with a five-user version of Novell's ZENworks tool, which offers desktop administration services such as hardware and software inventory, software distribution and remote control services.
Red Hat Linux doesn't offer much in the way of client administration features. You must control local users through Unix permission configuration mechanisms.
UnixWare is similar to Red Hat Linux in terms of client administration, but SCO provides some Windows binaries on the server to remotely set file and directory permissions from a Windows client, as well as create and change users and their settings. SCO and Red Hat offer support for the Unix-based Network Information Service (NIS). NIS is a store for network information like logon names, passwords and home directories. This integration helps with client administration.
Handling the staples: File and printA NOS is nothing without the ability to share file storage and printers. Novell and Microsoft collected top honors in these areas.
You can easily add and maintain printers in Windows 2000 using the print administration wizard, and you can add file shares using Active Directory management tools. Windows 2000 also offers Distributed File Services, which let you combine files on more than one server into a single share.
Novell Distributed Print Services (NDPS) let you quickly incorporate printers into the network. When NDPS senses a new printer on the network, it defines a Printer Agent that runs on the printer and communicates with NDS. You then use NDS to define the policies for the new printer.
You define NetWare file services by creating and then mounting a disk volume, which also manages volume policies.
Red Hat includes Linux's printtool utility for setting up server-connected and networks printers. You can also use this GUI to create printcap entries to define printer access.
Linux has a set of command-line file system configuration tools for mounting and unmounting partitions. Samba ships with the product and provides some integration for Windows clients. You can configure Samba through a rudimentary Web-based configuration tool called swat.
UnixWare provides a flexible GUI-based printer setup tool called Printer SetUp Manager. For file and volume management, SCO offers a tool called VisionFS for interoperability with Windows clients. We used VisionFS to allow our NT clients to access the UnixWare server. This service was easy to configure and use.
Storage managementWindows 2000 provides the best tools for storage management. Its graphical Manage Disks tool for local disk configuration includes software RAID management; you can dynamically add disks to a volume set without having to reboot the system. Additionally, a signature is written to each of the disks in an array so that they can be moved to another 2000 server without having to configure the volume on the new server. The new server recognizes the drives as members of a RAID set and adds the volume to the file system dynamically.
NetWare's volume management tool, NWConfig, is easy to use, but it can be a little confusing to set up a RAID volume. Once we knew what we were doing, we had no problems formatting drives and creating a RAID volume. The tool looks a little primitive, but we give it high marks for functionality and ease of use.
Red Hat Linux offers no graphical RAID configuration tools, but its command line tools made RAID configuration easy.
To configure disks on the UnixWare server, we used the Veritas Volume Manager graphical disk and volume administration tool that ships with UnixWare. We had some problems initially getting the tool to recognize the drives so they could be formatted. We managed to work around the disk configuration problem using an assortment of command line tools, after which Volume Manager worked well.
While we did not probe these NOSes extensively to expose any security weaknesses, we did look at what they offered in security features.
Microsoft has made significant strides with Windows 2000 security. Windows 2000 supports Kerberos public key certificates as its primary authentication mechanism within a domain, and allows additional authentication with smart cards. Microsoft provides a Security Configuration Tool that integrates with MMC for easy management of security objects in the Active Directory Services system, and a new Encrypting File System that lets you designate volumes on which files are automatically stored using encryption.
Novell added support for a public-key infrastructure into NetWare 5 using a public certificate schema developed by RSA Security that lets you tap into NDS to generate certificates.
Red Hat offers a basic Kerberos authentication mechanism. With Red Hat Linux, as with most Unix operating systems, the network services can be individually controlled to increase security. Red Hat offers Pluggable Authentication Modules as a way of allowing you to set authentication policies across programs running on the server. Passwords are protected with a shadow file. Red Hat also bundles firewall and VPN services.
UnixWare has a set of security tools called Security Manager that lets you set up varying degrees of intrusion protection across your network services, from no restriction to turning all network services off. It's a good management time saver, though you could manually modify the services to achieve the same result.
Stability and fault toleranceThe most feature-rich NOS is of little value if it can't keep a server up and running. Windows 2000 offers software RAID 0, 1 and 5 configurations to provide fault tolerance for onboard disk drives, and has a built-in network load-balancing feature that allows a group of servers to look like one server and share the same network name and IP address. The group decides which server will service each request. This not only distributes the network load across several servers, it also provides fault tolerance in case a server goes down. On a lesser scale, you can use Microsoft's Failover Clustering to provide basic failover services between two servers.
As with NT 4.0, Windows 2000 provides memory protection, which means that each process runs in its own segment.
There are also backup and restore capabilities bundled with Windows 2000.
Novell has an add-on product for NetWare called Novell Cluster Services that allows you to cluster as many as eight servers, all managed from one location using ConsoleOne, NetWare Management Portal or NWAdmin32. But Novell presently offers no clustering products to provide load balancing for applications or file services. NetWare has an elaborate memory protection scheme to segregate the memory used for the kernel and applications, and a Storage Management Services module to provide a highly flexible backup and restore facility. Backups can be all-inclusive, cover parts of a volume or store a differential snapshot.
Red Hat provides a load-balancing product called piranha with its Linux. This package provides TCP load balancing between servers in a cluster. There is no hard limit to the number of servers you can configure in a cluster. Red Hat Linux also provides software RAID support through command line tools, has memory protection capabilities and provides a rudimentary backup facility.
SCO provides an optional feature to cluster several servers in a load-balancing environment with Non-Stop Clustering for a high level of fault-tolerance. Currently, Non-Stop Clustering supports six servers in a cluster. UnixWare provides software RAID support that is managed using SCO's On-Line Data Manager feature. All the standard RAID levels are supported. Computer Associates' bundled ArcServeIT 6.6 provides backup and restore capabilities. UnixWare has memory protection capabilities.
DocumentationBecause our testing was conducted before Windows 2000's general availability ship date, we were not able to evaluate its hard-copy documentation. The online documentation provided on a CD is extensive, useful and well-organized, although a Web interface would be much easier to use if it gave more than a couple of sentences at a time for a particular help topic.
NetWare 5 comes with two manuals: a detailed manual for installing and configuring the NOS with good explanations of concepts and features along with an overview of how to configure them, and a small spiral-bound booklet of quick start cards. Novell's online documentation is very helpful.
Red Hat Linux comes with three manuals - an installation guide, a getting started guide and a reference manual - all of which are easy to follow.
Despite being the most difficult product to install, UnixWare offers the best documentation. It comes with two manuals: a system handbook and a getting started guide. The system handbook is a reference for conducting the installation of the operating system. It does a good job of reflecting this painful experience. The getting started guide is well-written and well-organized. It covers many of the tools needed to configure and maintain the operating system. SCO's online documentation looks nice and is easy to follow.
Wrapping upThe bottom line is that these NOSes offer a wide range of characteristics and provide enterprise customers with a great deal of choice regarding how each can be used in any given corporate network.
If you want a good, general purpose NOS that can deliver enterprise-class services with all the bells and whistles imaginable, then Windows 2000 is the strongest contender. However, for high performance, enterprise file and print services, our tests show that Novell leads the pack. If you're willing to pay a higher price for scalability and reliability, SCO UnixWare would be a safe bet. But if you need an inexpensive alternative that will give you bare-bones network services with decent performance, Red Hat Linux can certainly fit the bill.
The choice is yours.
Which NOS is best?
Argue it out with the vendors in our Face-off.
How we did it
Details of the testing.
Scorecard and NetResults
Find out the exact ratings and pros and cons.