Network-modeling tools

We recently tested the ability of three network modeling packages: Opnet Technologies’ IT Guru 10.5, Analytical Engines’ NetRule Version 6.0, and Shunra Software’s Shunra/ Storm Version 3.1.

Simple questions sometimes don’t have simple answers. When questions like: “Can we increase the bandwidth to our regional offices?”; “Can we provide more redundancy for our critical links?”; and “Can we do this and reduce costs at the same time?” are asked of enterprise network executives, it becomes more difficult. Keeping the complex beast of the various network sections running efficiently at maximum bandwidth and minimum cost is a monumental task.

Fortunately, network-modeling tools let designers or operators test changes to network topology before they are implemented in a production network. We recently tested the ability of three packages – Opnet Technologies’ IT Guru 10.5, Analytical Engines’ NetRule Version 6.0, and Shunra Software’s Shunra/ Storm Version 3.1 – to model changes to predict their impact on the University of California, San Francisco (UCSF) campus network. (See How we did it.)

Archive of Network World reviews

Subscribe to the Product Review newsletter

We found that IT Guru was the most-accomplished of the three products and thus it wins the Clear Choice Award . It scaled easily to accommodate just about any production enterprise network and had powerful tools for analyzing network issues. NetRule was very easy to configure and learn, providing good modeling tools, especially for LANs. Shunra/Storm did an especially fine job modeling WANs . Although each package has specific strengths, all are competent packages and share many common analysis tools and capabilities.

Modeling accuracy

The key to network modeling is the ability to closely match the generated network model map to the real network topology. We wanted to find out how accurately each product modeled events such as link failure, link change, device failure, load change, route change and link overloading.

IT Guru handled all these issues with aplomb. It let us implement changes on the fly. We could change factors such as Open Shortest Path First (OSPF) link costs, Hot Standby Router Protocol (HSRP) timers and OSPF timers, and immediately see the impact on the network. IT Guru took every configuration change we threw at it, and it accurately predicted the behavior of routing protocol and topology changes. There seemed to be nothing too complex or tricky for IT Guru.

Changes in our model did require accuracy on our part. For example, we modeled two HSRP-connected routers. We entered all the HSRP attributes in both routers, but the HSRP model refused to work. It turned out we had forgotten to set the default gateway in the router configurations. Without this information, IT Guru simply would not establish the HSRP session. Because this is exactly the same thing that occurs in a real-world network, we were impressed with this feature.

While NetRule could accurately predict the impact of many network topology changes, the product has some limitations that concerned us. For example, to test load sharing we created two links, a 1G bit/sec link and a 100M bit/sec link. We reduced the 1G bit/sec link to an OSPF cost of zero, expecting the model to show a preference for the higher-bandwidth, lower-cost link. But the NetRule model assumed an equal cost between the 1G bit/sec and 100M bit/sec links. Mystified, we called NetRule support and were informed that to change the OSPF cost we had to delete one of the existing links and then re-create it with the new cost. Analytical Engines has since indicated that it is possible to change OSPF costing on the fly, but that we were unable to verify this in our testing.

The NetRule library contains a reasonably large selection of vendors and products. However, it does allow customization of any network models, such as changing the CPU processing power and interfaces. The tool does not check for configuration errors or device inconsistency, nor does it check for IP address conflicts or protocol errors. It seemed that the model assumes that users do not introduce input errors. That is a large, and probably unreasonable, assumption.

A very nice feature of Shunra’s tool is that it can record network conditions directly and play them back. This enables real applications to run against the simulation. For example, we could send voice packets and streaming video between devices connected to Shunra/Storm’s StormAppliance. When we introduced delay, packet loss or jitter, or reduced the link bandwidth, we actually could see the video degrade or hear the audio drop out. This feature makes it extremely easy to determine the minimum acceptable criteria required for specific applications across various links. We found this a very useful feature. With IT Guru and NetRule, we could get this same information, but it was presented numerically.

Configuration and performance

With its optional Multi-Vendor Import (MVI) module, IT Guru can import configuration directly from Cisco and Juniper devices. For other vendors’ gear, Opnet requires the optional Virtual Network Environment (VNE) server. The VNE server supports network devices from Cisco Catalyst IOS, Extreme Networks, Foundry Networks, Nortel and other vendors. We were disappointed that IT Guru could not import more non-Cisco IOS devices’ configurations directly, instead of requiring optional products that cost extra.

There are multiple ways to handle the IT Guru configuration. For example, it can use the optional VNE server, import text files from CiscoWorks or create network objects manually. It also can query an HP OpenView server, which lets it build a model of the production network that is automatic and up-to-date. However, this requires modifying the OpenView server configuration to accept connections from the workstation that is running IT Guru. Our IT security team wouldn’t give us access to a production OpenView server from a test lab machine, so we couldn’t test this feature. Instead, we modeled the Cisco devices in our network using data provided by a CiscoWorks text file export, along with manually created Foundry network devices.

IT Guru imported the backbone network device configuration for 120 Cisco devices from our CiscoWorks file in less than 3 minutes. Because IT Guru is very sensitive to syntax errors, creating manual configurations for the Foundry gear was a painful and time-consuming process. To make matters worse, there was no warning or alert that flagged syntax errors. The model simply refused to work properly.

For performance, IT Guru led the pack. The product is very flexible and scalable, providing many customization features. The user interface offers several options and templates that let users drag and drop several kinds of network topologies. This creates a product that is sophisticated, powerful and complex. At times, we found ourselves overwhelmed by its rich features and complexity.

NetRule required little time and effort to configure and build a network model. The product provided a palette of Cisco-specific and generic network devices. Model creation is straightforward. Simply drag and drop devices from the NetRule library to create the network, adding the appropriate links and routing protocol information. Next, define the source and destination, and the required resources such as priority, message bytes, delay and traffic type. We felt this was as straightforward as a model configuration can be.

Unfortunately, NetRule does not interface with management packages such as OpenView and CiscoWorks. NetRule can read a CiscoWorks text file, but not as directly as IT Guru. The text file has to be in the predefined format that NetRule expects. This made the replication of our large-scale production environment a labor-intensive process. As a result, we couldn’t easily import our 120-device network topology into the product. Instead, we manually created a 12-router subset of our production network. It took us about 30 minutes to model the 12-router, full-mesh network topology.

Once the model was completed, we could create reports based on configuration changes to the devices and links. The NetRule reporting feature was very good, and provided a graphical display of network utilization, delay, traffic flow and the like.

NetRule’s performance was acceptable. It took longer than we expected for the application to launch, but once it started, NetRule seemed to operate relatively quickly in displaying the result whenever we implemented changes to the model.

Shunra’s approach to network modeling is more empirical – it literally records the network conditions directly and plays them back, while enabling applications to run against the recorded model. Because of this approach, some typical modeling parameters don’t apply to the product, such as importing network configurations from CiscoWorks or OpenView.

Configuration involved right-clicking on a link or device in Visio and making changes through a drop-down menu. We could then observe the changes on the network by watching how the applications running through StormAppliance behaved. A built-in Sniffer-like protocol-decode function was a plus. Shunra’s reporting function produces a nice real-time chart that displays throughput per second in and out, queues, delay and packet counts.

Shunra is a hardware-based, network-modeling product. That gives it somewhat of an edge in terms of performance over an entirely software-based modeling application. In most instances its speed was as good as or better than the other products in terms of implementing network-modeling changes. However, the appliance did have an annoying tendency: Whenever we changed certain model parameters, we had to warm-boot the hardware. For example, switching StormAppliance between Layer 2 and Layer 3 (switching and routing) required a reboot, as did turning on and off multicast. (It’s often preferable to change between switching and routing in the same network to observe the performance characteristics of a network model using Layer 2 switching or Layer 3 switching/routing.) Adding insult to injury, the reboot process was slow – it took more than 2 minutes.

Installation issues

Opnet’s IT Guru 10.5 is a complete product. We received four CDs that contained the Report Server module, the IT Guru application, documentation and the Opnet Model Library. The package also includes the Application Characterization Environment (ACE), Simulation Runtime, Flow Analysis, Net Doctor and the ACE Decode Module. There are numerous modules (some come with the base product, some are included depending on the specific configuration ordered, and some are optional at extra cost.) The number of modules available for IT Guru reflects the flexibility and complexity of the product. This isn’t a package that you just install and walk away from. It has a plethora of options and capabilities that need to be mastered before a user can take full advantage of it.

IT Guru installed without problem, although it took nearly 10 minutes to install the model library, which checks in at a whopping 830M bytes of hard drive space. We were surprised that the installation process required two complete reboots to make the program ready for use.

NetRule was far more compact, taking up a mere 13M bytes of space, but it doesn’t not contain the tools and modules that IT Guru has. NetRule took just more than a minute to install, with no reboots required.

The Shunra/Storm package included the StormAppliance hardware, StormCatcher (enables the capture and replay of network activity) and StormConsole. StormAppliance is responsible for emulating link conditions such as bandwidth, packet loss, delay and out-of-order packets. StormConsole (a Microsoft Visio macro) creates the network model and is used as the interface to StormAppliance. Installation of StormCatcher and StormConsole was brief. We had some problems attaching our laptop to the network ports on StormAppliance. The Dell Latitude notebooks equipped with 100M bit/sec Ethernet PC cards could not connect to StormAppliance, while another Dell laptop with an integrated Gigabit Ethernet port connected without a problem. After searching the documentation, we found that PCs must be connected to the appliance using the supplied crossover cables. We were using regular Ethernet cables. The Gigabit Ethernet-enabled laptop supported auto-sensing of cable configuration. Shunra should have pointed out the need to use the crossover cables in the installation section instead of burying this in an appendix.

Documentation

The IT Guru documentation is on a CD-ROM and online. If a customer requires printed documentation, there is an extra fee. We were surprised that a product this complex and expensive did not include professionally printed documentation.

NetRule’s printed documentation appeared to come from a color laser printer. We would have preferred a higher-quality presentation, although the necessary information was included.

Shunra provided a professionally printed user manual. The configuration procedure is nicely laid out. We were disappointed in the amount of effort it took to dig out some specific information from the documentation (for example, our crossover cable issue).

Deployment considerations

IT Guru is extremely powerful and feature-rich. Users are expected to possess both a thorough knowledge of networking and the product itself, giving it the highest learning curve of any of the products we tested.

Even after a full day’s training by Opnet, we sometimes scratched our heads when we tried to use several of the product’s advanced features. However, once we began to understand the product, it became clear that IT Guru was almost unlimited in its capabilities.

Network engineers looking for a sophisticated package that can model almost any enterprise network would find IT Guru to their liking.

NetRule doesn’t have all the bells and whistles of IT Guru, but what it lacks in muscle power it makes up for in user friendliness.

After attending a one-day training session, we could create network models without problem. This easy-to-learn package does a decent job modeling smaller-sized networks, making it ideal for modeling building or departmental LANs.

While Shunra doesn’t compare to IT Guru and NetRule for in-depth enterprise network modeling, it does show how applications and networks can be affected by bandwidth throttling, link limitations, packet delay, jitter and the like. Using Visio as the device interface was a brilliant idea.

It is the rare network engineer or designer who isn’t at least somewhat familiar with Visio. This makes it a terrific front end to Shunra/Storm, and significantly reduces the learning curve. It has excellent capabilities to simulate frame relay, T-1s and Gigabit links. This, coupled with drop-downs that allow on-the-fly changes to links, make Shunra/Storm a natural for modeling WAN connections.