First Look: NetBeez net management tool creates a buzz

In our testing, we found NetBeez to be an easy-to-use product for network managers primarily interested in early fault detection and quick troubleshooting of complex wide area networks. While NetBeez does little to provide application performance analysis, it covers the network layers very well.

Managers looking for long-term performance analysis, service-level agreement reporting, and capacity planning tools will not be as satisfied with NetBeez, due to weak reporting and performance visualization capabilities. Where the focus is on identifying and solving problems, NetBeez hits the mark with economy, simplicity and ease of deployment.

The NetBeez system has two main components: agents and a dashboard. Agents are small, inexpensive appliances that perform nearly continuous testing on your network infrastructure. Agents play the role of clients, and they run network tests to your existing servers. There are three main form factors for agents: small hardware appliances (about 2-inch x 3.5-inch x 1-inch) with either wired or wireless connections, a virtual machine for you to run on your own virtualization infrastructure, or a cloud-based appliance running on NetBeez’ infrastructure for answering the question, “How’s our network look out there?”

All of the agents are in constant communication with the NetBeez dashboard, a central server responsible for collecting performance information, tracking history, and generating alerts and reports. The dashboard can either be deployed on-premises as a virtual appliance, or as a cloud-based instance. We used the cloud-based instance NetBeez set up for us.

Setup couldn’t be made much easier. We plugged the agents into various network segments, and by the time we logged into the dashboard that had been provisioned for us, the agents were already visible. We gave them meaningful names, hard-coded a few interface settings—the agents start with DHCP configuration—and we were ready to start defining tests.

NetBeez agents have a network-centric view of things, with the ability to perform four infrastructure tests: ping, DNS lookups, HTTP (and HTTPS) connects, and traceroute. To define a test, you start by creating a target. For example, a target might be as fine-grained as a single multi-protocol server, or as broad as an entire data center. Picking your targets can take some trial-and-error, because you’ll have to get some experience with the reporting and alerting that comes out of each target to know how to proceed. But the web-based GUI is so easy to use that you can try different approaches without wasting a lot of time.

Once you’ve defined your targets, you match up agents, and the agents start testing the targets, running whatever tests you want, as often as you want. We installed agent appliances on two private networks in U.S. and Italy, and a third cloud-based agent. Then, we defined four targets to test with multiple test types for each. When we were done, we had 12 specific target tests and three agents testing every target, giving us a total of 36 tests running around the clock, all reporting back to the NetBeez dashboard.

NetBeez tests the network from the point of view of the end user, with agents in every branch office. It’s the opposite approach from traditional tools such as Nagios, WhatsUp or SolarWinds. Instead of seeing the network from the data center point of view, you see it from the user point of view—a huge difference.

All those tests can generate some bandwidth, so we pulled statistics on one of our agents to see just how much. With a mix of 12 tests running, the agent ran at an average speed of just over 2kbps, with peaks during certain tests (such as hitting the home page on a web server that generated 35KB every time). If you’re concerned about bandwidth, you can slow down the frequency of tests, but we found that the overhead on modern network links seemed reasonable for the value NetBeez brings.

The NetBeez agents also have an Iperf client built-in, useful for doing brief stress tests of network paths using either TCP or UDP. Iperf is a familiar tool to network engineers, and it’s a welcome addition to the NetBeez agent. The agent does have an available Iperf server, but unless you've got the right set of firewall rules and Netbeez agents running in the right places, you may need to supply an Iperf server in your target data centers. Iperf tests are also not part of the regularly scheduled tests, so they are only run on-demand. It’s a nice debugging tool, but it would have been nicer if we could have scheduled regular Iperf tests to be run for link testing and reporting. (NetBeez told us that adding this capability is in their plans.)

Results and reporting

The NetBeez dashboard is where you see the results of your testing. The dashboard makes it easy to pivot through the current status of your NetBeez agents. Using this information, it’s equally easy to help identify problems and then zero in on potential operational issues: Is it the network? The data center? The remote site? The Internet?

If you’re debugging a network problem, you’ll find that the dashboard is nicely designed. Information is summarized and color-coded. You can look from the point of view of the agents, which helps to quickly show remote site network problems, simply by looking for blocks of yellow and red. If that doesn’t give you a quick answer, you can instantly pivot to the point of view of the targets, again scanning for anything that isn’t in green.

[ ALSO: NetBeez is busy as a bee monitoring networks from an end user perspective ]

You can also pivot down to the type of test (ping, DNS, HTTP, or traceroute), which provides a nice matrix showing agents and targets and the omnipresent color-coded blocks indicating test status.

From each test, you can quickly drill down to the statistics behind each test. Network engineers accustomed to tools such as MRTG that graph performance over time spans from one hour to one year will not be happy with NetBeez, because you can only look at a graph one day at a time. Thus, issues such as gradually changing performance statistics are impossible to see. We found the graphs to be frustrating, and in need of work.

This highlighted the strength and weakness of NetBeez. For a particular instantaneous problem, the dashboard is great, and we used it multiple times to help identify problem sources during our extended testing period. If you’ve got a problem report and want to check the network status, even on a very large network, this tool lets you see the results of hundreds or even thousands of tests very quickly and visualize where problems might be. It’s a very well-done system.

Trying to use NetBeez for longer-term analysis and monitoring is hard or impossible in this version. With no way to include non-NetBeez data (such as SNMP performance information from remote sites), the focus is very much on what can be measured with the most basic tests in networking: ping, traceroute, and simple-minded analysis of DNS and HTTP servers. Everything above the network is out-of-scope of the product.

We found reporting -- which can only be done in an ad-hoc way, not scheduled -- to be weak, with no way to export reports or statistics. Although NetBeez has all the information you’d need for both capacity planning and SLA reporting, there’s no way to get this information out of the system, which is a definite weakness. Most network managers need to do more than diagnose problems, but we found NetBeez to be so single-mindedly focused on problem solving that it misses other easy-to-satisfy requirements of a typical enterprise.

However, because the brains of NetBeez are really in the dashboard, it’s easy to improve things.  NetBeez is a small company.  The founders, Panayiotis Neophytou and Stefano Gridelli, were both quick to provide technical support and take our feedback. During our six months of testing, we were shown beta versions and went through upgrades, all of which show movement in the right direction with reporting. Network managers willing to be early adopters of NetBeez will be in a good position to influence future versions, especially in a product undergoing rapid revision.

Alerting

The obvious next step is receiving alerts from the NetBeez system when there are problems with the configured tests. We found the NetBeez feature set here adequate, but not as reliable as the instantaneous status information of the dashboard.

For each of the test targets (such as a web or DNS server), you can define triggers that will generate alerts. Then, if you want, you can define an email address to get the alert, receive it via SNMP (we did not test this), or you can just view it in the dashboard. A network operations center with a set of screens showing system status would probably use the dashboard approach, while smaller or more distributed enterprises could use SMTP alerts.

The spectrum of alert types is ambitious and well done. There are three types: outages, baselines, and thresholds. Outages are easy to understand: if there are more than a certain number of failed tests, the alert goes off. Thresholds are also easy. For example, if the time for a DNS query to respond is above a certain time value for 15 minutes, or if the number of failed HTTP connections is higher than a certain percentage for an hour, then an alert will go off.

The third type of alert is more interesting, because it can be used to reflect changes in the network by comparing current performance to baselines. For example, rather than trying to define a particular threshold for a ping time -- which will vary for each and every office in your network -- you can use a baseline. In NetBeez, you might say, “If the ping time is greater than three times the daily average for 15 minutes, then alert.” This type of alert has the potential to save a lot of time in defining management metrics for large networks because you don’t have to know particulars for every single site; you can simply alert when there are issues against the normal baseline.

+ ALSO: Review: PathSolutions solves the network monitoring maze +

We found the baseline alerts to be difficult to use because of the way that NetBeez handles statistics internally. In our initial configuration, we set up baseline testing such as “HTTP more than twice the daily average for 15 minutes.” What we discovered was that we got a lot of false alarms because NetBeez was letting outlier values skew their results. To make the alerts useful, we had to back everything off, such as only alerting when a statistic went to three times the daily average for a period of an hour.

Overall, we found the alerting to be weaker than we expected. Some of this is from the alerts being too sensitive, and text of the alerts being too ambiguous -- a problem that NetBeez worked on during our testing. But the bigger problem was the lack of context from an alert. Although everything comes to you in an HTML mail message, including the nice color-coded status page, you don’t get a performance graph. You don’t get clickable links. In some cases, the information provided was enough to determine what was going on, but not always. Many alerts had to be investigated manually, and the NetBeez alerting system wasn’t giving us information it already had which would have helped to quickly dismiss or diagnose alarms.

Alerting, statistics, and the links to email are all places where NetBeez is obviously going in the right direction, beyond the capabilities of older network monitoring tools. But there’s definitely room for improvement.

Snyder, a Network World Test Alliance partner, is a senior partner at Opus One in Tucson, Ariz.