InfoWorld review: Free Java application performance monitors

Innovative tools from AppDynamics and New Relic make it easier than ever to monitor the performance of complex websites

The code is polished, the site map is frozen, and the UI is dripping with gorgeous colors. Everything is perfect -- aside from worrisome reports that some beta testers are finding that buttons are slow or even completely unresponsive. The AJAX calls that seemed so snappy in the lab were working well -- except for those inexplicable moments when they aren't. And those moments can't be duplicated.

The best way to find the bottleneck is by adding telemetry to every part of the stack from top to bottom and tracking every request from beginning to end. The tools from CA Wily and Mercury Interactive (now part of Hewlett-Packard) were the first generation of monitoring software that made it possible to keep the biggest Java sites humming along. They deployed a collection of monitoring agents in a single JAR that slyly insinuated itself into the class loader, where it automated the process of timing the most important calls. Then the data was uploaded to a monitoring server that offered a dashboard for watching everything at work.

[ Groovy and JRuby lead a strong field, with Scala, Fantom, and Jython following behind. See InfoWorld's "Top five scripting languages on the JVM." ]

These solutions are now more than a decade old, and their code is well established in many of the biggest software stacks in the business. That means the time is ripe for another round of innovation. In this round, several new companies are dashing into the arena with tools that make application monitoring simpler than ever and that offer big installations even more abilities to track data flows across machines.

For this review, I concentrated on two newer product lines with substantial free versions that are perfectly adequate for small sites and initial testing. The tools are also working advertisements for fuller versions that monitor multiple servers with different processes. While many people will be well served by the free versions, many others will find the full versions seductive enough to consider upgrades.

These products come from two companies that are relatively new, but the people involved are quite experienced. Both AppDynamics and New Relic are run by teams that include a large number of people who used to work for Wily.

AppDynamics Lite The first generation of application monitoring tools focused on the server, a perfectly natural place to begin and that works well until there are dozens or hundreds of servers. As websites grow more complicated and roll together the data from many Web services and Web 2.0 sources, it's harder and harder to parse a table of data about all of the servers working together to create their Web pages. Instead of focusing on the servers as independent entities and counting their CPU cycles, AppDynamics looks for incoming requests for information and tracks the flow of data through the server farm that completes the requests.

The free version offers a peek at everything that the full product can do. Adding the collection agent to a server is as simple as adding the right JAR file to the path and modifying the startup parameters for the JVM with a javaagent parameter. The agent takes it from there, looking for all of the significant entry points in the stack, cataloging all the requests, and watching their performance. This data is relayed to a separate agent, the AdLite Viewer, which chews up the information and delivers it through port 8990 in HTML. You watch in a window of your browser.

AppDynamics has worked hard at simplifying the hefty data that flows into the collection agent. The simplification is generally desirable but can obscure some of the lower layers. For instance, the tool watches for some of the most common architectural idioms like Struts and Spring, but it flags only the first calls to the system. If you're fielding the new requests with servlets or JSPs first, they'll be the only ones watched at the beginning. But the list of entry points discovered automatically can be edited to include anything that you think is worth tracking.

The discovery tool can be eye opening. I forgot my application was making occasional calls to load a piece of XML during configuration, but AppDynamics noticed it going by and added the external source as a server in the application. AppDynamics creates a cute graphic display of the various servers with lines illustrating the flow of data.

This self-discovery of the collection of machines that work together to deliver the information is incredibly valuable. The service emphasizes understanding how these machines work and don't work together, communicating the interactions visually with a picture of the network and the speed of the requests flowing between the machines.

The rest of the main display is given over to the current success or failure of the requests coming into the server. Fast responses get green bars, stalled responses get red bars, and pokey threads get mixed colors.

Underneath the red bars is more information about different methods and database queries that go into assembling the Web page. The time-consuming spots are flagged and easily identifiable.

The free Lite version is useful for many basic tasks, and smaller websites -- and some larger ones, I'm sure -- will be able to get by without paying. But the free tool is ultimately a brochure for the paying ones, and there are ample reasons why even a single-server site might choose to spend.

For instance, the Lite version records data from only the last two hours. If you want to know what happened last night or even while you were gone on a long lunch, you're out of luck. The full Standard version sucks down much more data about memory leaks, deadlocks, and other glitches. It also does a better job of presenting the information by grouping the servers into tiers and "learning" the behavior of the different methods and pages. For example, it will figure out that some distant call is always a bit slow and will stop flagging it with red every day.

Then there's the Cloud edition, named for AppDynamics' big target. The tool monitors instances of the application in virtual or cloud environments and helps with dynamic scaling in response to changes in demand. This will undoubtedly be where most of the development will come in the product over the next few years.

The main page for AppDynamics highlights failing requests and graphs the load and response times. Just drill down for more detail and diagnostics on slow and failed requests.

New Relic RPM Lite New Relic's telemetry begins just like AppDynamics with a JAR file tossed in the lib directory and a javaagent parameter added to the start command. The big difference is what you don't install: a monitoring server for digesting the data. This is all handled remotely on New Relic's servers. It's software as a service.

This sounded very inviting because I've wrestled with setting up a dashboard and server for Wily in the past. Letting someone else run this infrastructure was as good as it sounds. All of your account information is stuffed in the newrelic.yml file, and the statistics from your machines start appearing on your Web pages at

This simplicity seemed less exciting, though, after I started up the AppDynamics stats daemon without any trouble or confusion. The only extra thought AppDynamics required was to type the command line starting up the daemon in the right sequence with the server -- which I had to try two or three times. Everything else was pretty automated, alleviating most of the advantages of leaving the stats collection to someone else. Both of these tools are dramatically easier to start up than what everyone endured even several years ago.

I'm not sure there's much of a security hole with sending information to New Relic; most of it will be dreadfully boring and useless information about which URL responds within how many milliseconds. New Relic does encrypt the data during the trip, but when it gets there, it's still in the control of another company. And then most of this data is recordable by outside visitors to your site, and they already know full well whether your website is working.

But some of the holes can be more subtle, and code can reveal secrets by mistake. For example, a supposedly confidential password might be used in the URL of some internal Web service call that would normally stay secure inside the firewall. Some JDBC URLs, for instance, include database passwords. Or the programmer might punt on adding more security in the name of time. Will this information find its way to New Relic's servers? It may be hard to know because the person implementing the monitoring may just pop in the agent JAR and never think about the sensitive data.

Compared to AppDynamics, the style of New Relic's performance data presented by the website seems a bit more traditional. The main page tracks the average response time, the throughput, and the Apdex score, a stat that roughly measures the percentage of users who get their information quickly. The speediness of each Web page is also tallied and graphed on another table. The free version shows the last 30 minutes. As with AppDynamics, if you want more information -- both more metrics and a longer history -- you'll need to upgrade.

More profiling information can be found by starting up a separate session. This records the amount of time spent on each of the major routines and subroutines in your system. In the past, I've usually relied upon the profiler with Eclipse to gather this information, but that's not always a good indication of what part of the system is exercised most with real client data in a working environment.

Some advantages of a central server become more apparent when you dig into deeper features, such as Web integration and the API. New Relic's central server will export your incident reports to other websites such as Campfire, Lighthouse, or Pivotal Tracker. The monitoring server can start up a discussion on its own. There's also a connection with Twitter, although why anyone would want to share deployment and incident notifications with the entire world is something I'll never understand. Kids today want to get their server information mixed up with invitations to drink and reminders from Mom that the root word in "smartphone" is a hint that there's a voice feature buried among those apps.

New Relic's top-down view of the threads in the application shows how much time is devoted to each thread and the methods used inside. The Wall Clock Time is useful for debugging problems with response time for users. An alternative view shows CPU Burn Time, which is useful for diagnosing long computations.

You don't need to use any of these outside features if you want to roll your own dashboard. There's also a generous API that exports most of the basic metrics about your application so that you can integrate this with other tools. Most of the data comes in XML, but there's even a cute little HTML fragment made just for AJAX calls. It's very simple to add website stats to any internal page.

All of these touches will be welcome by some of the more far-flung development teams. If your application is hosted on some distant server and your team works out of a virtual office using Campfire and Twitter to synchronize your workload, then this may be very attractive. If you're a bank that's paranoid about letting anything but heavily scrutinized HTML through your firewall -- well, it's just another reason to worry.

The free version of New Relic is just as much a brochure as the free version of AppDynamics. New Relic offers more thinly sliced pricing plans (Bronze, Silver, Gold, and Enterprise). Upgrading brings better data and longer storage. The free version is good enough if you want to know how fast your machines have been for the last 30 minutes, but it's still a bit too hard to pinpoint the error without some guessing and constant monitoring. The commercial levels offer more and better tools for identifying the errant transactions.

New Relic or AppDynamics? The quality and simplicity of these tools show how much easier it is today to run sophisticated websites at little or no cost. Both of these free tools are excellent mechanisms for tracking the performance of basic websites. They could also be adequate for some larger, more complicated constellations of servers, but the results from big installations will be so complex that any sane person will start paying quickly.

Is one a better deal? If you're running a basic server and you just want to make sure that the database is responding relatively quickly, you can't go wrong with either one. There aren't many potential bottlenecks and both identify them quickly.

There are differences in approach, though, that can tip the scale one way or the other. Before helping Java developers, New Relic began aiming at the world of Ruby developers, who are often consultants working together on projects, often out of virtual offices. Centralizing the data analysis as a service is ideal for groups like this. There's no issue of firewalls. To sweeten the deal, New Relic throws in a number of neat features that Web 2.0 junkies will like.

1 2 Page 1
Page 1 of 2
IT Salary Survey: The results are in