By Joel M. Snyder
Network World, 12/07/98

From the moment your company intranet goes online, you'll be asked to quantify its success. Which pages are visited most? When is traffic heaviest? Are employees in remote offices taking advantage of online resources?

Web server log analyzer software can help answer these questions, but none of products we looked at from the five market leaders is best for every intranet or extranet. WebTrends' WebTrends Log Analyzer and net.Genesis' net.Analysis 3.5 provide the most useful and interesting reports right out of the box, minimizing the need to customize or write additional tools. Net.Analysis supports the broadest range of platforms, while Microsoft's Site Server 3.0 is limited to Windows NT. WebTrends and Marketwave's Hit List Commerce 4.0 process logs and deliver reports in the shortest amount of time, while Site Server and WebManage Technologies' NetIntellect 3.0.2 take far longer.

The primary functions of all five products are almost identical, because there's only so much you can do with a Web server log file. All have a built-in scheduler, which handles basic tasks such as retrieving log files, running reports, and saving, uploading and mailing reports.

In addition, all of the programs allow you to download logs during off-peak hours and schedule nightly, weekly and monthly report runs. And all of the log analyzers provide standard, built-in reports; however, some report sets are more comprehensive than others.

We tested all the products on Windows NT; running the schedulers as NT services makes the process fairly painless. Hit List, WebTrends and NetIntellect support Windows 95 and NT; net.Analysis adds Solaris and AIX support as well. Site Server runs only on NT, but has a number of additional Web management features, such as content analysis, built into the base product (NW, May 25, page 1).

Where the Web server log analyzers differ significantly is in performance, number and type of standard reports produced, and the ability to handle sizable log-viewing audiences.

The slowest analyzers took hours longer than the fastest products to process data. If your log files are larger than 50M or 100M bytes per day, you should make performance a top priority or you may not be able to process the data in a reasonable amount of time.

Most analyzers simply generate reports and deliver them somewhere. If you're producing reports for a small audience, that's all that really matters. But if you plan to distribute 50 different reports to 50 different departments, you need more power. Net.Analysis, and to a lesser extent WebTrends and Hit List, build a separate subsystem to manage such high-volume reporting.

Looking at logs

We found it's essential to move the log file to an analyzer system that's separate from the Web server. Our test file, a 250M-byte weekly log, placed a significant burden on the analyzer, a 200-MHz Pentium Pro with 128M bytes of memory. In every case, the analyzer was essentially locked up during the processing of the log file - a period of no less than two hours for Hit List and WebTrends and as many as 16 hours for NetIntellect and Site Server.

Once the log files are on the analyzer's local disk, most of the products preprocess the log files and put them into a database. Hit List works with any ODBC database; net.Analysis requires Microsoft SQL Server; and Site Server comes with Microsoft Access but can use SQL Server. NetIntellect and WebTrends are a little different: They use an internal, proprietary format for storing log files.

We found little reason to prefer one method of storing log files over the other, with an important exception. Intranet managers who plan to run many reports on the same data with different filters should make sure the processed logs are stored in a database where users can access them with other tools.

Most analyzers discard data to save space and increase performance. For example, it's common for an analyzer to throw out image records, such as GIF and JPEG files, because they add little information to reports. This capability can result in immense savings - in our sample log file, more than one million of the 1.3 million hits were for graphics.

But be warned: With some analyzers, you can't retrieve data that you've thrown out without sorting through the entire log file again. When we asked WebTrends for a second report on the same data, the program took nearly an hour to come up with the results. NetIntellect and Hit List, however, were able to generate a second report in just seconds. Neither Site Server nor net.Analysis requires a full database reload, but each software package still took several hours to complete the second reports we requested.

If you plan simply to run one report over and over again, either storage approach will work, and you should look for the fastest product. If you want lots of reports, you need to consider both analysis and reporting times. If you plan to run a variety of reports on the same data, consider Hit List, which delivered the fastest overall response time. We suggest avoiding NetIntellect and Site Server for performance reasons, unless your log files are tiny or your log analyzer workstation is very fast.

Running reports

For example, no two analyzers agreed on the mix of Web browsers and operating systems, and some reports were wildly inaccurate. Hit List filed all Internet Explorer browsers as "Netscape 4.X." On the other hand, Hit List was the only analyzer that let us easily fix this bug once we discovered it.

All the Web server log analyzers spend a great deal of time providing information about the geographic location of Web site visitors. While this feature isn't a major selling point for developers of access-controlled intranets and extranets, it's worth noting - particularly because the results are less than reliable.

In our sample, approximately 35% of the log file entries were numeric, meaning that no organizational information was associated with them in the Internet's Domain Name System. Another 20% of the entries came from AOL's proxy servers. This means that 55% of the log entries could not be tracked to any geographic location. Yet all of the log analyzers were perfectly comfortable drawing pie charts that divided site visitors among the U.S., Canada, Australia and Japan, for example.

What's most alarming is that none of the vendors adequately warn users about how misleading and incorrect this information can be. The worst offender in this case is Hit List, which in our test not only didn't point out that more than one-third of the data had to be discarded, but also labeled graphs to say that all users with ".com" at the end of their domain name - including the 20% that came from AOL proxy servers - were in the U.S.

NetIntellect, Site Server and WebTrends also miserably failed this test, providing deceptive statistics as well as poor and misleading interpretations of the geographic and organizational information available.

Log Twists

WebTrends doesn't have the same flexibility as NetIntellect or Hit List, but it offers the most useful and interesting set of reports in its complete package, including such cool queries as most popular search engines, search strings and Web crawlers.

The king of the report family is undoubtedly net.Analysis. Although many of the analyzer's 180-plus standard reports are small variations of each other ("page views per day for the current month" vs. "page views per day for the previous month"), net.Analysis offers some things that no other analyzer does.

For example, you can request a clickstream analysis, which determines the precise page sequences accessed most often by users. However, these reports are so costly to generate that they may be of little use to most companies. What's more, we abandoned our efforts to generate a full set of clickstream reports after our analysis workstation had worked on them for six hours and still wasn't finished.

Viewing the results

Some packages allow you to generate reports remotely. Hit List and WebTrends have fairly simple procedures for remote report generation, while net.Analysis has a massive Web-based report generation and management facility. In fact, net.Analysis reports are more naturally managed and viewed through their built-in Web server than through any other mechanism. Net.Analysis can generate reports for specific users, authenticate users for access and automatically clean up old reports. Net.Analysis has an elegant interface, which helps if you have lots of users looking at many different reports.

Choosing a product

For most companies, Hit List and WebTrends are excellent choices. Hit List is more flexible and a practical necessity if you have to generate many different reports with many different criteria. But while you get these useful drill-down features, you sacrifice some reports that the product doesn't provide, such as search engine query strings.

WebTrends offers a better set of reports with more interesting information out of the box. If you want to set up your analyzer and let it run, and if you don't plan to try and finesse every graph, WebTrends gives you lots of great information quickly and easily. And for only $200 more than the standard version, WebTrends Professional adds management features such as content analysis and link checking.

NetIntellect and Site Server did respectable jobs of generating reports, but because of their snail-like pace when handling the log file we threw at them, we wouldn't put them in our top tier.