Symantec's Enterprise Vault wins top spot in 10-vendor test

E-mail archiving systems can serve a variety of purposes. At the most basic level, they free end users and messaging administrators from that onerous task of keeping individual inboxes as trim as possible by automatically archiving messages to less expensive storage.

E-mail archiving systems can serve a variety of purposes. At the most basic level, they free end users and messaging administrators from that onerous task of keeping individual in-boxes as trim as possible by automatically archiving messages to less-expensive storage.

Performance of these systems

Archive of Network World tests

In addition, these systems facilitate e-discovery processes in which IT staff, auditors and lawyers use specialized search functions to find messages relating to a specific topic.

The software can also be used as a policy enforcement tool that can be configured to look out for messages that defy corporate rules and forward those messages to the appropriate person for review.

Finally, compliance and retention functions help IT organizations keep messages on hand to comply with corporate or legislative requirements.

In this Clear Choice Test of 10 vendors, we found that many are attempting to hit on all use cases defined above – to varying degrees of success -- while others are homing in on one or two main areas.

We invited almost two dozen vendors to participate in this test and a dozen took up the challenge, 10 of which we could compare side-by-side. Those include: Atempo's Digital Archive for Messaging (ADAM), C2C Systems' Archive One, GFI's MailArchiver, Intradyn's ComplianceVault, MessageSolution's Enterprise Email Archive, Mimosa Systems' NearPoint, Sunbelt Software's Exchange Archiver, Symantec's Enterprise Vault, Tangent's DataCove and Waterford Technologies' MailMeter.

We also tested a product from ProofPoint, but because they work differently from other 10 presented here, we've opted to cover those products in a separate story.

The overall top performer was Symantec's Enterprise Vault, with Mimosa Systems NearPoint and C2C's ArchiveOne suite close behind. All three offer a complete array of features, with enterprise-class functionality in management, granularity of permissions and scalability.

We've structured our findings in the context of the four main e-mail archiving purposes. For a full product-by-product see the drop-down menu above.

Mailbox management

When used for mailbox management, e-mail archiving software mainly deals with moving messages and their attachments off the main server to the archive server, reducing space used on the central mail server.

The processes of mailbox management can be divided into two areas – backing up messages so they can be replaced if deleted by accident or lost, and moving the body of the message or any attachment to another server, leaving only a 'stub' on the Exchange server as a place holder.

Backing up an Exchange server via an e-mail archiving product is certainly not a substitute for regular backup process. For one thing, there is the issue of network performance. As noted in our story on performance, we saw a typical backup rate of 30,000 messages per hour. Extrapolating from that rate, a server with 5 million messages would then take over six days to either completely back up or restore via the message-by-message method used by most products tested.

While six days of background processing is acceptable for the one-time job of establishing a new archive, it would indeed be much faster to use a dedicated backup product such as BackupExec. Some of the products tested – namely C2C, Mimosa and Symantec -- integrate with third-party backup applications to provide a single interface for both archiving and disaster recovery of whole servers at good speeds.

But using these products to restore one mailbox or a single message sent to several users is a different matter, and all of the products offer self-service mechanisms for retrieving lost messages, which means that instead of an administrator having to access the backup application to find lost e-mails, users will be able to do that themselves.

Users have two options for finding and restoring a message – the Outlook plug-in or the archiving server's Web portal.

Using the Web portal is a kinder, gentler process. Users log in with their Exchange e-mail login and password and are presented with a simple search interface. While some users may need help constructing an effective search, this is not an area where a better interface will help much – only user training to explain the concepts.

Mimosa NearPoint and Symantec Enterprise Vault are a little more usable than the rest, by virtue of having had more previous versions to wear the sharp corners off, but GFI's MailArchiver also did quite well in this category.

The other application falling under the mailbox management umbrella is offloading messages from the user's mailbox to the archive server. The process is the same as one done for the disaster-recovery process -- stubbing the messages on the original Exchange server and moving the body of the message and their respective attachments to the archive server.

But the driving purpose is decreasing the space used on the Exchange server to take advantage of less-expensive storage on the archive server. Because the archive server doesn't need to handle the kinds of input/output loads that the Exchange server might, it doesn't need high performance storage – SATA-based drives are perfectly adequate, and can cost a tenth of Fibre Channel or SAS drives.

Testing in this area was pretty much pass/fail, because messages were either stubbed correctly or they were not. All of the products passed.

Once messages were stubbed, the effective size of the message was reduced to 2KB, the minimum that any message takes up (at least as far a reporting the message size through Exchange).

One differentiator here is that not all products do de-duplication of the archived messages – if an attachment is the same for six users, then only one copy is stored in the archive, and links are used for the rest. While the big enterprise players --Symantec, Mimosa, Atempo and C2C – can all de-dupe attachments, it's important to know that some of the products targeted at the small-to-midsize market, namely Sunbelt and GFI – do as well.

Once messages have been archived, ideally an Outlook plug-in is used to improve the display of stubbed messages and facilitate the retrieval of messages from the archive server.

All of the Outlook plug-ins provide the same functionality – the ability to show stubbed messages clearly as stubbed, and to retrieve them seamlessly from the archive if the user double-clicks on a message to open it, essentially making the stubbing process transparent to the end user. All also extend the search capabilities of the archive server to the Outlook interface, providing the same interface that users are used to in Outlook, but with orders of magnitude better response times for searches of large mailboxes.

The standout among plug-ins is Symantec, not because it offers more or less functionality or an easier interface, but because of the extremely granular level of control given to the administrator over which functions the user has access to. For instance, the admin can grant or deny the ability to mark messages for archiving, restore accidentally deleted messages, and about 30 other specific functions.

Atempo, C2C, MessageSolution, Mimosa, Sunbelt and Symantec provide forms for Exchange and for Internet Information Services that provide the same level of transparent user experience inside the Outlook Web Access (OWA) interface. This does not mean that the other vendors don't support OWA – users will just have to click on a link in the opened stubbed message to access the original content. Installing the forms can be complicated for the administrator, because the process involves an abstruse Microsoft management system (not necessarily the fault of the messaging archive vendors), and produces a somewhat less transparent degree of functionality. It shows a different icon to identify stubbed messages, requiring an additional click inside the stubbed message to retrieve the full message.

Atempo, C2C, MessageSolution, Sunbelt all support access to archived messages through the Blackberry and other mobile devices. We did not test this feature.

Products from ADAM, C2C, Mimosa, Symantec and Tangent did an excellent job of mailbox management from an administrative standpoint. They all had very flexible policy engines which enable the administrator to create sophisticated archiving policies. The policy engines can let you, for instance, archive only messages that the user keeps for more than 30 days, messages with specific keywords in the subject line, or messages with attachments.

All could find and import PST files from user's workstations to consolidate all e-mail, allowing administrators who previously forced users to archive messages in a local PST file on their respective PCs, to bring those messages back into the e-mail server and ensuring they are archived and backed up properly. While all products tested were able to import PST files, Mimosa, Sunbelt and Symantec offer specific tools for finding PST files on user's local PCs, consolidating and automatically importing them.

The products from GFI, Intradyn, MessageSolution, Sunbelt, Waterford didn't fail at any point in our mailbox management use case testing, they just don't provide the depth of functionality and management granularity that the top products do. While enterprises may need that functionality, many small or midsized businesses might be perfectly happy with the 'good' products.

E-discovery: Finding what you need

E-discovery is basically a jazzed up term for an advanced search of archived e-mail. Because most of the products tested use the same SQL database, search performance is much the same for all the products, returning results from even complex searches of more than 68,000 messages in less than a second in our test set up. Of course, your mileage will vary based on the power of the system upon which you install the archiving server and the speed of the attached storage mechanisms (see How we did it for the systems used in our test).

What differentiates products is the granularity with which IT staff can grant access to the search engine and its results. This granularity must allow users unfettered access to their own mail while at the same time extending rights to an auditor to search through e-mail belonging to various groups or users; must produce reports; and help define what can be done with messages once they are found – such as whether they can be exported to .PST files or be passed to others for review.

Of these top performers, Symantec offered the most granular control over how much an auditor could do, which users' mail the auditor could see, what types of searches could be set up, and what could be done with the e-mails that were found. Atempo, C2C and Mimosa were all close behind Symantec.

GFI, Sunbelt and Tangent all offer fairly flexible search engines, but not as many options for creating roles with very specific limitations.

Access to the search engine is available to all users for their own mailbox through the Outlook plug-in, and via a Web portal for more general searches. All of the products provide the same end-user functionality, with very little to differentiate them in this area.

Moving into the realm of searching among e-mail across users, most products allow for one person, such as an auditor or e-mail admin, to be designated to create searches. Then another person can be designated to actually review the messages found by the search.

Symantec Enterprise Vault offered the most flexibility by an extremely tiny margin. Many organizations won't need or care about the ability to grant an auditor the ability to view message headers but not open the messages, or to open messages but not view attachments, but in large organizations were security, HR and legal roles are extremely compartmentalized, this level of granularity can be useful. (Compare e-mail archiving systems in Network World's Buyer's Guide.)

The difference between excellent and good ratings lay in the number of security options available – the good products generally had only a few predefined roles, such as auditor or administrator. The excellent products allow an IT department to define custom roles with very specific abilities. For instance, Atempo, C2C, Mimosa and Symantec all provide the ability to search e-mails for the marketing group and the engineering group, to view e-mails for the marketing group but not the engineering group, and to run specific search templates such as those checking for harassment issues, but not pertaining to Sarbanes-Oxley Act compliance, for example.

Compliance and retention

Compliance and retention come into play when IT must adhere to either legislative requirements or corporate policy. What you want from a product is both the ability to specifically support legislative standards -- such as the Gramm-Leach-Bliley Act, Health Insurance Portability and Accountability Act (HIPAA), Payment Card Industry and Sarbanes-Oxley – coupled with an interface that allows the administrator to set specific policies for retention: which documents will be retained, for how long, for which users and/or some groups, and what is done at the end of the retention cycle.