How we tested e-mail archiving products

We set up two Exchange servers, one Exchange 2003 server running on Windows 2003, and one Exchange 2007 server running on Windows 2008. The Exchange 2003 server had a 4GB mail store containing 11 years worth of mail in one mailbox (68,000 messages). The Exchange 2007 server had a 30GB mail store with just more than 1,000 users and about 5 million messages.

Both Exchange servers existed within the same domain, and were connected to a Fibre Channel storage-area network comprised of a Compellent Storage Center 3000 and a QLogic SANbox 5200 switch. The two Exchange servers and the Windows 2003 server used to run the archiving application (except in the case of the appliances) were all configured to boot from the Compellent Storage Center 3000 SAN system – this allowed for each test to be done with a fresh installation cloned from a single original setup – the servers were simply configured to boot from a different clone for each test. This configuration eliminated our having to re-install Windows and Exchange 10 times to prevent interference between products.

Each archiving product was configured to archive all old messages on the Exchange 2003 server and to archive all new messages on both servers. All products took between 1:58 and 2:24 to archive the Exchange 2003 server and archived messages on the larger Windows 2007 store at about the same rate of 30,000 messages per hour.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

We set up two Exchange servers, one Exchange 2003 server running on Windows 2003, and one Exchange 2007 server running on Windows 2008. The Exchange 2003 server had a 4GB mail store containing 11 years worth of mail in one mailbox (68,000 messages). The Exchange 2007 server had a 30GB mail store with just more than 1,000 users and about 5 million messages.

Both Exchange servers existed within the same domain, and were connected to a Fibre Channel storage-area network comprised of a Compellent Storage Center 3000 and a QLogic SANbox 5200 switch. The two Exchange servers and the Windows 2003 server used to run the archiving application (except in the case of the appliances) were all configured to boot from the Compellent Storage Center 3000 SAN system – this allowed for each test to be done with a fresh installation cloned from a single original setup – the servers were simply configured to boot from a different clone for each test. This configuration eliminated our having to re-install Windows and Exchange 10 times to prevent interference between products.

Each archiving product was configured to archive all old messages on the Exchange 2003 server and to archive all new messages on both servers. All products took between 1:58 and 2:24 to archive the Exchange 2003 server and archived messages on the larger Windows 2007 store at about the same rate of 30,000 messages per hour.

To measure the loads added to the mail server by archiving process, we ran LoadSim against the 2007 server with a volume of about 2000 messages per hour, then enabled archiving and ran the same LoadSim test, comparing server utilization between the two runs. Increases in loads with archiving enabled ran from 8% to 13%, making it difficult to call a winner here either – variations between runs were often greater than from product to product, due to the inevitable random loads caused by other server tasks (see story on performance of these products).

After load testing, we conducted searches to find messages with specific subjects and specific attachments. Search times were less than a second once archiving and indexing had completed across all products.

In addition, we restored messages that had been deleted from the user mailbox and tested restores of complete mailboxes and the smaller message store, although these products are not generally intended to be used to replace a backup/restore system for the entire mail system – the speed is much less for large amounts of data than would be achieved by backup applications such as BackupExec, which backed up the 5GB larger store in a few minutes, rather than the 166 hours that would have been required to complete archiving using the message-by-message methods used by these products.

Each product was also evaluated for ease of use, granularity of management and delegation of discovery capabilities, installation and documentation and flexibility of configuration to suit differing network architectures, including scalability, support for multiple domains, and other features such as PST file search on local systems.

< Previous story: Symantec's Enterprise Vault wins top spot in 10-vendor test >

Read more about data center in Network World's Data Center section.