The need to address more demanding recovery time objectives has meant that many companies now try to recover from locally maintained backup tapes. As RTOs become increasingly aggressive however, many shops would now like to take advantage of the speed of disk-based recoveries. The snag, of course, is that disk-based storage can still be somewhat pricey when compared to keeping data on tape, and we need some way to accommodate all that growth.

Now a new generation of virtual tape libraries (VTL) is beginning to show up in computer rooms. They can provide the immediacy of disk-based recovery while still reducing the total amount of data that must be stored - all that while still providing the same level of data protection.

Today, the story of VTLs to the rescue.

Date "de-duplication", sometimes called "single-instance storage", keeps a record of all data on the system, and then relies on pattern matching to identify and eliminate instances of duplicate data. By not storing duplicate bits of data, potentially huge savings in disk space result. For instance Diligent Technologies, an early mover with this technology, says its system can reduce the total amount of stored data by a ratio of 25:1 or more, depending on the environment.

How the various products accomplish this sort of thing - if indeed they do - will vary from vendor to vendor. All de-duplication technologies however will rely on an index that tracks the data in the repository and allows for the identification of data redundancy. The management software will look at the new data, compare it to data that already exists on the system, and then only store data that doesn't match existing patterns. Pattern matching algorithms will all be proprietary, as will the table look-up software used to scan the index. It should be an interesting exercise to see which vendor's algorithm works most efficiently, and it should be very benchmarkable.

If this stuff really works, we can expect to see a serious impact on at least two areas of enterprise IT: disk-buying habits and rapid data recoverability.

A data reduction ratio of 25:1 means that your disk requirement to provide online recoveries suddenly drops to 4% of what it presently is. In other words, if you are keeping a terabyte of disk backups on your VTL today, tomorrow that number reduces to 40GB. And the 760GB of storage that is left over means you can defer additional VTL storage purchases until ... well, even if your data grows at a clip of 25% per quarter, it will be years before you will need to add more spindles to your VTL's storage capacity.

Deni Connor is principal analyst for Storage Strategies NOW.