Perimeter DLP tools require fine tuning to effectively block 'bad' data from escaping the network
Finding the right data loss prevention tool means striking a balance between speed, success rate at detecting and/or blocking sensitive data from exiting the network, and adequate coverage across a broad range of rule-sets and protocols.
Finding the right perimeter-based data loss prevention tool means striking a balance between speed, accuracy at detecting and blocking sensitive data from exiting the network, and adequate coverage across a broad range of rule-sets and protocols.
DLP products come in three categories: perimeter-based, client-based and those that take a combined approach. In this test, we evaluated perimeter-based appliances from Fidelis Security Systems, Palisade Systems, Code Green Networks and GTB Technologies.
The DLPs were set up inline (except for Code Green's Content Inspector, which doesn't support in-line mode) between a simulated WAN and LAN and were configured with a set of 10 rules. We then ran about 1,100 files through each device, waiting about a minute between each file, to determine how accurately the device detected and blocked a total of 276 "bad" files and to what degree network performance was affected by the inline DLP.
Here are our key findings:
All of the products did an effective job detecting harmful files that were sent over the specific protocols that the product supports. But not all products support a wide range of protocols.
Some of the products that did well at detecting harmful files were less adept at blocking.
None of the products were able to analyze or block encrypted traffic.
There's a network performance hit that needs to be taken into account when running these products in-line.
Code Green's Content Inspector scored highest when it came to detection. Code Green also scored high on ease of configuration. But Code Green was limited in the range of protocols it could block.
Our Clear Choice winner is Fidelis' XPS because of its easy-to-use interface, flexible rule-set, amazing reporting, and better-than-average detection and blocking ability.
Palisade's Packetsure and GTB's Inspector were somewhat unrefined by comparison, requiring more work to understand the rule structure and adding unneeded complexity to the overall process. But they were still very competitive when it came to detecting harmful files.
Generally DLP vendors deploy engineers to the customer site to set up and configure the device, but we decided to do it ourselves to get a hands-on understanding of how the product works from installation through reporting.
For Packetsure and Content Inspector, the basic installation was fairly straightforward and the products were setup with little to no trouble. For the other two products, basic installation was a little more difficult, requiring numerous contacts – via e-mail and phone. But they eventually all were set up without the need for a technician to show up on-site.
After each product was set up and could pass data between the simulated LAN and WAN, we configured the device to our filtering specifications. This included a sample set of 10 rules chosen to test some of the basic features and blocking potential.
The DLPs were set up to look for Social Security and credit card numbers, certain pieces of source code, and five words in a row from a short story, which would be used to prevent any part of a specific report from leaving the network.
We also set up rules to check for maximum file sizes or .mp3 files. And we fingerprinted a data set containing a list of customer names, addresses and Social Security numbers and set up a rule blocking any combination of the three.
Configuration: Code Green is tops
Code Green's Content Inspector was the easiest product to configure and write rules for. The rule language is simple and the graphical interface is very usable. Code Green breaks rule creation down into two categories: data and policy. One defines data to be blocked using a variety of tools, and then configures a policy to check for it. This was very straightforward and easy to change, with no need to restart the device or reload the settings. In the configuration simplicity arena, Code Green goes above and beyond all the other products.
Fidelis' XPS sensor has a "Command Post" server to handle management and configuration, a mail sensor server (provided via built-in Postfix SMTP proxy), and a Web sensor (implemented via a third-party BlueCoat Web proxy appliance).
Rule creation is straightforward and simple using a Web GUI. XPS is the only product that allows you to submit sample files in order to test each rule before you make it live.
If you ever have a question about a specific rule or a page you are on, Fidelis has built in wonderful help links on each page that explain each check box or button. This is a life-saver and allowed us to create the majority of the rules without any technical support contacts.
Palisade's Packetsure provided a simple wizard to help with setup and was the only product to have such a helpful starting point. However, if one wants to add or change a rule outside of the wizard, the sailing is not quite so smooth.
Part of the problem may be that Packetsure is really two products trying to work together as one: there is a content analysis engine and a protocol analysis engine. The Palisade protocol analyzer only inspects the packet payload (instead of reassembling the data stream as the content analysis does). This two-pronged approach helps isolate each rule, but it makes managing the product difficult.
Also, in our testing the rules did not always work as expected. For example, one "content analysis checkbox" means packet analysis and another content analysis checkbox actually reassembles the data stream before it analyzes it (similar to all the other products).
Packetsure has a "connect to home" functionality which the user can enable right out of the box. This feature can be very useful when calling tech support or even with the initial setup as it allows Palisade to assist using a secure VPN.
GTB's Inspector has the most difficult configuration process of the four products. In order to write a rule, one must edit a text configuration file, add some regular expressions and format each line very specifically. For example, in order to write a rule to check for the words "Top Secret" in a file, a regular expression had to be written in a large text box on the Web management interface.
There is no wizard and no graphical interface. The other limiting factor with GTB's Inspector is the fact that its rule-set functionality is very limited. In our test it could only implement about half of the desired rules. Even a simple rule such as looking for specific filenames or maximum file size was not supported.
Performance: Fidelis is fastest; Code Green wins detection test
We tested how accurately the product blocked a total of 276 harmful files that we sent, or roughly 30 files for each of the nine protocols (including HTTP, SMTP, POP, IMAP, FTP and Telnet) in our test bed. We also measured how fast the product could pass data through the device, starting with a baseline of 581Mbps, which is the capacity of our network without any device present.
The best performance from a detection perspective was Code Green's Content Inspector, which detected 90% of the data we threw at it. And the 10% Content Inspector missed was because of the lack of support for encrypted traffic streams (SSH sessions), which no product supports.
However it can only block files on four of the tested protocols: HTTP, Secure-HTTP, FTP and SMTP. The first three are done using a third-party BlueCoat Proxy device and the SMTP is done using a built-in mail relay.
This lack of blocking ability across a wide variety of protocols was the major drawback in Code Green's Content Inspector. But if your company is only worried about those four protocols, this product would be recommended.
Fidelis' XPS had an 84% success rate in detecting and blocking across all protocols and streams of data. The marketing line for this company states that they can block data on all 65,535 ports and we would have to agree. This product blocked virtually everything it could detect, only failing on one file type - an archived Web site.
The product handled obfuscated data very well -- catching four of five files. POP and IMAP provided a little bit of trouble, but after a few custom patches from the engineers, it worked as expected.
The choice faced by all these products is a tradeoff between performance and blocking effectiveness. When data moves through a DLP device, the product can choose to either cache it, determine that it's good and then let it out, or try to do analysis on the fly, and suffer some data leakage.
Fidelis chose performance and won our speed test, passing traffic at 90% of network capacity. However, occasionally pieces of sensitive data leaked from the network. All the other products chose to prioritize blocking over speed.
Palisade's Packetsure is targeted at the basic protocols of HTTP, SMTP and FTP, and showed a high blocking rate on those specific protocols. But Packetsure, possibly because it seems to contain two products in one, was the slowest product, performing at only 55% of the allowable bandwidth.
Furthermore, blocking a specific protocol and scanning based on content analysis work as expected, but when you combine the two, problems emerge, creating unexpected results. For example when you try to limit content analysis to a certain protocol, you have to choose between using a weaker content analysis system (which won't reassemble the stream) or not limit your blocking based on protocols. The latter is the best way to handle this problem, but doing so reduces the flexibility and blocking capability of the product.
GTB's Inspector was the most consistent product. What it detected and blocked on one protocol it detected and blocked on every protocol with no extra work. The problem with this product was it only could check based on certain rules and those rules were limited. About half of our detection tests failed on this product because the rule types are not supported. However, even with its lack of rule support, it still caught 62% of the illegal files.
Across supported protocols, Inspector was the only product to score a 100%, catching every single file we could send through the machine at 80% of the allowed bandwidth.
Fingerprinting: GTB Inspector gets high marks
Fingerprinting is a concept that is implemented fairly well in these DLP products. Fingerprinting will hash a file and look for parts of that file leaving the network.
Fingerprinting is used to prevent sensitive information from leaving a network and at the same time to reduce false positives. For example, most organizations want to prevent Social Security numbers from leaving local networks. However, a lot of things can look like a Social Security number (e.g., a mistyped phone number or an online order number).
Fingerprinting takes any sensitive information you may have on your network and looks for a number of pieces that specifically correspond with it, to make it a piece of information that you don't want erroneously leaving your network.
One could fingerprint a list of names, addresses and Social Security numbers and, instead of triggering on any nine-digit number, the DLP will only trigger when a Social Security number is sent out with the associated full name. Or, instead of looking for a specific word phrase, it can look for a few sentences from a report.
All of the tested products support this feature, but GTB Inspector is the most powerful and flexible -- customers can fingerprint data from a variety of flat files, databases or spreadsheets.
That power and flexibility, however, comes at the cost of simplicity. GTB has its own program which one must use to fingerprint data, as opposed to other products that allow an administrator to upload and fingerprint a file from the main management interface.
While Palisade’s Packetsure can scan and hash the usual range of files that most of the others support, when it comes to database fingerprinting – linking two fields in a relational database – it requires the files to be exported into a flat file for analysis. Fidelis' XPS included the ability to test your fingerprints once you created them.
Code Green's Content Inspector could fingerprint data of all sorts and allowed you to set up scenarios on when this data would trigger an alert. For example, if you fingerprinted names, addresses and Social Security numbers, you could say alert me when you see two Social Security numbers and one has a matching name. No other product had as much granularity and yet remained simple to use.
Reporting: Code Green, Fidelis are tops
One of the most useful parts of a DLP product is its reporting feature. For an administrator, knowing what a product is seeing and blocking is extremely useful.
Code Green's Content Inspector and Fidelis' XPS have the best reporting systems. Both do a great job of allowing flexibility, ease of use, exporting capabilities and beautiful (and meaningful) graphs to help make this data easy to digest. Plus, Code Green's product allows for simple integration into many alert software applications (such as Crystal Reports) or even custom applications, as it uses a simple Postgres database.
Palisade's Packetsure tries to implement the functionality needed in report generation, but doesn't quite get there. The interface seems very clunky and there is an annoying wait of 3 to 5 seconds whenever you want to generate a report. However, Packetsure has a very useful protocol graphing tool that allows you to see, in real time, what kind of traffic is moving across your perimeter (even allowing an administrator to drill down to specific applications). It would be nice if this was tied to the blocking feature in some way, but it's not.
GTB's Inspector lagged behind the competition in terms of reporting. It provided acceptable, straightforward reports and even included the ability to generate graphs to help interpret the data. It doesn't miss the mark on reporting; it just wasn't nearly as impressive as the other three products.
Fidelis XPS: Overall winner
Fidelis XPS was the most developed product in overall features, general flexibility and its ability to block.
It has a "Command Post" server to handle management and configuration, a mail sensor server (provided via built-in Postfix SMTP proxy), and a Web sensor (implemented via a third-party BlueCoat Web proxy appliance).
Installation isn't simple, but it didn't take more than a few hours to get XPS set up and running. The built-in help links are very useful when writing rules and the XPS includes the ability to test rules that you write. The XPS does a great job of remaining flexible across all protocols yet still maintaining the ability to block on these protocols. The management interface allows you to easily create rules and see reports.
This product was the fastest we tested, blocking 80% of harmful files, while only taking a 10% performance hit. If you are looking for a product to block a variety of protocols and applications, in addition to the standard HTTP and SMTP, look no further.
Palisade's Packetsure: Two products in one
Palisade's Packetsure product seems to contain two products in one: a protocol analyzer and a content analyzer. Packetsure had a high detection rate, but the slowest speed, performing at 50% of maximum bandwidth. This product has some interesting features such as the ability to help set up the product via a VPN and a useful graph showing data passing in and out of the network.
Installation was simple and straightforward, accomplished in less then an hour. The initial setup was assisted greatly by the use of a wizard. However, altering rules after using the wizard is bothersome and reporting is more difficult and clunky than it could be.
Code Green's Content Inspector: Tops in detection
Content Inspector was the best product tested when it comes to detecting data leakage. However because it can only block a few protocols, the detection is not well used.
Installation was very simple and configuration was easy to understand without reading any manuals. This is the only product that allowed every rule to be implemented. This product was able to detect 90% of the data we threw at it, which is almost double some of the other competitors. The 10% they missed was because of lack of support for encrypted traffic streams (SSH sessions), which no product supports.
However it can only block files on four of the tested protocols: HTTP, HTTPS, FTP and SMTP, three of which are done using a third-party BlueCoat Proxy device and the last is done using a built in mail relay. When blocking using one of these methods, this product was flawless, blocking every file it could detect. However this lack of blocking ability across a wide variety of protocols was the largest drawback in Code Green's Content Inspector.
GTB Inspector: Consistently solid
GTB's Inspector was a very consistent product but is limited in rule generation. Installation was a headache, taking nearly eight hours to set up. However after the product was set up and configured it was extremely consistent. What it detected and blocked on one protocol it detected and blocked on every protocol it supported.
The problem was that it was only able to check based on certain rules and those rules were limited. About half of our detection tests failed on this product because the rule types are not supported. However, even with its lack of rule support, it still caught 62% of the illegal files. Across supported protocols, this was the only product to score a 100%, catching every single file we could send through the machine at the 80% network bandwidth it allowed.
Another redeeming quality is that GTB's Inspector has a very powerful and robust fingerprinting ability allowing all sorts of customization.
Evans manages the Internet-Scale Event and Attack Generation Environment (ISEAGE) at Iowa State University, a testing facility built to simulate any network architecture. This lab also performs disaster recovery tests and network penetration tests. He is also a doctoral candidate in Computer Engineering as Iowa State. He can be reached at firstname.lastname@example.org.
Blakely is a concurrent graduate student in Information Assurance at the Iowa State University of Science and Technology. He is pursuing his Doctorate of Philosophy in Computer Engineering. He works as a research assistant at ISEAGE, and has 10 years of systems administration experience in the education, public, and private sectors. He can be reached at email@example.com.
Sponsored by AT&T
Sponsored by AT&T
From BGP to SSL, several Internet protocols are no match for today’s malicious hackers -- and should be...
“America’s Finest News Source,” better known as The Onion, has been poking fun at Google for more than...
Buyers of the earthly explanation for whatever fell from the sky in Roswell, N.M. back in 1947 are...
Sponsored by Brocade
Sponsored by AT&T
Microsoft faces not only its 40th anniversary in 2015, but a host of challenges that will define it for...
The breach of Sony Pictures Entertainment is clearly the biggest data breach of 2014, but theft of...
Announcement follows two-week investigation into major cyberattack
New Amazon competitors offer some compelling deals in the cloud, but at what price?