What's lurking in your network? Find out by decrypting SSL

Organizations have spent vast sums of money on security systems and, when deployed and operated correctly, they play a key role in safeguarding the organization. However, most systems have one critical dependency: The traffic flowing through must be readable. If the traffic is encrypted, many systems are almost completely useless, giving the system owner a false sense of security.

Exactly how much of a problem is this? A recent report published by Palo Alto Networks sheds some light. According to the company's Application and Usage Risk Report, 7th Edition, 36% bandwidth on corporate networks is encrypted. That's a 36 in 100 chance your network-based information security systems will miss the bad stuff. And in reality, the chance is greater than 36%, because the bad guys know where to hide the bad stuff so your tools can't see it. Furthermore, the percentage of traffic that is encrypted is increasing as more applications and websites use encrypt-by-default policies.

CLEAR CHOICE TEST: SonicWall stands tall in SSL decryption testing

MORE: SSL decryption may be needed for security reasons, but employees are likely to 'freak out'

So what can be done? Clearly, blocking all encrypted traffic at the enterprise edge is not feasible. The answer lies with a technological capability that allows us to peek inside the encrypted traffic: on-the-fly decryption. The remainder of this article is dedicated to explaining how this can be done. I won't be referring to any one vendor's implementation, but rather will attempt to stick to the basics and explain how the technology works.

Contrary to what you may be thinking, you do not need a team of mathematicians or NSA-grade supercomputers for the task. On the contrary, it's actually quite simple once you understand the basics.

Encryption 101

When you open the browser on your computer (or smartphone or tablet) and go to a secure website such as your bank, you notice the URL begins with HTTPS (notice the "S"). This indicates that all data being exchanged with the remote Web server is being encrypted by an encryption scheme called PKI (public key infrastructure). It works like this:

  • The Web server has a secret encryption key called a private key, which is just a long, seemingly random string of characters stored in a computer file. Only the Web server has access to the private key. It also has a public certificate (which is also just a computer file) that contains another encryption key, called the public key, that is different from the secret key.
  • The private key and the public key are mathematically related such that anything encrypted by the public key can only be decrypted by the private key. In other words, the encryption operation cannot be reversed using the public key. (Exactly how the mind-bending math works is beyond both the scope of this article and, frankly, my intelligence).

When your browser wants to communicate with an encrypted Web server, the following sequence of events occurs (depicted graphically in Figure 1 for those who like pictures).


  1. The Web browser asks the Web server for its public certificate (which, remember, contains the public key).
  2. The Web server gladly obliges and sends the certificate.
  3. The browser then generates a brand new random encryption key (I'll call it the session key because it is unique to this particular browsing session).
  4. Using the public encryption key from the server's certificate, the browser encrypts the session key. Remember, only the private key can decrypt something encrypted by the public key.
  5. The browser sends the encrypted session key to the Web server.
  6. The Web server decrypts the session key using its private key. The browser and Web server now have a shared secret key that they can use like a decoder ring to establish encrypted communication for the remainder of the browsing session.

Make sense? The key to understanding PKI encryption is the relationship between the public and private keys; the public key is used to encrypt, and the private key is used to decrypt. And since the only entity in the whole world that has access to the private key is the server, anything encrypted by the public key can only be decrypted by the Web server.

Now that we have a basic understanding of PKI, let's get back to the subject at hand. To decrypt traffic so your security tools can examine it we have to get in the middle of the session. How we do this depends on the function or type of traffic you are trying to decrypt. There are two categories:

  1. Inbound traffic initiated by a client computer in the outside world, probably on the Internet, and directed toward a server in your organization, perhaps a Web server.
  2. And outbound traffic, initiated by a client computer within your organization and directed toward a server in the outside world, probably on the Internet.

Let's take a look at each of these and explore how to decrypt each.

Decrypting inbound traffic

Let's say your company has a website hosted on a Web server in your data center. The website uses SSL, so all traffic to and from the website is encrypted. When a client on the Internet accesses the site using a computer, smartphone or tablet, an end-to-end SSL-encrypted connection is established between the client's browser and the Web server, making the connection completely invisible to your organization's network security tools.

Decrypting this traffic to make it visible to your security tools requires two steps:

  1. Placing a copy of the server's private key on a decryption-capable device
  2. Getting the data, or a copy of the data, to the decryption-capable device

The first step is easy, but the second step can be accomplished in several ways (depicted in Figure 2):

  1. Mirror the traffic to the decryption device by using a network tap or some other similar mechanism. The security device uses the server's private key to decrypt the session key sent by the client's browser the same way the actual server does, and then follows the SSL session, decrypting all traffic to and from the server. These devices are reactive in nature. That is to say, they cannot actively block an attack in progress because the security device is only looking at a copy of the traffic.
  2. Place the decryption device directly inline between the client and the server. In the same way as the scenario above, the decryption device uses the server's private key to decrypt the session key and is then able to follow the SSL session. The benefit to this solution over the one above is the security device's ability to proactively block attacks in progress due to the fact that the actual connection is being routed through the security device.
  3. Actually terminate the SSL connection on the security device itself. In this scenario, when a client browser initiates a connection to a website, the connection is actually established with the security device. The security device then establishes its own connection to the real Web server, retrieves the content requested by the client, and sends the content to the client. The connection between the security device and the real Web server may itself be encrypted as well (the security device becomes the client from the perspective of the Web server), but doesn't have to be. If the security device and the real Web server are both located in the same secure facility, for example, there may not be a need to secure the communication between them, thus saving resources on the Web server.

SSL decryption

Each of the above methods has its strengths and weaknesses, and which is used in a given architecture depends on many factors. However, they all share one key point: Unencrypted data never leaves the device. As a result, end-to-end data encryption is maintained.

Decrypting outbound traffic

With outbound traffic, the vast majority originates from employees browsing the Internet, checking their email, posting on Facebook, etc. This traffic is potentially damaging to your organization in numerous ways: Users may send proprietary company information over web-based email, post confidential data on social network sites, etc. Every user in your organization who accesses an encrypted website is a potential point of entry into your network, or point of exit for confidential information.

Decrypting outbound traffic is a little trickier than decrypting inbound traffic. As we just discussed, when decrypting inbound traffic we load the private key for the server onto the decryption device, giving it the ability to decrypt traffic to or from the server. But we can't load the private key for every single Web server on the Internet onto your security device, so another strategy is necessary.

The strategy for decrypting outbound traffic requires a somewhat more detailed understanding of PKI. Look back again at the Encryption 101 section above. I intentionally skipped an important detail right after Step 2 to keep it simple. What really happens after the server sends its certificate to the browser, before going any further, the browser decides whether or not it trusts the certificate. It makes this decision based on who signed the certificate. An entity that signs certificates is called a certificate authority, and computer browsers come pre-loaded with a list of trusted certificate authorities. Any website certificate signed by a certificate authority that the browser trusts will also be trusted. We can exploit this behavior to decrypt outbound traffic like this:

  • Make the decryption device a certificate authority, giving it the ability to sign certificates
  • Configure the users' browser to trust the new certificate authority
  • Place the decryption device inline between the users and the Internet

MOXIE MARLINSPIKE WEIGHS IN: The SSL certificate industry can and should be replaced

Do you see where we are going with this? When a user browses to an encrypted website, the encryption device intercepts the request, generates a new certificate on the fly pretending to be the Web server, signs it, and sends it to the user. And because the user's browser is configured to trust certificates signed by the decryption device certificate authority, it will have no idea that it had the wool pulled over its eyes and continue establishing the encrypted connection. The decryption device then establishes its own connection to the actual Web server and transparently proxies all requests between the user and the server.

Not all of the decryption methods described above are appropriate for every scenario. You'll have to analyze your architecture to determine which solution works for your environment. Many vendors produce decryption-capable systems and I recommend you take a look at the strengths and weaknesses of several before deciding which to deploy. Be sure you understand the limitations of each and test in a lab or pilot environment before a production deployment.

With the right tools in the right place, you can take a peek inside your traffic and see what's lurking inside.

Heder, CCIE No. 24788, is a network architect with NES Associates in Alexandria, Va., specializing in large-scale enterprise and data center network design. Heder holds a master's degree with a concentration in network architecture and design, and has a patent filed for an IPv6 technology. He can be reached at brian.heder@gmail.com.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2013 IDG Communications, Inc.

SD-WAN buyers guide: Key questions to ask vendors (and yourself)