Open Source Subnet An independent Open Source community View more

Encryption news: What you need to know

A recap of the week's big news in encryption.

We probably should have seen this coming, what with government spying dominating headlines for the summer, but encryption suddenly became a hot topic this week.

First was the surprise shut down of Lavabit, the encrypted email service that NSA whistleblower Edward Snowden reportedly used to communicate with a human rights activist. Once the activist let it slip that they had used Lavabit to evade the NSA, Lavabit founder, 32-year-old Ladar Levison, apparently started to receive some heat. Shortly thereafter, Levison says he had to shut down the service that he had been running for 10 years, claiming he "had to pick between the lesser of two evils."

In a discussion with Democracy Now, Levison spoke vaguely about the U.S. government and its impact on privacy and digital communications, but made it clear that he couldn't divulge details on account of an apparent government-issued gag order. 

Strangely, online security company Silent Circle shut down its encrypted email service shortly after Lavabit.

So, with several encrypted email service providers shielding themselves from government pressure, who better to step up in their place than Kim Dotcom? The eccentric founder of Megaupload, the file-hosting service that was shut down by the U.S. government in January 2012, now runs a similar service under the name Mega, and will soon launch an encrypted communications service with the same brand. Mega CEO Vikram Kumar confirmed to ZDNet that the company is indeed developing an encrypted email service, and is working to make its functionality similar to that of Google's Gmail.

"The biggest tech hurdle is providing email functionality that people expect, such as searching emails, that are trivial to provide if emails are stored in plain text (or available in plain text) on the server side," Kumar told ZDNet. "If all the server can see is encrypted text, as is the case with true end-to-end encryption, then all the functionality has to be built client side. [That's] not quite impossible, but very, very hard. That's why even Silent Circle didn't go there."

However, MIT researchers promptly dumped cold water on the encryption discussion with an announcement of its own - recent research suggests that the "information theory," a kep staple behind encryption that dates back to 1948, is actually false. Presenting the paper at the International Symposium on Information Theory in Ireland, the researchers pointed to a "notion of entropy" dictated by a professor who last taught at MIT 35 years ago.

The problem, Médard explains, is that information-theoretic analyses of secure systems have generally used the wrong notion of entropy. They relied on so-called Shannon entropy, named after the founder of information theory, Claude Shannon, who taught at MIT from 1956 to 1978.Shannon entropy is based on the average probability that a given string of bits will occur in a particular type of digital file. In a general-purpose communications system, that’s the right type of entropy to use, because the characteristics of the data traffic will quickly converge to the statistical averages. Although Shannon’s seminal 1948 paper dealt with cryptography, it was primarily concerned with communication, and it used the same measure of entropy in both discussions.But in cryptography, the real concern isn’t with the average case but with the worst case. A codebreaker needs only one reliable correlation between the encrypted and unencrypted versions of a file in order to begin to deduce further correlations. In the years since Shannon’s paper, information theorists have developed other notions of entropy, some of which give greater weight to improbable outcomes. Those, it turns out, offer a more accurate picture of the problem of codebreaking.When Médard, Duffy and their students used these alternate measures of entropy, they found that slight deviations from perfect uniformity in source files, which seemed trivial in the light of Shannon entropy, suddenly loomed much larger. The upshot is that a computer turned loose to simply guess correlations between the encrypted and unencrypted versions of a file would make headway much faster than previously expected.
Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10