Sometimes life gets too interesting. That’s when you blow a day tracking down some weird, esoteric issue, which was exactly what happened to us a couple of weeks ago after a story was posted in Gibbsblog.
The post concerned the surprising appearance of a warning by Firefox that a certificate had been presented by a Web site and the certificate’s issuer (otherwise called the certificate authority) was unknown. This meant the certificate couldn’t be verified, which meant that the site couldn’t be trusted, hence the warning.
This was odd because the certificate in question was for the Navy’s Warfighter Response Center and the issuer was the U.S. Department of Defense. The problem was that the page actually requested was a Google search result rather than the Navy site. As the search term entered into Google had been “binary explosives” it seemed plausible that some kind of monitoring was going on.
<aside>The reason we were looking for “binary explosives” was to find a story written just after the recent security brouhaha over passengers carrying liquids onto aircraft. The story in question was from The Register and is a “must read.”</aside>
Unfortunately, as interesting as being monitored might have been, the idea of some kind of conspiracy between Google and the Defense Department to watch what people search for was unlikely for two reasons.
First, would the spies show their hand by allowing an authentication certificate to load? Hardly. Second, could such a conspiracy remain hidden? Of course not.
Anyway, another question remained: How was it that a Web page for the Navy was being loaded when a page of Google results was being returned? The answer? Precaching.
Precaching (also called prefetching) is a technique used by the Firefox browser to speed up the loading of Web sites. If the feature is enabled, when a Web page is loaded the URLs in the page are collected. The browser then launches multiple threads and the contents of each of those URLs are loaded into a cache before you might ask for them.
What was happening in this case was one of the entries returned by the search was this, and because it is a Secure-HTTP connection the site presented its certificate when the precaching subsystem tried to access the page. As the Defense Department isn’t included in Firefox’s list of certificate authorities by default, and because we were configured to see the warnings, that’s what happened. Except the precaching wasn’t done as we thought by Firefox.
To test whether this was the cause we switched off Firefox’s precaching. Then we purged the cache and cookies and tried the search again. The warning happened again!
We’re sure that some of you have, at this point, had an “ah-ha!” moment and know the answer. We didn’t. We tried to figure out what was going on and did things like run Capsa (the subject of Gearhead two weeks ago) and trapped and analyzed all of the HTTP and Secure-HTTP traffic.
This didn’t solve our problem, and we found unexpected Web sites being accessed by unknown processes (what fun — more to investigate).
Then we had our “ah-ha!” moment: The precaching was something that a Firefox plug-in was doing! The culprit was Fasterfox an add-on that blocks pop-ups, times how long it takes to load pages, tweaks a whole range of network and browser rendering settings, as well as precaching.
We switched Fasterfox off and no more certificate warnings.
Precaching might sound like a good idea but it has a number of downsides, such as increasing bandwidth use, increasing server loads, and it can preload content that you might not want to have loaded,which sounds like a lawsuit waiting to happen.