Dial P-A-I-N for patching

Opinion
Feb 9, 20083 mins

Patching is an item in most [url=http://thinkingproblemmanagement.blogspot.com/2008/01/administrative-checklist-for-security.html]security checklists[/url], and can be a real pain. Last year Microsoft released a patch which stated that if you had a Realtek device in your PC you had to load a special separate patch or else the device would stop functioning after the patch. The notes said that this ONLY applied to Realtek devices. A significantly large number of PCs had Realtek devices which are used to provide sound on PCs. Now in the modern business sound has become business critical. As an example, there are guys who view and listen to business TV on their PCs for trading information, voice mails are emailed as message attachments to be listened from email clients and even voice conversations that have been recorded are played back on PCs. We could not have all of these sound devices disabled due to a patch. We thought about rolling the special patch out to all PCs, but the note with the special patch warned that deploying it was not a good idea if you did not have the specific problem which it is meant to correct. The legaleeze in the note made it sound that if this special patch was deployed on anything other than a Realtek device then there was a probablity that the PC would explode! OK, so we checked our inventory system and targeted all the PCs with Realtek devices for the special patch after testing on a sample group. The test group approved the main patch and the patch was deployed overnight. The next morning, I was told that the telephone lines were down, by the service desk as I walked in. Seeing a large number of people using their phones, I asked for greater clarification as it was not possible if people were still using their phones. A quick check determined that direct inward dial was working and so was outward calling. The switchboard consoles were not receiving any calls and the main switchboard hunt group was not being answered. So I hoofed off to eyeball the switchboard consoles. The consoles are PC based with proprietary boards that connect to the PBX. There are six consoles and the people manning the consoles confirmed that they were inoperable. One of them told me that when she switched on her PC that morning there was an error message. She rebooted her PC and I immediately recognized the error message from the special patch notes. Were the switchboard console PCs using Realtek devices and had we missed them? No, a check of device manager showed that the proprietary console board was a Siemens device. Whatever, I decided to deploy the special patch as I had my suspicions and after a reboot the switchboard console was operational. The switchboard operators had to start dealing with highly irate customers, who asked why they weren’t answering their phones. As a result of the incident, we have included one of the switchboard consoles in the patching test group but my underlying concern is that patching has resulted in a large number of major incidents, disproportionate to the vulnerabilities that the process mitigates. My opinion is that a monthly patching regime should be extended to a quarterly regime and that the security agents on the PCs (like antivirus and antispyware) should be mitigations to prevent a known vulnerability exploitation. The longer testing cycle would provide better testing and less major incidents. What are your opinions around the problems of patching?