VMware CEO apologizes for virtual-server bug

VMware's new CEO Paul Maritz has apologized to customers in a company blog posting after a major bug prevented VMware users from logging on to virtual servers this week.

Click to see: Paul Maritz, VMware CEO

"I want to apologize for the disruption and difficulty this issue may have caused to our customers and our partners. Your confidence in VMware is extremely important to us, and we are committed to restoring that confidence fully and quickly," Maritz wrote in a letter posted on the VMware official blog.

Maritz, who's in his second month on the job, said the incident has spurred an in-depth review of VMware's internal processes. "We are doing everything in our power to make sure this doesn't happen again," he wrote. "VMware prides itself on the quality and reliability of our products, and this incident has prompted a thorough self-examination of how we create and deliver products to our customers. We have kicked off a comprehensive, in-depth review of our QA [quality assurance] and release processes, and will quickly make the needed changes."

The problem came to light Tuesday, because of a time-clock bug in ESX 3.5 and ESXi 3.5 Update 2. The bug fooled the software into thinking customer product licenses had expired when the date changed to Tuesday, Aug. 12. Maritz noted this caused three problems: virtual machines that were powered off could not be turned on, virtual machines that were suspended failed to leave the suspend mode and virtual machines could not be moved to different physical hardware using VMotion. (Compare server products.)

VMware released emergency patches and new VMware ESX installations, available on its express patch Web site. 

"The issue was caused by a piece of code that was mistakenly left enabled for the final release of Update 2," Maritz wrote. "This piece of code was left over from the prerelease versions of Update 2 and was designed to ensure that customers are running on the supported, generally available version of Update 2."

Maritz, who took over the CEO role in July when the VMware board ousted co-founder Diane Greene, said VMware failed in two areas with its recent software updates. VMware, he wrote, failed to disable the faulty code in the final release of Update 2 and failed to uncover the error during its quality-assurance process.

This failure on the part of VMware, which has long dominated the x86-server virtualization market, could damage its credibility. "This certainly appears to be the most publicized bug for VMware so far, and I think it is damaging to VMware and virtualization as a whole. The hypervisor is the lowest software level on the server, and if you have an issue like this, boom, all your infrastructure is down," Gary Chen, a senior analyst with Yankee Group, told Network World on Tuesday. "Software will always have bugs, but a widespread issue like this that affects all VMs is really damaging, especially at this point in time where virtualization is starting to take off. VMware is going to have to fix this fast, provide an explanation and outline what they will do to strengthen their QA in the future."

Maritz's open letter to customers was posted later in the day on Tuesday, after VMware issued the emergency patches.