Microsoft’s Enterprise Class Server Virtualization Technology
In just over a year, Microsoft went from having basically no real enterprise-class server virtualization technology to one that is at par and has VMware scared and running. Back in the middle of 2008, Microsoft’s server vritualization offering was Virtual Server 2005 that didn’t provide 64-bit guest support and really didn’t scale beyond having 3-4 guest sessions on a single server. Then Microsoft released Windows 2008 and Hyper-V with 64-bit guest session support and the ability to easily host 8, 10, 15 guest sessions on a single server. However for a year, VMware kept saying that Hyper-V didn’t provide “live migration,” the ability to move guest sessions during planned physical host downtime, and thus Hyper-V wasn’t “enterprise ready.”
In true Microsoft competitive fashion, Microsoft not only released Live Migration in Hyper-V R2 that came with Windows 2008 R2, but Microsoft is including it for “free” as part of the normal Windows 2008 R2 enterprise (or datacenter) license, a technology that VMware used to charge $20,000+ for. A couple years ago, pretty much 100% of our server virtualization consulting business was focused on VMware. Today, half of our server virtualization consulting business is now based on Hyper-V R2, and it’s not that our VMware business has decreased, more and more organizations are virtualizing, and they are virtualizing under Hyper-V because it does come free with Windows, its gotten great reviews, it works extremely well, and Windows administrators that know how to run Microsoft Windows applications and thus know how to run, setup, and manage Hyper-V guest sessions.
The big thing that is new to Hyper-V is the whole Live Migration technology, which leverages Failover Clustering that allows a Hyper-V R2 guest session to failover to another Hyper-V R2 host server so that a Hyper-V host server can effectively be evaculated of any guest sessions to be patched, updated, upgraded, or maintained without ANY interruption to the operation of the guest session.
Quick Migration and Live Migration
There are two forms of automated migration provided by Windows Server 2008 R2 Hyper-V: Quick Migration and Live Migration. These migration processes can be used to increase service availability for planned and unplanned downtime.
Although both technologies achieve the same thing—moving virtual servers between Hyper-V hosts—they use different methods and mechanisms to achieve it. Both require at least two Hyper-V host servers in a cluster, attached to the same shared storage system. Usually, the shared storage is an iSCSI or Fibre Channel storage area network (SAN).
Quick Migration
Quick Migration provides a way to quickly move a virtual machine from one host server to another with a small amount of downtime.
In a Quick Migration, the guest virtual machine is suspended on one host and resumed on another host. This operation happens in the time it takes to transfer the active memory of the virtual machine over the network from the first host to the second host. For a host with 8GB of RAM, this might take about two minutes using a 1GB iSCSI connection.
Quick Migration was the fastest migration available for Windows Server 2008 Hyper-V. Microsoft made considerable investments in Hyper-V migration technologies, trying to reduce the time required to migrate virtual machines between Hyper-V hosts. The result was Live Migration, which has the same hardware requirements as Quick Migration, but with a near instantaneous failover time.
Live Migration
Since the release of Hyper-V V1 with Windows Server 2008, the number-one most-requested feature by customers is the ability to migrate running virtual machines between hosts, with no downtime. VMware’s VMotion has been able to do this for some time. Finally, with Windows Server 2008 R2, it can be done natively with Hyper-V for no extra cost. This makes it a compelling reason to move to Hyper-V.
Live Migration uses failover clustering. The quorum model used for your cluster will depend on the number of Hyper-V nodes in your cluster. In this example, we will use two Hyper-V nodes in a Node and Disk Majority Cluster configuration. There will be one shared storage LUN used as the cluster quorum disk and another used as the Cluster Shared Volume (CSV) disk.
NOTE
If there is only one shared storage LUN available to the nodes when the cluster is formed, Windows will allocate that LUN as the cluster quorum disk and it will not be available to be used as a CSV disk.
This following describes how to use Hyper-V Live Migration to move virtual machines between clustered Hyper-V hosts.
Configuring the Cluster Quorum Witness Disk
Live Migration requires a Windows Server 2008 R2 cluster configured to use shared storage. Typically, these are LUNs provisioned on an iSCSI or Fibre Channel SAN. One LUN will be used as the witness disk for quorum and another will be used as a Cluster Shared Volume (CSV) to store the virtual machine images.
The LUN for the shared witness quorum disk must be configured before the cluster is formed, so that cluster manager can configure the cluster properly. Connect this LUN via iSCSI or Fibre Channel to both nodes you will use for the cluster. The disk must be initialized and formatted with an NTFS file format prior to cluster use. When properly configured, both nodes share the same online Basic disk and can access the disk at the same time.
IMPORTANT
The Windows cluster service will always use the first shared disk as the cluster quorum disk. Provision this disk first on each node.
Now that the shared storage witness disk has been configured, we can move on to installing the Windows cluster.
Installing the Failover Clustering Feature
Before a failover clustering can be deployed, the necessary feature must be installed. To install the Failover Clustering feature, perform the following steps:
1. Log on to the Windows Server 2008 R2 cluster node with an account with administrator privileges.
2. Click the Server Manager icon in the Windows taskbar.
3. Select the Features node in the tree pane when Server Manager opens.
4. In the tasks pane, select the Add Features link.
5. In the Select Features window, select Failover Clustering, click Next, and click Install on the Confirm Installation Selections page to install the feature.
6. When the installation completes, click Close to complete the installation and return to Server Manager.
7. Close Server Manager and install the Failover Clustering feature on each of the remaining cluster nodes.
Running the Validate a Configuration Wizard
Failover Cluster Management is used to administer the Failover Clustering feature. It can be run from Server Manager or as a separate MMC console. After the feature is installed, run the Validate a Configuration Wizard from the tasks pane of the Failover Cluster Management console. All nodes should be up and running when the wizard is run. To run the Validate a Configuration Wizard, perform the following steps:
1. Log on to one of the Windows Server 2008 R2 cluster nodes with an account with administrator privileges over all nodes in the cluster.
2. Click Start, click All Programs, click Administrative Tools, and select Failover Cluster Management.
3. When the Failover Cluster Management console opens, click the Validate a Configuration link in the Actions pane.
4. When the Validate a Configuration Wizard opens, click Next on the Before You Begin page.
5. On the Select Servers or a Cluster page, enter the name of a cluster node, and click the Add button. Repeat this process until all nodes are added to the list and click Next to continue.
6. On the Testing Options page, read the details that explain the requirements for all tests to pass in order to be supported by Microsoft. Select the Run All Tests (Recommended) option button, and click Next to continue.
7. On the Confirmation page, review the list of servers that will be tested and the list of tests that will be performed, and click Next to begin testing the servers.
NOTE
For years, administrators have complained that the Validate a Configuration Wizard window is too small. In Windows Server 2008 R2, administrators can finally resize the window by dragging the lower-right corner. This is not obvious, but try it—it works!
8. When the tests complete, the Summary page displays the results, and if the tests pass, click Finish to complete the Validate a Configuration Wizard. If the tests failed, click the View Report button to review the details and determine which test failed and why the test failed.
Even if the Validate a Configuration Wizard does not pass every test, depending on the test, creating a cluster might still be possible. After the Validation a Configuration Wizard is completed successfully, the cluster can be created.
Creating a Node and Disk Majority Cluster
When the failover cluster is first created, all nodes in the cluster should be up and running. To create the failover cluster, perform the following steps:
1. Log on to one of the Windows Server 2008 R2 cluster nodes with an account with administrator privileges over all nodes in the cluster.
2. Click Start, click All Programs, click Administrative Tools, and select Failover Cluster Management.
3. When the Failover Cluster Management console opens, click the Create a Cluster link in the Actions pane.
4. When the Create Cluster Wizard opens, click Next on the Before You Begin page.
5. On the Select Servers page, enter the name of each cluster node, and click the Add button. When all the nodes are listed, click Next to continue.
6. On the Validation Warning page, select No. I Do Not Require….. The validation test can be run after the configuration is complete. Click Next to continue.
7. On the Access Point for Administering the Cluster page, type in the name of the cluster, complete the IPv4 address, and click Next. The name you choose for the cluster will become a cluster computer account in Active Directory.
8. On the Confirmation page, review the settings, and click Next to create the cluster.
9. On the Summary page, review the results of the cluster creation process, and click Finish to return to the Failover Cluster Management console. If there are any errors, you can click the View Report button to reveal the detailed cluster creation report.
10. Back in the Failover Cluster Management console, select the cluster name in the tree pane. In the tasks pane, review the configuration of the cluster.
11. In the tree pane, select and expand Nodes to list all the cluster nodes.
12. Select Storage and review the cluster storage in the tasks pane. The shared storage disk will be listed as the witness disk in quorum. This disk is used to maintain quorum.
13. Expand Networks in the tree pane to review the list of networks. Select each network and review the names of the adapters in each network.
14. Click Validate Configuration in the Actions pane to start an automated review of the cluster configuration. See the previous section, “Running the Validate a Configuration Wizard,” for more details. Keep in mind that Microsoft support for the cluster will require a successful execution of the validation process.
Adding Additional Shared Storage
At this point, we have a Node and Disk Majority cluster using a shared witness disk to maintain quorum. We can now add the shared storage that will be used as a Cluster Shared Volume.
Another LUN must be provisioned for the Cluster Shared Volume to hold the virtual machine images used in Live Migration. This LUN may be a new unpartitioned volume or one that already contains virtual machine images and data.
Connect this LUN via iSCSI or Fibre Channel to both nodes in the cluster. The disk must be initialized and formatted with an NTFS file format prior to cluster use in the cluster. When properly configured, the disk will show in Disk Management on both nodes.
Next, we add the new shared disk to the cluster.
1. On one of the cluster nodes, open Failover Cluster Management.
2. Expand the Cluster and select Storage.
3. Click Add a Disk in the Actions pane.
4. Select the disk to add and click OK. The disk will be added to available storage.
Configuring Cluster Shared Volumes
Cluster Shared Volumes (CSVs) are a new concept for Windows Server 2008 R2. They provide the new capabilities required for Hyper-V Live Migration to work, and is why only Windows Server 2008 R2 nodes can participate in the cluster.
With CSV, any node can host the virtual machine and any node can access the VHD on shared storage, so virtual machine and disk ownership can move freely across cluster nodes without impacting any other resources on that shared disk.
To enable and configure Cluster Shared Volumes, perform the following steps from the Failover Cluster Management console:
1. Select the cluster name in the Failover Cluster Management console.
2. Click Enable Cluster Shared Volumes in the Actions pane.
3. The Enable Cluster Shared Volumes information dialog box will be displayed. Read the important information explaining that CSVs are only supported for the Hyper-V role on Windows Server 2008 R2 servers. Click the check box that you have read the notice and click OK.
4. A new item, Cluster Shared Volumes, will be added to the tree view. Select Cluster Shared Volumes.
5. Click Add Storage in the Actions pane. The Add Storage window is displayed.
6. Click the check box to select the shared cluster disk and click OK. The shared disk will be added to Cluster Shared Storage
Unlike other shared volume disks, Cluster Shared Volumes are presented to the nodes as a folder in the local C: drive of each node.
From either node, open Windows Explorer and navigate to the path indicated in the Cluster Shared Volume disk, usually C:ClusterStorageVolume1, where each CSV is given a unique number beginning with 1. If another CSV disk is added to the same cluster, it will be accessible from C:ClusterStorageVolume2.
Deploying New Virtual Machines on CSV Failover Clusters
After the desired cluster configuration is achieved, the cluster is ready for the deploying of virtual machines.
1. On one of the cluster nodes, open Failover Cluster Management.
2. Expand the Cluster and select Services and Applications.
3. Now that Cluster Storage Volumes have been configured, the Virtual Machines application is available in the Actions pane. Click Virtual Machines, New Virtual Machine and select the cluster node on which to deploy the virtual machine.
4. The New Virtual Machine Wizard will launch, as it does from the Hyper-V Manager. Click Next at the introduction screen.
5. Provide a name for the new virtual machine and click the Store the Virtual Machine in a Different Location check box. Enter the path to the C:ClusterStorageVolume1 and click Next.
NOTE
It is recommended on Hyper-V servers using Live Migration to change the default location to store virtual machines to the Cluster Shared Volume path. This is configured in Hyper-V Settings of the Hyper-V Manager console.
6. Assign the desired amount of memory for the new virtual machine.
7. Select the virtual network, or choose Not Connected to configure it later. Click Next.
8. Create a new virtual hard disk in the C:ClusterStorageVolume1 folder or select an existing VHD and click Next.
NOTE
Both the virtual machine configuration file and its associated virtual hard disk (VHD) files must reside in the CSV folder location for Live Migration to work.
9. Select how you will install the operating system for the new virtual machine, either using a boot CD-DVD ROM, ISO image, floppy disk, or from a network-based installation server, and click Next.
10. Review the summary of the options you have selected and either click Finish if the settings you’ve chosen are fine, or click Previous to go back and make changes.
11. Click Finish to create the new virtual machine. After the virtual machine is saved to the CSV path, the High Availability Wizard will configure the virtual machine for use in Live Migration. Click View Report to review the step the High Availability Wizard used to configure the virtual machine for Live Migration.
NOTE
It is normal for the High Availability Wizard to report a warning if the operating system for the virtual machine will be installed from the host’s physical CD-DVD ROM, an ISO file, or a floppy drive. This is because the drive or file used for installation is not in a location available to the cluster. Most of the time, this does not matter, but it can be overcome if needed by installing the operating system from an ISO located on the CSV location.
12. Click Finish to complete the configuration of the new virtual machine.
13. Change the virtual machine settings, if desired, to increase the number of virtual processors, change the drive configuration, and so on.
14. Right-click the virtual machine in Failover Cluster Manager and select Start Virtual Machines to start the virtual machine and install the operating system.
Once the operating system is installed, Live Migration can be used to move the cluster from one node to another.
Deploying Existing Virtual Machines on CSV Failover Clusters
If the LUN provisioned as a CSV disk in the cluster contains existing virtual machine images, these can be made highly available. You can also copy any virtual hard disk to the CSV volume and make it highly available:
1. On one of the cluster nodes, open Failover Cluster Management.
2. Expand the cluster and select Services and Applications.
3. Right-click Services and Applications and select Configure a Service or Application. This will open the High Availability Wizard.
4. Click Next on the Before You Begin page.
5. On the Service or Application page, click Virtual Machine and click Next.
6. Select the virtual machine(s) to be made highly available and click Next.
7. Review the Summary page in the wizard and click Finish.
8. Select the virtual machine in the Service and Application pane and click Start Virtual Machines in the Actions pane.
Performing a Live Migration
The virtual machine runs on one of the cluster nodes, known as the owner. When a Live Migration is performed, multiple steps are performed. These steps can be broken down into three stages: preflight migration, virtual machine transfer, and final transfer/startup of the virtual machine.
The first step in Live Migration occurs on the source node (where the virtual machine is currently running) and the target node (where the virtual machine will be moved) to ensure that migration can, in fact, occur successfully.
The detailed steps of Live Migration are as follows:
1. Identify the source and destination machines.
2. Establish a network connection between the two nodes.
3. The preflight stage begins. Check if the various resources available are compatible between the source and destination nodes:
► Are the processors using similar architecture? (For example, a virtual machine running on an AMD node cannot be moved to an Intel node, and vice versa.)
► Are there a sufficient number of CPU cores available on the destination?
► Is there sufficient RAM available on the destination?
► Is there sufficient access to required shared resources (VHD, network, and so on)?
► Is there sufficient access to physical device resources that must remain associated with the virtual machine after migration (CD drives, DVDs, and LUNs or offline disks)?
Migration cannot occur if there are any problems in the preflight stage. If there are, the virtual machine will remain on the source node and processing ends here. If preflight is successful, migration can occur and the virtual machine transfer continues.
4. The virtual machine state (inactive memory pages) moves to the target node to reduce the active virtual machine footprint as much as possible. All that remains on the source node is a small memory working set of the virtual machine.
The virtual machine configuration and device information are transferred to the destination node and the worker process is created. Then, the virtual machine memory is transferred to the destination while the virtual machine is still running. The cluster service intercepts memory writes and tracks actions that occur during the migration. This page will be retransmitted later. Up to this point, the virtual machine technically remains on the source node.
5. What remains of the virtual machine is briefly paused on the source node. The virtual machine working set is then transferred to the destination host, storage access is moved to the destination host, and the virtual machine is reset on the destination host.
The only downtime on the virtual machine occurs in the last step, and this outage is usually much less than most network applications are designed to tolerate. For example, an administrator can be accessing the virtual machine via Remote Desktop while it is being Live Migrated and will not experience an outage. Or a virtual machine could be streaming video to multiple hosts, Live Migrated to another node, and the end users don’t know the difference.
Use the following steps to perform a Live Migration between two cluster nodes:
1. On one of the cluster nodes, open Failover Cluster Management.
2. Expand the Cluster and select Services and Applications.
3. Select the virtual machine to Live Migrate.
4. Click Live Migrate Virtual Machine to Another Node in the Actions pane and select the node to move the virtual machine to. The virtual machine will migrate to the selected node using the process described previously.
NOTE
If there are processor differences between the source and destination node, Live Migration will display a warning that the CPU capabilities do not match. To perform a Live Migration, you must shut down the virtual machine and edit the settings of the processor to “Migrate to a Physical Computer with a Different Processor Version”.
The good portion of the above excerpt came from my book “Windows Server 2008 R2 Unleashed”, a 1550-page hardcover book covering everything from Active Driectory Design and migration, to Remote Desktop Services (“terminal services”), to Windows administration, to configuring DHCP/DNS, to Hyper-V R2, and more.
