Virtualization running on hosts with only 2 on-board pNICs and 3 pNIC in a slot have security, redundancy, and performance challenges. Administrators are forced to make choices on how much redundancy is required and where to place it.
How to setup virtual networking when 5 pNICs are involved follows:
pNIC0 -> vSwitch0 -> Portgroup0 (service console)
pNIC1 -> vSwitch0 -> Portgroup1 (VMotion)
pNIC2 -> vSwitch0 -> Portgroup2 (Storage Network)
pNIC3 -> vSwitch1 -> Portgroup3 (VM Network)
pNIC4 -> vSwitch1 -> Portgroup3 (VM Network)
With 5 pNICs you can setup two redundant vSwitches each for different purposes one for the service console/management appliance, VMotion, and your Storage Network. While the second vSwitch is solely for the VM Network granted a higher level of redundancy.
The networks attached to vSwitch0 work best with VLANs however subnets will also work. With 3 pNICS on vSwitch0 each network gains its own pNIC and therefore better performance and security unless there is a failure case.
In this configuration pNIC0 and pNIC1 are the failover pNICs for Portgroup2, pNIC1 and pNIC2 are the failover pNICs for Portgroup0, and lastly pNIC0 and pNIC2 are the failover pNICs for Portgroup1. For vSwitch1 there is no need for predefined failover modes, just the default which includes vSwitch Port ID based load balancing. In other words, load balancing is outbound only and based on the port to which the VM is connected.
This method will grant the most redundancy, security, and performance for a 5 pNIC configuration.
Virtualization expert Edward L. Haletky is the author of VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers. He recently left HP, where he worked in the Virtualization, Linux, and High-Performance Technical Computing teams. Haletky owns AstroArch Consulting, providing virtualization, security, and network consulting and development. Haletky is also a Guru and moderator for the VMware discussion forums, providing answers to security and configuration questions.
Edward's latest book was selected as the March, 2011, book giveaway for Cisco Subnet.Read a chapter excerpt of VMware ESX and ESXi in the Enterprise: Planning Deployment of Virtualization Servers.