Virtualization running on hosts with only 2 on-board pNICs and 2 pNIC in a slot have security, redundancy, and performance challenges. Administrators are forced to make choices on how much redundancy is required and where to place it.
How to setup virtual networking when 4 pNICs are involved follows:
pNIC0 -> vSwitch0 -> Portgroup0 (service console)
................. -> Portgroup1 (VMotion)
pNIC1 -> vSwitch0 -> Portgroup2 (Storage Network)
pNIC2 -> vSwitch1 -> Portgroup3 (VM Network)
pNIC3 -> vSwitch1 -> Portgroup3 (VM Network)
With 4 pNICs you can setup two redundant vSwitches each for different purposes one for the service console/management appliance, VMotion, and your Storage Network. While the second vSwitch is solely for the VM Network granted a higher level of redundancy.
The networks attached to vSwitch0 for Service Console and VMotion work best with VLANs however subnets will also work. In addition, it is important to keep the Storage Network on its own pNIC except in a failure case as this will increase its performance.
In this configuration pNIC0 is the failover pNIC for Portgroup2, while pNIC1 is the failover pNIC for Portgroup0. For vSwitch1 there is no need for predefined failover modes, just the default which includes vSwitch Port ID based load balancing. In other words, load balancing is outbound only and based on the port to which the VM is connected.
This method will grant the most redundancy, security, and performance for a 4 pNIC configuration.
Virtualization expert Edward L. Haletky is the author of VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers. He recently left HP, where he worked in the Virtualization, Linux, and High-Performance Technical Computing teams. Haletky owns AstroArch Consulting, providing virtualization, security, and network consulting and development. Haletky is also a Guru and moderator for the VMware discussion forums, providing answers to security and configuration questions.
We have 15 copies of Haletky's book up for grabs. Go here for entry details (competition will open Nov. 1) and go here for a sneak peek of the book.
The Leader in Network Knowledge
Whereabouts does one put
Whereabouts does one put 'VMKernel' so one can use a NAS datastore??
Re: Whereabouts does one put
In this case the vmkernel network for NAS storage is what I refer to as the storage network. Whether it is NAS or iSCSI, any IP storage network should be setup as described.
Best regards,
Edward L. Haletky
Thank you for replying, I
Thank you for replying, I will save this and set up our new hosts this way.
Why the need for Portgroup4 on Vswitch1
Why is there a need to assign Portgroup 4 for Vswitch1? if pNIC3 or pNIC2 would fail wouldn't either one act a redundant connection for the other? I guess my confusion sets in when you have to assign a VM's nic a specific port group. In this example would each VM require two nics?
Portgroup 4
You are correct that is not needed it should have been portgroup 3. Typo, that I will fix.
ESX Newbie Q
So, if the additional card is a dual-port card (in a slot), and it fails, then does the entire VM Network go down (as per the config above, pNIC2 and pNIC3 would both be on the same card).
Why oh why would you not
Why oh why would you not start with redundancy in the service console and vmotion networks ?
I know the response is going to be "its hard with the gui to do some of these things, we sometimes make our system unreachable" - then script out the CLI commands needed and make that part of a deployment template from day one.
Kind of ditto with the storage network - "increase its performance" ... what are you talking about ? "Decrease" maybe.
In the example above you have 4 NICs and 2 of them are used for client facing traffic but only 1 for storage - what is YOUR IP vs SAN traffic ratios on ESX servers ? On mine it is pretty much loaded in favor of the SAN especially with database apps - suck a lot of data from tables .. spit out subset to clients, repeat and then store updates.
Even fileservers are going to be 1 to 1.
I kind of want equal bandwidth pathways for storage and client traffic - if you throttle SAN traffic you are essentially throttling client traffic as well.
Look at the 10Gb FCOE adapters - 4 Gb of bandwidth max allocated for fiber which is roughly 1/2 & 1/2 IP/SAN instead of 2 to 1 as in this example and my guess is that 4Gb is legacy 4Gb fiber technology
Post new comment