Virtualization running on hosts with only 2 on-board pNICs and 4 pNIC in a slot have less security, redundancy, and performance challenges than other topologies. Administrators are forced to make only one major choice on which security zones to combine upon a pair of pNICs. Yet, the combination of service console/management appliance with VMotion is an accepted practice for 6 pNICs when an IP storage network is involved.
How to setup virtual networking when 6 pNICs are involved follows:
pNIC0 -> vSwitch0 -> Portgroup0 (service console)
pNIC1 -> vSwitch0 -> Portgroup1 (VMotion)
pNIC2 -> vSwitch1 -> Portgroup2 (Storage Network)
pNIC3 -> vSwitch1 -> Portgroup2 (Storage Network)
pNIC4 -> vSwitch2 -> Portgroup3 (VM Network)
pNIC5 -> vSwitch2 -> Portgroup3 (VM Network)
With 6 pNICs you can setup three redundant vSwitches each for different purposes one for the service console/management appliance and VMotion. Another just for the Storage Network to grant higher levels of redundancy. While the third vSwitch is solely for the VM Network which includes redundancy.
The networks attached to vSwitch0 work best with VLANs however subnets will also work. With 2 pNICS on vSwitch0 each network gains its own pNIC and therefore better performance and security unless there is a failure case.
In this configuration pNIC0 is the failover pNIC for Portgroup1 while pNIC1 is the failover pNIC for Portgroup0. For vSwitch1 and vSwitch2 there is no need for predefined failover modes, just the default which includes vSwitch Port ID based load balancing. In other words, load balancing is outbound only and based on the port to which the VM is connected.
This method will grant the most redundancy, security, and performance for a 6 pNIC configuration.
Virtualization expert Edward L. Haletky is the author of VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers. He recently left HP, where he worked in the Virtualization, Linux, and High-Performance Technical Computing teams. Haletky owns AstroArch Consulting, providing virtualization, security, and network consulting and development. Haletky is also a Guru and moderator for the VMware discussion forums, providing answers to security and configuration questions.
We have 15 copies of Haletky's book up for grabs. Go here for entry details (competition will open Nov. 1) and go here for a sneak peek of the book.
The Leader in Network Knowledge
Why not just solve the problem? Get a second 4-port card.
Better to get two 4-port cards and put one redundant port for each of your four networks (Vmotion, SAN, VMs, Service) on a port on a different card. This solves a number of card failure scenarios, remembering that the PHY can blow all 4 ports at once, and the controllers handle 2 ports at a time. Also, realize that NICs are constrained by both slot type and # of controllers on the card. Thus, higher performance comes when you spread the load across two cards rather than 2 ports on the same card. Although, of course, Linux use of link aggregation is retarded and ESX doesn't do anything to improve that, preferring failover to load balancing unless you do mild handstands.
But ignoring all that nonsense: if you're going to have a big server that is worth having that many networks, blow an extra $400 and put two 4-port NICs in (Intel Pro 1000/PT Quads are the best if you need density, and they come in low profile flavor if you need them for typical 2U servers) into different PCI Express busses. That's the performance edge you want, to match up to what you're proposing for security/redundancy.
Re: Why not just solve the problem? Get a second 4-port card
8 Ports would be ideal in some situations. I know some vNetworks where 12 pNICs would be ideal. However, not every one can place even 6 pNICs within their equipment. Consider the case for blades and 1U boxes. Some blades only have a maximum of 2 pNICs while some 1U boxes can only place a single extra card within it.
The limitations on pNICs is generally either physical or cost related.
Best regards,
Edward L. Haletky
Post new comment