Update: @SSauer points out that this is vShield, and posted some information here: A New Generation of vShield Security Products

vSwitch, vNetwork, vShield?

Most hypervisor products include an internal virtual network switch, but VMware’s ESX has multiple choices. The original “dumb” virtual Ethernet switch was augmented by vSwitch back in the ESX 3 days, bringing more-advanced configuration options.VMware improved and renamed the vSwitch in vSphere 4, creating the vNetwork Standard Switch (vSS).

But it was the introduction of vNetwork Distributed Switch (vDS) in vSphere 4 that really set VMware’s network capabilities apart. The champion of this field is Cisco, whose Nexus 1000V virtual switch extends their NX-OS datacenter networking OS right into the ESX world.

The Cisco Nexus 1000V runs both the supervisor and Ethernet modules inside the virtual ESX environment

As illustrated above, the Nexus 1000v consists of two key components:

1. The Virtual Supervisor Module is an implementation of NX-OS on an ESX virtual machine, and provides the interface and configuration of the virtual network
2. The Virtual Ethernet Module runs at a lower level in ESX, replacing the vSwitch for networking between VMs

One can think of the Cisco Nexus 1000v as a specialized replacement for the more-generic vNetwork Distributed Switch. Both include plug-in vSwitch replacements and centralized management, and both implement more-advanced network protocols like private VLANS and receive-rate limiting as well as supporting vMotion. But Cisco’s 1000v goes much further, adding PortChannel, LACP, security and QoS, and advanced management features.

See this comparison of vSwitch, vSS, vDS, and 1000v as well as Rich Brambley’s vSS/vDS post

From a technology standpoint, the key to both vDS and 1000v is the ability to replace the core ESX vSwitch with a more-capable alternative. Now let’s turn to what VMware might be introducing next.

What Do We Know?

Howie Xu, VMware Director of R&D, released a video discussing his sessions at VMworld, entitled ”The Future Direction of Networking Virtualization” (TA8361). This video begins with a quick pan past Xu’s whiteboard (pictured below) and includes a discussion of the state of the art, vision, and product and technology roadmap for VMware’s networking-related efforts.

The part that really piqued my interest was later in the video, when Xu talks about creating a “networking virtual chassis or hypervisor” to allow third-parties to develop and roll-out advanced networking devices within vSphere. VMware has already steamrolled through the heart of server-based applications, making VMware-based virtual appliances as common an installation format as the DVD. Now the company is turning its attention to the network. This is huge!

Xu speaks of both a platform and a service to support this ”open extensible networking virtual chassis platform,” and goes on to suggest that it could be used by “networking security, load balance, application acceleration, IP address management, and performance management” products. The virtual appliance marketplace is already populated by the familiar names in networking, from F5 to Bluecoat to Checkpoint. Therefore, VMware must be talking about something much deeper and more advanced than merely encouraging the creation of more virtual appliances!

The core question is whether VMware is opening up the “green box” in my diagram above to run third-party applications and what level of system access they will get.

Consider this a vFabric pre-intro!

Then there is the name. The whiteboard prominently includes the words, “vFabric Intro” in the corner. Judging by the rest of the readable content, this indicates that this new technology will indeed be called “vFabric” as Stu speculated.

Update: It appears that vFabric is not the name of this virtual chassis (thanks, Stu and Howie!). Good thing, too, since the name, “vFabric” is a registered trademark of QLogic Corporation, for “computer software for managing computer hardware, namely switches used in networks.”

What’s Coming?

I’m more of a storage guy, so I rang up my friend Greg “Etherealmind” Ferro and ran some ideas past him. Greg and I talked about the needs of network-based devices, and how they differ from traditional server-based applications.

Greg suggests that vFabric will really assist vMotion

Networkers have been conditioned with the belief that custom silicon is the best way to achieve low latency and high performance for network devices. The same could be said of the storage world, where companies like HDS, 3PAR, and BlueArc pride themselves on their custom ASICs. But EMC, HP, and others are proving that Intel’s server-class CPUs and peripheral busses now have the guts to go head-to-head with custom silicon. The networking world is no different, with many newer companies basing their products around industry-standard hardware.

But deploying these systems in a virtual environment is more challenging. Can a virtual machine hypervisor prioritize threads for network devices? Can it handle the overhead related to networking operations in real-time? What happens in the event of a DDoS or network flood? Most network devices run real-time operating systems like VxWorks or QNX to ensure packet throughput, but virtual environments are notorious for “overflow” of I/O or CPU load between guest machines.

The whiteboard provides some hints as to how VMware will tackle these issues. First, we spot the term, “latency-aware queueing,” which suggests that a mechanism will monitor the hypervisor and alter the queues for virtual network devices as the load changes. As latency rises, the hypervisor can move workloads to different processor cores or even alternate hardware using vMotion. We also spot a reference to “non-blocking”, suggesting an asynchronous I/O mechanism will reduce the likelihood that one of these virtual network devices will have to wait for data.

Both of these technologies are hallmarks of real-time operating systems (RTOS), and are critical to the design of scalable hypervisors like VMware’s ESX. It is likely that the company is developing an advanced hypervisor environment for these specialized devices, an evolution of vDS and the API that allows the Cisco Nexus 1000v to run its Virtual Ethernet Module.

Stephen’s Stance

If our assumptions are true, then this is an exciting development indeed. If VMware exposes the “green box” in our diagram above to third-party developers, we could see an entirely new and more-powerful ecosystem evolve around VMware vSphere. Running virtual network devices in a quasi-real-time environment will enable even-greater integration and flexibility.

The Nexus 1000v has not eliminated purchasing of Cisco hardware, and vFabric will not destroy the larger network device market. But we expect wide vendor support for the concept, especially those involved in lower-end and remote-office applications. We would love to see Palo Alto Networks, Infoblox, SolarWinds, and Vyatta, to name a few, developing next-generation applications for vFabric. Virtualization-aware integrated networking shouldn’t be the sole domain of Cisco.

© Stephen Foskett for Gestalt IT, 2010. | A VMware Hypervisor For Networkers?
Read more posts categorized as Networking, Server Virtualization

For new users who are seeking a thorough explanation of how VMware ESX networking functions, I’ll recommend a series of articles by Ken Cline titled The Great vSwitch Debate. Ken goes into a great level of detail. Go read that, then you can come back here.

Before I get started it’s important to understand that, for the most part, the information in this article applies only to VMware ESX. VMware ESXi doesn’t have a Linux-based Service Console like VMware ESX, and therefore doesn’t have a readily-accessible CLI from which to run these sorts of commands. There is a remote CLI available, which I’ll discuss in a future post, but for now I’ll focus only on VMware ESX.

The majority of all the networking configuration you will need to perform on VMware ESX boils down to just a couple commands:

• esxcfg-vswitch: You will use this command to manipulate virtual switches (vSwitches) and port groups.
• esxcfg-nics: You will use this command to view (and potentially manipulate) the physical network interface cards (NICs) in the VMware ESX host.

Configuring VMware ESX networking boils down to a couple basic tasks:

1. Creating, configuring, and deleting vSwitches
2. Creating, configuring, and deleting port groups

I’ll start with creating, configuring, and deleting vSwitches.

Creating, Configuring, and Deleting vSwitches

You’ll primarily use the esxcfg-vswitch command for the majority of these tasks. Unless I specifically indicate otherwise, all the commands, parameters, and arguments are case-sensitive.

To create a vSwitch, use this command:

esxcfg-vswitch -a <vSwitch Name>

To link a physical NIC to a vSwitch—which is necessary in order for the vSwitch to pass traffic onto the physical network or to receive traffic from the physical network—use this command:

esxcfg-vswitch -L <Physical NIC> <vSwitch Name>

In the event you don’t have information on the physical NICs, you can use this command to list the physical NICs:

esxcfg-nics -l (lowercase L)

Conversely, if you need to unlink (remove) a physical NIC from a vSwitch, use this command:

esxcfg-vswitch -U <Physical NIC> <vSwitch Name>

To change the Maximum Transmission Unit (MTU) size on a vSwitch, use this command:

esxcfg-vswitch -m <MTU size> <vSwitch Name>

To delete a vSwitch, use this command:

esxcfg-vswitch -d <vSwitch Name>

Creating, Configuring, and Deleting Port Groups

As with virtual switches, the esxcfg-vswitch is the command you will use to work with port groups. Once again, unless I specifically indicate otherwise, all the commands, parameters, and arguments are case-sensitive.

To create a port group, use this command:

esxcfg-vswitch -A <Port Group Name> <vSwitch Name>

To set the VLAN ID for a port group, use this command:

esxcfg-vswitch -v <VLAN ID> -p <Port Group Name> <vSwitch Name>

To delete a port group, use this command:

esxcfg-vswitch -D <Port Group Name> <vSwitch Name>

To view the current list of vSwitches, port groups, and uplinks, use this command:

esxcfg-vswitch -l (lowercase L)

There are more networking-related tasks that you can perform from the CLI, but for a new user these commands should handle the lion’s share of all the networking configuration. Good luck!

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

© Scott for Gestalt IT, 2009. | New User’s Guide to Configuring VMware ESX Networking via CLI
Read more posts categorized as All, Featured, Networking, Server Virtualization

Cisco today introduced their B-Series UCS blade server

Did you feel the earth move today? No? Despite months of hyperbolic warnings, Cisco’s release today of their Unified Computing System blade servers (code named Project California) is gentle and evolutionary. The networking giant is challenging HP, IBM, and Dell, to be sure, but not with a slap in the face. Cisco is easing into the server pool, unlike Sun who is reportedly preparing to take a cannonball leap into the networking pool.

Cisco will add blade servers to their lineup, that much is true. But these new servers are meant for virtualization only, using VMware and Microsoft Hyper-V. Suddenly the integration of Cisco’s technology into VMware in the form of the Nexus vSwitch makes much more sense.

They’ll be a solid alternative in the VMware server market, to be sure, but what will the long-term effect be? Certainly, they reveal Cisco’s approach to the new virtual datacenter world, “Data Center 3.0“. Rather than bothering with a soup-to-nuts line of servers, Cisco will focus where the most value is for integrated systems: high-end virtual servers. This fits well into their traditional network-focused strategy, as well as VMware’s vision of data center evolution, vSphere.

So is this introduction just a bunch of hot air? Not at all! The UCS includes some innovative technology that separates it from the pack:

• I/O is converged using 10 GbE with (pre-standard) Data Center Bridging technologies
• The blade servers are beefy, and the 7-server plus 1-vSwitch architecture is scalable
• A new management platform (UCS Manager) integrates blade server hardware and network provisioning and configuration
• It leverages enterprise storage (EMC and NetApp are happy!) with fabric extenders

Is it a slam dunk? No way! Cisco faces some serious challenges:

• Do companies want to integrate networking and servers politically and organizationally?
• It looks solid (with backing from VMware, Intel, and Accenture), but how well will it be supported at the customer side?
• How will Cisco partners and new partial-competitors, HP, Dell, and IBM react to this challenge?
• Will the 10 GbE DCB technologies that Cisco is leveraging really scale in the real world?

We shall see…

© Stephen Foskett for Gestalt IT, 2009. | Cisco Enters the Virtual Server Hardware Market
Read more posts categorized as Featured, Server Virtualization

When you decide to maintain a server’s existing iSCSI partitions as a VM, there are several configuration considerations to plan for.

Multipathing Support for iSCSI is no longer needed in the VM

When you were configuring the iSCSI initiator chances are you used two physical network interface cards (NICs) for a redundant connection in the server operating system to the storage. You then used the NIC manufacturer’s drivers/management software to create a team and a virtual ip address. Your SAN was configured to allow an iSCSI initiator to connect via that NIC team virtual ip address.

As a VM that same team ip address will probably still be maintained as the initiator, but the need for two NICs and the former manufacturer’s drvers and software will be removed. The VM only needs a single vNIC to the iSCSI storage. The virtual host should be configured with a vSwitch mapped to two pNICs. Therefore the virtual host provides the redundant connection to the storage.

Be sure to remove the team configuration and the old NIC drivers and software.

Dedicate a vSwitch with it’s own pNICs for the VM iSCSI traffic

Separate the VM’s iSCSI traffic from the virtualization host’s iSCSI traffic. You could add an extra portgroup to your iSCSI vSwitch in VMware ESX for example, but ideally, you want 2 NICs dedicated to the host, and 2 other NICs dedicated to the VM(s). This requires separate vSwitches. This will maximize performance and provide redundancy.  

Consider the cables needed to the SAN switches

Before P2V each server needed 2 cables to the storage switches for redundancy. After P2V, each virtualization host will need 4 cables. Two of the cables will be replaced by the connection to the host’s dedicated LUNs where the VM’s operating systems and and other partitions are encapsulated. The second set of two cables will be for the VM’s initiator to access it’s own iSCSI partitions. 

Disconnect the iSCSI initiator before P2V

This is not a must do, but rather a safety net for the P2V migration process. Disconnecting the server’s iSCSI initiator ensures the LUNs you need to maintain will not be selectable as disks to be converted during the migration.

Be prepared to recreate any file shares and permissions

If you disconnect the iSCSI initiator as previously mentioned then be prepared to recreate any file permissions and shares that were configured. To be honest, I am not sure of the best way to prepare for this or if it’s even necessary, but in my experience I have had to recreate shares. Thank goodness it was never a complex user or department hierarchy as you can imagine the impact and administrator time needed that overlooking this would cause.

Check out this VMTN Communities thread on this topic too: VMware Communities: P2V when server has a LUN through iSCSI? …

© Rich for Gestalt IT, 2009. | P2V strategy for a Physical Server with an iSCSI Partition
Read more posts categorized as Server Virtualization