The PXE environment components included with the VCVA are not configured and turned off by default. So, if you’re ready to configure your VCVA for PXE, time to roll up your sleeves, crack those knuckles, and get ready to get your hands dirty.

Before we get started, though, and little caution (and disclaimer so I can sleep better at night):

I know nothing about your environment. You are following these instructions at your own risk. This setup will impact DHCP functionality on your network. Follow these instructions at your own risk and make the appropriate adjustments to work in your environment.
Additionally, I do not know everything about everything. So, you are going to need to rely upon your sleuthing abilities to help resolve issues that may arise.

These instructions assume some knowledge of CLI-based file editing (vi). So, please research how to use it if you are unsure.

Overview

A PXE environment via the VCVA requires the following components in your environment

• DHCP server
• TFTP server
• Web Server (for kickstart scripts)
• SYSLINUX (for pxeboot.0)
• Access to an ESXi 5.0 installation CD (perhaps you created on using my Image Builder tutorial)
• vCenter Virtual Appliance deployed
• Blank server to PXE boot and install ESXi 5.0 on (aka – the client)
• ESXi 5.0 installation .ISO
• HTTP server on the network (for hosting kickstart files – customization during installation)

For this exercise:

• Network: 192.168.226.0/24
• VCVA: 192.168.226.21
• DHCP Range: 192.168.226.200 – 254
• Default Gateway: 192.168.226.1

Configuration

0 – Log into the appliance as ‘root’

1 – Configure DHCP

‘dhcpd‘ will listen to IP address requests, provide an IP to use, direct the client to the “next-server” to continue PXE booting, and which file (filename) to download from the server.

• cd /var/lib/dhcp/etc
• cp -a dhcpd.conf dhcpd.conf.orig
• vi dhcpd.conf

Once inside of the file, ensure the following exists (highlighted for your ease of identification)

ddns-update-style ad-hoc;
allow booting;
allow bootp;

#gPXE options
option space gpxe;
option gpxe-encap-opts code 175 = encapsulate gpxe;
option gpxe.bus-id code 177 = string;
class “pxeclients”{
match if substring(option vendor-class-identifier, 0, 9) = “PXEClient”;
next-server 192.168.226.21;
filename “pxelinux.0″;
}
subnet 192.168.226.0 netmask 255.255.255.0 {
range 192.168.226.200 192.168.226.254;
}

Save the file and exit (hint: :wq)

2 – Configure TFTP

TFTP services are provided by the ‘atftpd’ daemon

• cd /etc/sysconfig
• cp –a atftpd atftpd.orig
• vi atftpd

Once inside the file adjust the “ATFTP_OPTIONS” line to read: “–daemon –user root”. Typically, the atftpd daemon runs as ‘nobody’. However, the TFTP root (/tftpboot/) is configured as owned by the ‘root’ user.

Save and exit the file.

3 – Get the SYSLINUX packages on the server

There is one package missing to make the PXE installation process work: ‘pxelinux.0′. ‘pxelinux.0‘ is an executable that is downloaded by the client in order to properly continue the PXE process (aka – download the files, execute the installer, etc…). ‘pxelinux.0‘ is provided by the SYSLINUX package. In order for PXE to work properly with the ESXi 5.0 installation, SYSLINUX version 3.86 (or higher) is needed.

• cd /tmp
• wget http://www.kernel.org/pub/linux/utils/boot/syslinux/3.xx/syslinux-3.86.tar.gz
• gunzip syslinux-3.86.tar.gz
• tar xvf syslinux-3.86.tar

Note: you can use YUM or copy the files to the server another way if you’d like. Regardless, get the files there. This example will continue to use the /tmp file as the landing area for the SYSLINUX files.

Copy the pxelinux.0 file to your TFTP root

• cp /tmp/syslinux-3.86/core/pxelinux.0 /tftpboot

4 – Prep the TFTP root for PXE

The TFTP root configured on the VCVA is located at /tftpboot. We are going to need to get the directory structure built out to support PXE.

• cd /tftpboot
• mkdir esxi50

By adding a directory, we are able to organize the TFTP server and support additional versions of ESXi going forward.

5 – Get the ESXi 5.0 CD contents onto the server

Seeing as the VCVA is a virtual appliance, it is easy to get the contents of the installation media onto the server.

• Mount the installation CD to the VCVA as a CD-ROM drive using the vSphere Client.
• mount /dev/cdrom /media
• cp –a /media* /tftpboot/esxi50/
• umount /dev/cdrom

6 – Configure PXELINUX

pxelinux is the utility that enables the PXE functionality. As mentioned before, pxelinux.0 is an executable that the server downloads. The executable provides functionality to parse a menu system, load kernels, options, customizations, modules, etc…, and boot the server. Since PXE can be used by multiple physical servers for multiple images, we need to configure pxelinux for this specific image.

• cd /tftpboot
• mkdir pxelinux.cfg
• cd pxelinux.cfg

pxelinux.0 looks for configuration files in the TFTP:/pxelinux.cfg directory.

pxelinux looks for a large number of configuration files… specific to a default/generic value. This allows server administrators to define a file based on a complete MAC address, partial MAC address, or none at all to determine which image to boot from. Since this is the first configuration on the VCVA, we are going to configure a default. Do your research if you want to adjust this from the default value.

The installation media contains a file called isolinux.cfg. We can use this as the basis for our file called ‘default’. Copy it from the installation media and start customizations:

• cp –a /tftpboot/esxi50/isolinux.cfg default
• chmod a+w default
• vi default

Ensure the appropriate lines match the following lines:

DEFAULT /esxi50/menu.c32
KERNEL /esxi50/mboot.cfg
APPEND -c /esxi50/boot.cfg

Save and Exit

7 – Configure the Kickstart file

Using a kickstart file, we can configure ESXi 5.0 automatically during installation. This requires that a file be placed on a server that is available to the client.  Sadly, the HTTP areas on the VCVA are not readily available… and, they may be erased during future upgrades. So, we need to use an external HTTP server somewhere on your network. (Note: NFS and FTP are options as well).

Add the following contents:

# Accept the EULA
vmaccepteula

#Set root password
rootpw supersecretpassword

#Install on first local disk
install –firstdisk –overwritevmfs

#Config initial network settings
network –bootproto=dhcp –device=vmnic0

 

In this example, we are saving the file to:

• HTTP Server: 192.168.226.1
• Directory: ks
• File: esxi50.txt
• URL: http://192.168.226.1/ks/esxi50.txt

8 – Configure the installation files

The CD installation media for ESXi 5.0 assumes a single installation point. Thus, all the files are placed at the root of the image. However, since we want to actually organize our installation root, we added the ‘/tftpboot/esxi50‘ directory and copied the files into it. We need to adjust the installation files in /tftpboot/esxi50 to reflect the change.

• cd /tftpboot/esxi50
• cp -a boot.cfg boot.cfg.orig
• vi boot.cfg
• Using the following picture as reference, add “/esxi50” to the paths for ‘kernel’ and ‘modules’

Save and quit

9 – Restart services to load the service configurations and configure to start with server

• /etc/init.d/dhcpd restart
• /etc/init.d/atftpd restart
• chkconfig –add dhcpd
• chkconfig –add atftpd

10 – Take a break

You made it this far… great job. At this time, we have configured DHCP, TFTP, pxelinux, copied installation media to the TFTP root, and configured the installation for our organizational purposes.

11 – Start your host and install away

[BELOW] Reading the Kickstart Script. No need to enter customization info anymore.

[BELOW] Checking contents of Kickstart file. You will see errors here if errors in file.

© Bill for Gestalt IT, 2011. | vSphere 5–PXE Installation Using vCenter Virtual Appliance
Read more posts categorized as All, Featured, Server Virtualization, Top Story

For a great overview of the entire U1 release check out Rick Vanover’s post vSphere 4.1 update 1 released from his Rickatron Blog and via his Servers and Storage Column/Blog at TechRepublic.

The rest of this post contains cut and pastes from the Release Notes and some commentary about them from me. I want to stress again that these are issues that have now been fixed!

Finally, I’ll point out the one huge VM backup issue (that I can think of right now) that still does not appear to be resolved.

• Cannot take quiesced snapshots of Microsoft Windows Server 2008 R2 virtual machine running vCenter Server 4.1 When creating a snapshot of a Microsoft Windows Server 2008 R2 virtual machine that has vCenter Server 4.1 installed, the snapshot operation might fail to complete. This issue occurs on Microsoft Windows Server 2008 R2 virtual machines when the ADAM database is installed. The issue is resolved in this release.

The change from 2008 to 2008 R2 has caused quite a few applications to stumble, so it is good to see that VMware users can now install vCenter on the latest and greatest OS from Microsoft running as a virtual machine without having to worry about various hassles, not to mention coming up with a separate backup or business continuity solution for arguably one of the most important VMs in the environment.

• Creation of large .vmdk files on NFS might fail. When you create a virtual disk (.vmdk file) with a large size, for example, more than 1TB, on NFS storage, the creation process might fail with an error: A general system error occurred: Failed to create disk: Error creating disk. This issue occurs when the NFS client does not wait for sufficient time for the NFS storage array to initialize the virtual disk after the RPC parameter of the NFS client times out. By default the timeout value is 10 seconds. This fix provides the configuration option to tune the RPC timeout parameter using the esxcfg-advcfg -s <Timeout> /NFS/SetAttrRPCTimeout command.

Although specific to users of NFS storage, this one was definitely a head scratcher. Possible scenarios where this problem could rear it’s ugly head include full VM restores and even VM replication. Add the possibility of dedicating a large .vmdk as the backup repository for your backup server VM.

Let me know if I missed any other resolved issues specific to VM backups.

The issue still unresolved

Unfortunately, nothing in vSphere 4.1 U1 appears to address the loss of connectivity when committing snapshots for VMs using Change Block Tracking (CBT) while running on NFS storage.  This leaves VMware admins stuck between a rock and hard place – disable CBT or move VMs off of NFS. More about the problem can be found in VMware’s KB Article:

Virtual machine freezes temporarily during snapshot removal on an NFS datastore in a ESX/ESXi 4.1 host

© Rich for Gestalt IT, 2011. | vSphere 4.1 U1 Released. Fixes Specific For VM Backups
Read more posts categorized as All, Server Virtualization

VMware Auto Deploy

The auto deploy application which comes as an OVT template is basically just a jumped-up vMA, with the added extra’s of DHCP, TFTP, HTTP servers and a deploy-cmd CLi and Database.

Here is a brief overview of how VMware Auto Deploy works once configured:

• PXE boot the target server
• ESXi is installed onto the target server from the auto deploy app
• The ESXi host will then be added into your vCenter
• The ESXi host will then have a Host Profile applied to it.

This makes life pretty easy.

What I didn’t know and it didn’t mention on the labs site was that the ESXi install was Stateless. The ESXi install is only held in memory. So if you reboot the server you’d see a “No Operating System Found” message.

Before VMware Auto Deploy, I hadn’t ever given Stateless ESXi a second thought. The configuration of a host once ESXi was installed was a lot more detailed than the initial install itself and took time to complete. Now with the use of Host Profiles we are able to Install and Configure an ESXi host within a matter of minutes and 100% automatically. At this rate I’ll be doing myself out of a job! However, I also believe it’s the way most large deployments will head in the not so distant future. We are beginning to see an increase in the amount of diskless servers/blades coming onto the market, which is ushering us down the route of using Stateless installs.

I’m not going to go into the in’s and out’s of the application configuration as it’s all available in the VMware Auto Deploy Administrator’s Guide. It’s very simple.

© Simon for Gestalt IT, 2010. | VMware Auto Deploy – Stateless ESXi
Read more posts categorized as All, Server Virtualization

This is something I did a while ago, but it came to my attention that people didn’t; a) Know that it’s recommended to use a syslog server with ESXi b) You could use an application built in to vMA called vilogger.

Although it is stated in The Architecture of VMware ESXi…..

Because the in-memory file system does not persist when the power is shut down, log files do not survive a reboot. ESXi has the ability to configure a remote syslog server, enabling you to save all log information on an external system.

…..it is not a well known fact. So that is partly the reason for writing the post. The other reason is to introduce you to vilogger, which is part of the vMA. Of course you can use which ever syslog server you wish,  if you plan to use your own, be sure to checkout Managing VMware ESXi page #68 to view the configuration steps.

I’m not going to take you through the steps of installing vMA, nor am I going to tell you all about what the vMA (vSphere Manage Assistant) does. If you want to read more about that please find the relevant links in the Sources section at the bottom of the page. But I am going to take you through the steps I took to use vMA as my ESXi syslog server.

First of all, download the vMA from here. Import and configure it using the steps in the vSphere Management Assistant Guide

Changing vMA localtime settings to avoid incorrect log timestamps

When vMA collects the logs from your ESXi Host, sometimes the logs have the ESXi Host timestamp and sometimes they will have the vMA Localtime timestamp. I’m not exactly sure why this happens, but it does. (You may or may not know that ESXi uses UTC as its timezone when it timestamps the logs. You can read more about that here. VMware have told me this cannot be changed.)

So to get around the issue of differing timestamps in your collected logs we need to change the localtime on the vMA to UTC. This can be done using the following steps:

• sudo rm /etc/localtime
• sudo ln -s /usr/share/zoneinfo/UTC /etc/localtime

If you use NTP to sync your environments time, it might be worth adding in your ntp servers onto the vMA.

• sudo nano /etc/ntp.conf
• Add in your ntp servers under the heading: # Use public servers from the pool.ntp.org project.
• Configure ntpd to start on reboot: sudo /sbin/chkconfig ntpd on
• Restart ntpd: sudo /sbin/service ntpd restart
• Make sure your ntp servers are reachable: sudo ntpq -p

Whilst your at it you could change your keyboard for those of you outside the US.

• sudo vi /etc/sysconfig/keyboard
  • change KEYTABLES=”us” to use the keyboard you have, for example: KEYTABLES=”en”

Adding a second Hard Disk to store your logs

Because of the amount of ESXi Hosts I have to manage I decided that I would add a second Virtual Disk to house my logs.  You may not need to do this if you only have 1 or two Hosts.

• Add a second Hard Disk to the vMA VM
• PowerOn vMA
• Login using vi-admin
• Format the new Disk with the following command: fdisk /dev/sdb
  • Use the n command to create a new partition
  • Use the p command to make the new partition a primary partition
  • Press 1 to make it partition #1
  • Use the default for the First Cylinder
  • Use the default for the Last Cylinder
  • Use the p command to verify the partition table
  • Use the w command to write the partition table to the hard disk
• Press Enter a couple of times to confirm it’s finished and return to the command prompt
• Now we need to format the partition using the follow command: mkfs -t ext3 /dev/sdb1

Now we have our new disk ready, we now need to mount it.

• Edit /etc/fstab using the following command: nano /etc/fstab
• Enter in the following line: /dev/sdb1               /var/log/syslog            ext3    defaults,auto        1 2
• Use Cntrl+X then Y to save the file
• Next: cd /var/log/ then mkdir syslog
• finally mount /var/log/syslog to mount the disk

Ok, now our new Disk is mounted, we need to tell the vilogger application to store your logfiles there. The default location is /var/log/vmware so we need to change it.

• Edit the vilogger config file using: nano /etc/vmware/viconfig/vilogdefaults.xml
• Change the file to match this: <location>/var/log/syslog</location> (This appears twice, change both)
• Use Cntrl+X then Y to save the file
• Restart vilogger by: vmware-vilogd restart

Now vilogger is set to store your ESXi logfiles onto your new disk.

Configure vMA to collect your logs

Next we configure vMA and vilogger to collect your ESXi logfiles.

• Connect your ESXi Host to vMA using the following command: sudo vifp addserver <FQDN of ESXi Host>
• Enter the Root Password for the Host
• Once the Host has been added you can double check by running the following command: vifp listservers
• Now set vilogger to start collecting your logs: vilogger enable –server <FQDN of ESXi Host> –numrotation 20 –maxfilesize 10 –collectionperiod 10

You will see the above displayed, showing that vilogger is now collecting logs from your  Host.

Values –numrotation 20 –maxfilesize 10 –collectionperiod 10 can be changed to suit your needs. If you just run the vilogger –help command you will see what options and values are available.

Viewing your logs

Now your vMA is collecting your logs you can view them by: dir /var/log/syslog/<FQDN of Host> You can view your logs in real time if you wish, read more about there here.

• An example: tail -f /var/log/syslog/<FQDN of Host>/vpxa.log

Sources:

• Setting timezone in ESXi or not
• The Architecture of VMware ESXi
• vSphere Management Assistant Guide
• VMware KB1007551

© Simon for Gestalt IT, 2010. | Using vMA As Your ESXi Syslog Server
Read more posts categorized as Server Virtualization

Just a quick post about using ESXTOP with VMware ESXi. Obviously in ESXi there is no Service Console so we have to use the vMA (vSphere Management Assistant) to help us. If you haven’t installed the vMA on your infrastructure yet, you can download it here: http://www.vmware.com/support/developer/vima/

Once install and configured, login and run the following command: resxtop –server <server name>

You will be prompted to login, use the root user/pass of the Host you want to run ESXTOP on (Note: Logging in as root will not work if the Host is in “Lockdown Mode“.). You should then be presented with ESXTOP, I believe it has all of the same function as it did in the Service Console

Here you can see all avaliable options when connecting to a Host using RESXTOP

usage: resxtop [-h] [-v] [-b] [-s] [-a] [-c config file] [-d delay] [-n iterations]
               [--server server-name [--vihost host-name]] [--portnumber socket-port] [--username user-name]
              -h prints this help menu.
              -v prints version.
              -b enables batch mode.
              -s enables secure mode.
              -a show all statistics.
              -c sets the esxtop configuration file, which by default is .esxtop4rc
              -d sets the delay between updates in seconds.
              -n runs resxtop for only n iterations.
              --server      remote server name.
              --vihost      esx host name, if --server specifies vc server.
              --portnumber  socket port, default is 443.
              --username    user name on the remote server.
       for more information on interactive and batch modes
       please see man page for resxtop.

© Simon for Gestalt IT, 2010. | Using ESXTOP With VMware ESXi
Read more posts categorized as Server Virtualization

Failing Disk

I started by shutting down and moving my production ESXi Server out and back into the standard rack it occupies.  On power up, the server failed to reboot, claiming the boot disk was no longer present.  A quick check inside showed that the SAS connector on the boot disk had come loose, so I plugged it back in and tried again (Oh, SAS specification guys – bad design, no retainers on the plugs).  Unfortunately, the boot disk had somehow become corrupted and the server wouldn’t come up.  No problem, I thought, just repair using the installation media.  This is where things started to get complicated.

My ESXi server runs off a Seagate Savvio 2.5″ 15K 73GB drive, one of four Seagate generously loaned me last year for long term testing.  More on that another day.  The server has two disks installed, one of which has VMs on it.  During the repair process I wasn’t sure which disk was the O/S and which was data.  ESXi doesn’t help much, only indicating that both disks contained data in partitions, data that would be lost if I reinstalled.

Lesson 1 – Make sure you know exactly how your hardware is configured, down to the SAS ports each drive is plugged into.

Actually having multiple drives of the same type is a pain.  So rather than risk data loss, I removed both drives and re-installed the ESXi O/S from a third Savvio drive.  All good.  Now I need to locate and import all my VMs, however some were on the removed Savvio disks.  This meant installing each disk independently and checking the contents to determine which contained VMs and which contained the broken O/S.

Lesson 2 – Wherever possible, place your VMs on disks separate from the server itself.

Yes, I do have most of my VMs on my Iomega ix4-200d, but, rather crucially, not my Windows 2008 AD Server, which needed to be moved from internal disk to the ix4 before I continued (schoolboy error there).  The AD server was rather important for accessing my, ahem, ix4, which is configured to validate logins using AD.  This creates a bit of a circular reference which could have been a disaster.

Lesson 3 – Place your Windows domain controller on a physical server, or have another independent backup elsewhere.

Having a physical server just for AD control isn’t part of my total virtualisation plan, so I’m looking at whether I can host a backup controller with Amazon AWS and use VPN to secure it into my private network.  This way, if I ever have an issue, I can still authenticate.  The issue of course is cost, which may make a dedicated server the cheaper option.

So, by 1am everything was back up and running.  Did I learn anything else?  Well yes…

Lesson 4 – after 22 years in IT, I should remember that adequate documentation and a DR plan are crucial.  In fact, in a virtualised environment, they are essential due to the concentration of risk placing all systems on a single server causes.

So what next for my virtual infrastructure?  I have a few changes planned; I’ll create a backup ESXi server that can import and run the VMs in the event of a future server failure.  I will also be investigating AWS with Windows 2008 and VPN to create a backup domain controller and see if I can continue to work if both server’s hardware failed.

That leaves one Single Point of Failure… my ix4-200d.  Anyone want to donate me a spare one?

© Chris for Gestalt IT, 2010. | Virtualisation: Learning The Hard Way
Read more posts categorized as Featured, Server Virtualization, Storage

This is the second in a series of posts looking at the DroboPro from Data Robotics Inc.  Previous post(s):

Review: DroboPro – Part I

Previously I discussed a few frustrations with connecting my new DroboPro to ESXi, the target environment for my new device.  I’ve now managed to get the ‘Pro connected and visible within ESXi.  See the attached screenshot image taken from my production ESXi server.

So far, I’ve configured eight 2TB LUNs (although the DroboPro itself only has a capacity of 5TB, 2x 2TB and a single 1TB drive) which are numbered 0-7; the LUN column on the screenshot.  Compare this to the LUNs presented from my ix4-200d, which has three LUNs all numbered ‘0′.  The difference is in the way the DroboPro is choosing to present iSCSI devices and the Runtime Name column gives a clue.  In standard Controller/Target/LUN or CxTyLz/CxTyDz methodology, the ix4 disks are using separate targets to map out each iSCSI LUN, with the LUN number always ‘0′.  The Target setting is analagous to a storage port, typically a physical port on large fibre channel arrays.  The LUNs are then numbered on that port.  The DroboPro presents all its iSCSI LUNs on the same target (in this instance number 3) and so the LUNs are numbered from 0.

This method of presentation doesn’t necessarily cause a problem, but is just a different way of presenting the LUNs.  Presumably with the Drobo Elite the multi-user functionality is achieved by enabling multiple targets per Drobo, each mapped to a separate user.  In any case, I now have visibility of my DroboPro within ESXi.  Time, to write some data.

© Chris for Gestalt IT, 2009. | Review: DroboPro – Part II
Read more posts categorized as Featured, Server Virtualization, Storage

Since I started working with VMware vSphere (now officially available as of 5/21/2009), I have been evaluating how to replicate the same sort of setup using ESX/ESXi 4.0. For the most part, the configuration of VMkernel ports to use jumbo frames on ESX/ESXi 4.0 is much the same as with previous versions of ESX and ESXi, with one significant exception: the vNetwork Distributed Switch (vDS, what I’ll call a dvSwitch). After a fair amount of testing, I’m pleased to present some instructions on how to configure VMkernel ports for jumbo frames on a dvSwitch.

How I Tested

The lab configuration for this testing was pretty straightforward:

• For the physical server hardware, I used a group of HP ProLiant DL385 G2 servers with dual-core AMD Opteron processors and a quad-port PCIe Intel Gigabit Ethernet NIC.
• All the HP ProLiant DL385 G2 servers were running the GA builds of ESX 4.0, managed by a separate physical server running the GA build of vCenter Server.
• The ESX servers participated in a DRS/HA cluster and a single dvSwitch. The dvSwitch was configured for 4 uplinks. All other settings on the dvSwitch were left at the defaults.
• For the physical switch infrastructure, I used a Cisco Catalyst 3560G running Cisco IOS version 12.2(25)SEB4.
• For the storage system, I used an older NetApp FAS940. The FAS940 was running Data ONTAP 7.2.4.

Keep in mind that these procedures or commands may be different in your environment, so plan accordingly.

Physical Network Configuration

Refer back to my first article on jumbo frames to review the Cisco IOS commands for configuring the physical switch to support jumbo frames. Once the physical switch is ready to support jumbo frames, you can proceed with configuring the virtual environment.

Virtual Network Configuration

The virtual network configuration consists of several steps. First, you must configure the dvSwitch to support jumbo frames by increasing the MTU. Second, you must create a distributed virtual port group (dvPort group) on the dvSwitch. Finally, you must create the VMkernel ports with the correct MTU. Each of these steps is explained in more detail below.

Setting the MTU on the dvSwitch

Setting the MTU on the dvSwitch is pretty straightforward:

1. In the vSphere Client, navigate to the Networking inventory view (select View > Inventory > Networking from the menu).
2. Right-click on the dvSwitch and select Edit Settings.
3. From the Properties tab, select Advanced.
4. Set the MTU to 9000.
5. Click OK.

That’s it! Now, if only the rest of the process was this easy…

By the way, this same area is also where you can enable Cisco Discovery Protocol support for the dvSwitch, as I pointed out in this recent article.

Creating the dvPort Group

Like setting the MTU on the dvSwitch, this process is pretty straightforward and easily accomplished using the vSphere Client:

1. In the vSphere Client, navigate to the Networking inventory view (select View > Inventory > Networking from the menu).
2. Right-click on the dvSwitch and select New Port Group.
3. Set the name of the new dvPort group.
5. Set the number of ports for the new dvPort group.
6. In the vast majority of instances, you’ll want to set VLAN Type to VLAN and then set the VLAN ID accordingly. (This is the same as setting the VLAN ID for a port group on a vSwitch.)
7. Click Next.
8. Click Finish.

See? I told you it was pretty straightforward. Now on to the final step which, unfortunately, won’t be quite so straightforward or easy.

Creating a VMkernel Port With Jumbo Frames

Now things get a bit more interesting. As of the GA code, the vSphere Client UI still does not expose an MTU setting for VMkernel ports, so we are still relegated to using the esxcfg-vswitch command (or the vicfg-vswitch command in the vSphere Management Assistant—or vMA—if you are using ESXi). The wrinkle comes in the fact that we want to create a VMkernel port attached to a dvPort ID, which is a bit more complicated than simply creating a VMkernel port attached to a local vSwitch.

Disclaimer: There may be an easier way than the process I describe here. If there is, please feel free to post it in the comments or shoot me an e-mail.

First, you’ll need to prepare yourself. Open the vSphere Client and navigate to the Hosts and Clusters inventory view. At the same time, open an SSH session to one of the hosts you’ll be configuring, and use “su -” to assume root privileges. (You’re not logging in remotely as root, are you?) If you are using ESXi, then obviously you’d want to open a session to your vMA and be prepared to run the commands there. I’ll assume you’re working with ESX.

This is a two-step process. You’ll need to repeat this process for each VMkernel port that you want to create with jumbo frame support.

Here are the steps to create a jumbo frames-enabled VMkernel port:

1. Select the host and and go the Configuration tab.
2. Select Networking and change the view to Distributed Virtual Switch.
3. Click the Manage Virtual Adapters link.
4. In the Manage Virtual Adapters dialog box, click the Add link.
5. Select New Virtual Adapter, then click Next.
6. Select VMkernel, then click Next.
7. Select the appropriate port group, then click Next.
8. Provide the appropriate IP addressing information and click Next when you are finished.
9. Click Finish. This returns you to the Manage Virtual Adapters dialog box.

From this point on you’ll go the rest of the way from the command line. However, leave the Manage Virtual Adapters dialog box open and the vSphere Client running.

To finish the process from the command line:

1. Type the following command (that’s a lowercase L) to show the current virtual switching configuration:
   esxcfg-vswitch -l
   At the bottom of the listing you will see the dvPort IDs listed. Make a note of the dvPort ID for the VMkernel port you just created using the vSphere Client. It will be a larger number, like 266 or 139.
2. Delete the VMkernel port you just created:
   esxcfg-vmknic -d <dvPort ID>
3. Recreate the VMkernel port and attach it to the very same dvPort ID:
   esxcfg-vmknic -a -i <IP addr> -n <Mask> -m 9000 <dvPort ID>
4. Use the esxcfg-vswitch command again to verify that a new VMkernel port has been created and attached to the same dvPort ID as the original VMkernel port.

At this point, you can go back into the vSphere Client and enable the VMkernel port for VMotion or FT logging. I’ve tested jumbo frames using VMotion and everything is fine; I haven’t tested FT logging with jumbo frames as I don’t have FT-compatible CPUs. (Anyone care to donate some?)

As I mentioned in yesterday’s Twitter post, I haven’t conducted any objective performance tests yet, so don’t ask. I can say that NFS feels faster with jumbo frames than without, but that’s purely subjective.

Let me know if you have any questions or if anyone finds a faster or easier way to accomplish this task.

This article was originally posted on blog.scottlowe.org. Visit the site for more information on virtualization, servers, storage, and other enterprise technologies.

© Scott for Gestalt IT, 2009. | VMware vSphere vDS, VMkernel Ports, and Jumbo Frames
Read more posts categorized as All, Networking, Server Virtualization, Storage

Fortunately the guys from the Fishworks team had made contact and I can ask them questions. They suggested reducing the number of virtual disks, so I’ve dropped down to two disks now. One for the Fishworks binaries and one to store data. This has made all the difference, it now works and doesn’t die.

So people, if you are actually trying to use the Fishworks simulator to do useful stuff, you might want to consider reducing the number of virtual disks; I might try increasing the number of virtual disks and putting them on discrete spindles but I don’t have 15 disks to play with; so I might try 3+1.

As per usual, I get all my VSAs built then both Nexenta and EMC release new versions. And someone has found me another VSA to play with. Falconstor seem to be ignoring my attempts to register for their free version.

At moment, I’m ignoring the VSAs which provide features like backups, dedupe but VSAs seem to be really taking off. I do like the concept of providing discrete infrastructure capabilities as a bundled virtual appliance and at some point I shall branch out into looking at some of these as well.

© Martin for Gestalt IT, 2009. | Fishworks Simulator Hint and other VSA stuff
Read more posts categorized as All, Storage