When designing VDI environments, a couple of things that you should consider are the .vswp file and the GuestOS paging file. In this article I am going to focus on the Paging file and hopefully in the not so distant future I will write a post about the .vswp file in a VDI environment.

What is point of the paging file (also known as the pagefile.sys)?

RAM is a limited resource. Virtual memory was introduced to help remove that limit.

Most modern operating system now use Virtual Memory. Virtual memory is a memory management technique. Applications running on a GuestOS reference memory using virtual memory addresses which are then automatically translated into RAM addresses by the hardware. These virtual memory address spaces are divided in pages or block, usually of 4KB.

If RAM resource is exhusted, the operating system will move 4KB pages of the virtual memory onto the computers hard disk to free up the physical memory (RAM) for other processes. In Windows operating systems, these pages are stored in the pagefile.sys.

A good way to think of this is;

Imagine a restaurant that has just open for the evening. When customers (Processes) arrive they get allocated a table (RAM) to sit and eat at. As the night goes on the restaurant get busier and free tables (RAM) begin to run out for the new customers (Processes) coming through the door. To free up spare tables (RAM) the waiter asks customers (Processes) who have finished eating if they wouldn’t mind moving to the bar (Virtual Memory) where they can continue drink.

Without the paging file, if the physical memory becomes full, applications including the operating system will have to waiting until physical memory becomes available before it can be stored in RAM ready for the CPU to process. As you can imagine this causes massive performance problems.

In summary, you NEED to have a paging file.
Read the rest of this entry »

© Simon for Gestalt IT, 2012. | VDI Paging Files – Big? Small? Or Non At All?
Read more posts categorized as Desktop, Server Virtualization

When testing this, customers can often see unexpected results. This is often due to their misconception of how this function of View works. 

Why? View 4.5 looks for the datastore/s that have the highest "weighted available space". The formula that View 4.5 uses to calculate this is:

weighted_available_space = datastore_capacity * overcommit_factor – virtual_usage

datastore_capacity = As you would expect, the size of the datastore.

overcommit_factor = The Storage Overcommit setting applied to each datastore when selecting which datastores to use for your desktop pools.

• None - Storage is not overcommitted.
• Conservative - x4 the size of the datastore. (This is the default level.)
• Moderate - x7 the size of the datastore
• Aggressive - x15 the size of the datastore.

virtual_usage = The maximum possible usage of all existing VMs on this datastore, which is calculated by adding up current sizes of all disks and configured sizes of memories of all existing VMs on the datastore.

Examples

• DS1 - 1000GB (Datastore Size) * 4 (Conservative Overcommitment) – 0 (No VM's deployed) = 4000
• DS2 - 1000GB (Datastore Size) * 4 (Conservative Overcommitment) – ((20GB + 130MB)x5) (5 VM's already deployed) = 3865
• DS3 - 1000GB (Datastore Size) * 7 (Moderate Overcommitment) – ((20GB + 130MB)x5) (5 VM's already deployed) = 6865

Common Scenario

You have a desktop pool with DS1 as it's only datastore. You then decided to add a second datastore, DS2. You perform a Rebalance on the pool to spread the VM's across the two datastores, but nothing happens… Why? Because DS2's weighted_available_space is less than that of DS1's.

• If you increased the Overcommit level on DS2 to Moderate (x7) and ran Rebalance, the VM's would all migrate completely over to DS2 because it's weighted_available_space would be more than that of DS1 even though there are already 5 VM's on DS2. (See DS3 example)
• If you Storage vMotioned the 5 VM's off of DS2 and ran Rebalance, the VM's in the pool should evenly distribute themselves across both DS1 and DS2.

Summary

As you can see, by default you may not get your desired effect. But with a little bit of maths you can get the reaction you want from the Rebalance feature. I must stress, this is only for VMware View 4.5 and older. 

© Simon for Gestalt IT, 2011. | VMware View 4.5: Rebalance
Read more posts categorized as All, Desktop, Server Virtualization

From the information below, I would personally recommend using the following:

• Windows XP: LSI Logic Parallel or SAS
• Windows 7: LSI Logic SAS

The PVSCSI controller will be the controller of the future so you may also want to keep this in mind. Of course you can make your own decisions. Most of the points made are desktop focused, however most will apply outside of a View environment.

IDE vs SCSI

Most people’s main argument around reasons to not use IDE are;

• The IDE adapter completes one command at a time – SCSI can queue commands.
• The SCSI adapter is better optimized for parallel performance.
• If you ever need to increase the size of an IDE Disk you would have to use VMware Converter to enlarge it. – Using SCSI it’s easier
• Maximum of 4 IDE Devices per VM. (This includes CDROM) – SCSI allows 60.

BusLogic vs LSI Logic

If you have decided to use SCSI, you now have to decide on whether to use the BusLogic or LSI Logic SCSI controller.

• BusLogic is not supported in a lot of the newer Operating Systems.
• BusLogic is not actively being improved by VMware or Guest OS’s (so bugs come along as things change in the OS and noone fixes them).
• XP doesn’t contain the LSI Logic driver so you will have to load it into the OS manually.
• “Out of the box” LSI out performs BusLogic. With a few additional configuration changes to the Queue Depth, BusLogic can perform on a par with LSI in VM’s that have less than 4GB of RAM – BusLogic is a 32 bit PCI device and is limited to DMA (Direct Memory Access) in the first 4GB of the guest memory, so if the guest uses more than 4GB memory the OS will need to use bounce buffering (doing IO to lower memory and copy up/down with the CPU), which adds overhead

*VMware’s recommendation is to use the LSI Logic adapter. “The LSI Logic adapter has improved performance and works better with generic SCSI devices.”

LSI Logic SAS

The default controller for a Windows 7 virtual machine is the LSI Logic SAS. There is no performance between this controller over the regular LSI Logic Parallel controller, so I therefore see no reason to change it.

It is worth noting, in Linux guests SCSI disk hotplug works better with SAS than with original parallel LSI. (You do not have to rescan the SCSI bus yourself to find a new device; on Windows you do not have to do rescan ever with ESX4.x, neither with LSI nor LSI-SAS)

Why not PVSCSI?

Virtual Desktops don’t tend to have a really high I/O. Using the PVSCSI driver with low throughput workloads may introduce a small increase in latency.

Pre ESX 4.1 the PVSCSI adapter had only been recommended for VMDK’s backed by fast (2,000+ IOPS) storage. Why? The PVSCSI driver only coalesces OIOs (Outstanding IOs) and not throughput (IOPS). What this means is; when the VM is requesting a lot of IO and the storage cannot deliver it, the PVSCSI driver is coalescing interrupts. Without the steady steam of IOs from the storage, there are no interrupts to coalesce. This can cause extra latency on low throughput environments.

The LSI Logic driver increases coalesce as OIO and IOPS increase. So if there is a low OIO and IOPS load no coalescing will occur, reducing latency.

Typically the case was:

• PVSCSI is best for workloads that drive more than 2000 IOPS and 8 outstanding I/Os.
• LSI Logic is best for workloads that drive lower I/O rates and fewer outstanding I/Os.

However! In ESX 4.1 this issue explained above has been fixed. This means that the PVSCSI could perform as well as the LSI Logic driver with low throughput. I have yet to see any official performance test results. If you have already used PVSCSI adapters and are not seeing any performance problems, I wouldn’t worry too much about making the change to LSI.

Summary

As you can see, there is a lot of Pro’s and Con’s to digest. I chose the SCSI LSI Logic controller for the following reasons:

• SCSI offers better performance than IDE
• LSI Logic is continually supported with newer OS’s and VMware releases, where BusLogic is being fazed out. So to help future proof your design, it would be better to use LSI.
• LSI offers better “Out of the box” performance over BusLogic
• LSI SAS is the default for Windows 7, great for Hot-Add disks and a must for MSCS.
• PVSCSI with ESX 4.1 is rumored to have equal performance with LSI Logic, but I have yet to see the test results.

What do you use on your site and why? Have I missed anything? let me know. I am interested to hear other peoples take on this, so please comment if you feel you have anything to add.

Sources

• http://www.vmware.com/pdf/ESX2_Storage_Performance.pdf
• http://communities.vmware.com/docs/DOC-10971.pdf
• http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf
• http://www.vmware.com/pdf/vsp_4_san_design_deploy.pdf
• http://www.vmware.com/pdf/GuestOS_guide.pdf
• http://www.vmware.com/pdf/vsp_4_pvscsi_perf.pdf
• PVSCI and low IO workloads
• SCSI vs IDE
• http://myvirtualcloud.net/

© Simon for Gestalt IT, 2011. | VMware View Desktops: IDE or SCSI? BusLogic, LSI Logic or PVSCSI?
Read more posts categorized as All, Server Virtualization, Storage

Most people would have heard of the two guys (Duncan Epping and Frank Denneman) who wrote this book and most of you will know that these guys are smart, real smart. This is something that has drawn me to read this book. Being a regular reader of their blogs I know that they are very “straight to the point” people. There will be no bullshit. Just good, complicated, technical content.

So far, I’ve read it on the bus, train, plane, taxi and tram. To be honest, any spare time I’ve had, I’ve picked it up and read a little. Usually, with me this wouldn’t happen. I’ve often found many IT books to be too large/heavy. So much so, I don’t carry it around with me. This book is a good size, 220 pages. It’s really easy to read, it’s like reading a long blog post. The language is informal, there is a little humor in there to. The chapters are short so you don’t always need loads of spare time to read a section. It could even used as a Toilet Book, you could get a chapter read in a single visit.

I like to think I know quite a bit about virtualisation, I work for VMware so I really should.  I know how to setup/configure HA and DRS and I’ve done it many many times. Select this setting, click a few drop-down menus and hey presto, HA and DRS does it’s thing – Feet up, relax. It’s the same for Reservations/Share/Limits, we all have a good idea of what they do, but do you fully understand the impact of the changes you are making? Are you making the right decisions? Read this book and you will be able to answer Yes to both of those questions.

In addition to the in-depth technical information, there are also Pro’s, Con’s and recommendations for various design configurations helping you to make decisions on what is best for you and your environment.

It’s one of those books where you can take as much or as little as you like. If you want a better basic knowledge or HA and DRS, then this book explains everything in a very clear and easy to understand context. If you really want to know how the DRS algorithm calculates the Priority Level of a Migration Recommendation this book will also service you well.

This book is available NOW from the following:

• Amazon
• CreateSpace

© Simon for Gestalt IT, 2010. | vSphere 4.1 HA and DRS Technical Deepdive – Book Review
Read more posts categorized as Server Virtualization

Question: “Assuming that I enable jumbo frames, is there a way on in the VM to determine is jumbo frames are truly supported end-to-end?”

Answer:

In your VM, ping your destination with a large message and specify don’t fragment.

• Linux VMs:         ping –M do –s 8000 <ip address or destination>
• Windows VMs:  ping –f –l 8000 <ip address or destination>
• ESX(i):                vmkping –d –s 8000 <ip address or destination>

© Simon for Gestalt IT, 2010. | Is Jumbo Frames Working?
Read more posts categorized as All, Server Virtualization

VMware Auto Deploy

The auto deploy application which comes as an OVT template is basically just a jumped-up vMA, with the added extra’s of DHCP, TFTP, HTTP servers and a deploy-cmd CLi and Database.

Here is a brief overview of how VMware Auto Deploy works once configured:

• PXE boot the target server
• ESXi is installed onto the target server from the auto deploy app
• The ESXi host will then be added into your vCenter
• The ESXi host will then have a Host Profile applied to it.

This makes life pretty easy.

What I didn’t know and it didn’t mention on the labs site was that the ESXi install was Stateless. The ESXi install is only held in memory. So if you reboot the server you’d see a “No Operating System Found” message.

Before VMware Auto Deploy, I hadn’t ever given Stateless ESXi a second thought. The configuration of a host once ESXi was installed was a lot more detailed than the initial install itself and took time to complete. Now with the use of Host Profiles we are able to Install and Configure an ESXi host within a matter of minutes and 100% automatically. At this rate I’ll be doing myself out of a job! However, I also believe it’s the way most large deployments will head in the not so distant future. We are beginning to see an increase in the amount of diskless servers/blades coming onto the market, which is ushering us down the route of using Stateless installs.

I’m not going to go into the in’s and out’s of the application configuration as it’s all available in the VMware Auto Deploy Administrator’s Guide. It’s very simple.

© Simon for Gestalt IT, 2010. | VMware Auto Deploy – Stateless ESXi
Read more posts categorized as All, Server Virtualization

Watching this brought back the excitement I felt after the VMware MVP presentation that VMware made back at VMWorld 2008. Since then it’s all been quiet…..Until now.

I came across this news article today; Arm’s next chip design to support virtualization. It looks as though ARM are now building in Virtualization capabilities into their new Cortex A processor which is going to be released shortly.

So does this mean we could soon see VMware finally releasing their Mobile Virtualization Platform? My guess would be Yes! VMware is a partner of ARM, so the chances are they are working closely with ARM to help develop Virtualization capabilities for their processors and have probably already started developing the MVP further to work on these processors.

I’ve not seen anything mentioned by VMware about this as yet, if you know of something please let me know as I am very interested in where this technology is heading.

So in the near future will we something like the image represents below? Android and iOS on the same mobile device? Who knows – We’ll just have to wait and see.

© Simon for Gestalt IT, 2010. | VMware MVP – Coming Soon?
Read more posts categorized as Server Virtualization

This is something I did a while ago, but it came to my attention that people didn’t; a) Know that it’s recommended to use a syslog server with ESXi b) You could use an application built in to vMA called vilogger.

Although it is stated in The Architecture of VMware ESXi…..

Because the in-memory file system does not persist when the power is shut down, log files do not survive a reboot. ESXi has the ability to configure a remote syslog server, enabling you to save all log information on an external system.

…..it is not a well known fact. So that is partly the reason for writing the post. The other reason is to introduce you to vilogger, which is part of the vMA. Of course you can use which ever syslog server you wish,  if you plan to use your own, be sure to checkout Managing VMware ESXi page #68 to view the configuration steps.

I’m not going to take you through the steps of installing vMA, nor am I going to tell you all about what the vMA (vSphere Manage Assistant) does. If you want to read more about that please find the relevant links in the Sources section at the bottom of the page. But I am going to take you through the steps I took to use vMA as my ESXi syslog server.

First of all, download the vMA from here. Import and configure it using the steps in the vSphere Management Assistant Guide

Changing vMA localtime settings to avoid incorrect log timestamps

When vMA collects the logs from your ESXi Host, sometimes the logs have the ESXi Host timestamp and sometimes they will have the vMA Localtime timestamp. I’m not exactly sure why this happens, but it does. (You may or may not know that ESXi uses UTC as its timezone when it timestamps the logs. You can read more about that here. VMware have told me this cannot be changed.)

So to get around the issue of differing timestamps in your collected logs we need to change the localtime on the vMA to UTC. This can be done using the following steps:

• sudo rm /etc/localtime
• sudo ln -s /usr/share/zoneinfo/UTC /etc/localtime

If you use NTP to sync your environments time, it might be worth adding in your ntp servers onto the vMA.

• sudo nano /etc/ntp.conf
• Add in your ntp servers under the heading: # Use public servers from the pool.ntp.org project.
• Configure ntpd to start on reboot: sudo /sbin/chkconfig ntpd on
• Restart ntpd: sudo /sbin/service ntpd restart
• Make sure your ntp servers are reachable: sudo ntpq -p

Whilst your at it you could change your keyboard for those of you outside the US.

• sudo vi /etc/sysconfig/keyboard
  • change KEYTABLES=”us” to use the keyboard you have, for example: KEYTABLES=”en”

Adding a second Hard Disk to store your logs

Because of the amount of ESXi Hosts I have to manage I decided that I would add a second Virtual Disk to house my logs.  You may not need to do this if you only have 1 or two Hosts.

• Add a second Hard Disk to the vMA VM
• PowerOn vMA
• Login using vi-admin
• Format the new Disk with the following command: fdisk /dev/sdb
  • Use the n command to create a new partition
  • Use the p command to make the new partition a primary partition
  • Press 1 to make it partition #1
  • Use the default for the First Cylinder
  • Use the default for the Last Cylinder
  • Use the p command to verify the partition table
  • Use the w command to write the partition table to the hard disk
• Press Enter a couple of times to confirm it’s finished and return to the command prompt
• Now we need to format the partition using the follow command: mkfs -t ext3 /dev/sdb1

Now we have our new disk ready, we now need to mount it.

• Edit /etc/fstab using the following command: nano /etc/fstab
• Enter in the following line: /dev/sdb1               /var/log/syslog            ext3    defaults,auto        1 2
• Use Cntrl+X then Y to save the file
• Next: cd /var/log/ then mkdir syslog
• finally mount /var/log/syslog to mount the disk

Ok, now our new Disk is mounted, we need to tell the vilogger application to store your logfiles there. The default location is /var/log/vmware so we need to change it.

• Edit the vilogger config file using: nano /etc/vmware/viconfig/vilogdefaults.xml
• Change the file to match this: <location>/var/log/syslog</location> (This appears twice, change both)
• Use Cntrl+X then Y to save the file
• Restart vilogger by: vmware-vilogd restart

Now vilogger is set to store your ESXi logfiles onto your new disk.

Configure vMA to collect your logs

Next we configure vMA and vilogger to collect your ESXi logfiles.

• Connect your ESXi Host to vMA using the following command: sudo vifp addserver <FQDN of ESXi Host>
• Enter the Root Password for the Host
• Once the Host has been added you can double check by running the following command: vifp listservers
• Now set vilogger to start collecting your logs: vilogger enable –server <FQDN of ESXi Host> –numrotation 20 –maxfilesize 10 –collectionperiod 10

You will see the above displayed, showing that vilogger is now collecting logs from your  Host.

Values –numrotation 20 –maxfilesize 10 –collectionperiod 10 can be changed to suit your needs. If you just run the vilogger –help command you will see what options and values are available.

Viewing your logs

Now your vMA is collecting your logs you can view them by: dir /var/log/syslog/<FQDN of Host> You can view your logs in real time if you wish, read more about there here.

• An example: tail -f /var/log/syslog/<FQDN of Host>/vpxa.log

Sources:

• Setting timezone in ESXi or not
• The Architecture of VMware ESXi
• vSphere Management Assistant Guide
• VMware KB1007551

© Simon for Gestalt IT, 2010. | Using vMA As Your ESXi Syslog Server
Read more posts categorized as Server Virtualization

The VCAP is comprised of two Exams:

• VMware Certified Advanced Professional – Datacenter Administrator (VCAP4-DCA)
• VMware Certified Advanced Professional - Datacenter Design (VCAP4-DCD)

You must be a VCP4 before being able to sit the VCAP4 exams. So if you pass the VCP 4, Datacenter Administrator, and Datacenter Design exams you will gain both VCAP4 certifications, and you will only be a design defense away from the VCDX4 certification.

You will be able to register for the VCAP4-DCA Exam from July 12th and registration for the VCAP4-DCD Exam will be available in August.

A few other pieces of information:

• The Datacenter Administrator (VCAP4-DCD) exam will be 100% live labs!
• The Datacenter Design (VCAP4-DCD) exam will be a mixture of multiple-choice questions, and a design exercise, as it is today for VI3
• The testing centres for the Datacenter Administrator (VCAP4-DCD) exam will be expanded to a significantly larger pool than it is today
• The new exams will be scheduled directly with Vue, only the VCDX design defense will need arranging directly with VMware
• The blueprint documents for the new exams are still in development, but they will both be published in advance of the release of the exams
• The new exams will not have compulsory training course requirements
• The vSphere: Manage for Performance, vSphere: Troubleshooting, and yet-to-be-released vSphere: Manage and Design for Security courses will all be recommended as preparation for the Datacenter Administrator (VCAP4-DCD) exam, the exam content is centred around those 3 key areas.
• The vSphere: Design Workshop will be excellent preparation for the Datacenter Administrator (VCAP4-DCD) exam, the workshop content covers all of the objectives in the exam

As soon as the Exam Blueprint become available, rest assured that I will be putting together more online Practice Exams to help me and anyone else planning to take these exams. Watch this space!

Source:

VMware Training and Certification

© Simon for Gestalt IT, 2010. | VMware Certified Advanced Professional – VCAP
Read more posts categorized as Top Story

Here is how I decided the script should function:

1. Send myself an email telling me that the backup process has started
2. Import the name of the VM to be backed up and the destination datastore from a CSV file
3. Create a Snapshot of the VM I want to backup
4. Create a Clone’d VM from the Snapshot
   • Place the backup onto my Backup Datastore
   • Name the backup <master VM name>-<date stamp>
   • Thin provision the backup
5. Remove the Snapshot from the master VM
6. Send myself an email telling me that the backup process has completed

So that’s what I wanted to do, this is what I ended up with.

Backup Script

# Import Backup CSV
$backupinfo =  Import-Csv C:\scripts\mybackups.csv

#Set Date format for clone names
$date = Get-Date -Format "yyyyMMdd"

#Set Date format for emails
$time = (Get-Date -f "HH:MM")

#Connect to vCenter
Connect-VIServer "<vc server>"

foreach ($customer in $backupinfo)
{
	$vm = Get-VM $customer.MasterVM

	#Send Start Email
	C:\scripts\backupstartedemail.ps1

	# Create new snapshot for clone
	$cloneSnap = $vm | New-Snapshot -Name "Clone Snapshot"

	# Get managed object view
	$vmView = $vm | Get-View

	# Get folder managed object reference
	$cloneFolder = $vmView.parent

	# Build clone specification
	$cloneSpec = new-object Vmware.Vim.VirtualMachineCloneSpec
	$cloneSpec.Snapshot = $vmView.Snapshot.CurrentSnapshot

	# Make linked disk specification
	$cloneSpec.Location = new-object Vmware.Vim.VirtualMachineRelocateSpec
	$cloneSpec.Location.Datastore = (Get-Datastore -Name $customer.BackupDS | Get-View).MoRef
	$cloneSpec.Location.Transform =  [Vmware.Vim.VirtualMachineRelocateTransformation]::sparse

	$cloneName = "$vm-$date"

	# Create clone
	$vmView.CloneVM( $cloneFolder, $cloneName, $cloneSpec )

	# Write newly created VM to stdout as confirmation
	Get-VM $cloneName

	# Remove Snapshot created for clone
	Get-Snapshot -VM (Get-VM -Name $customer.MasterVM) -Name $cloneSnap | Remove-Snapshot -confirm:$False

	#Send Complete Email
	C:\scripts\backupcompletedemail.ps1
}
#Disconnect from vCentre
Disconnect-VIServer -Confirm:$false

Send Started Email Script

#Set Date format for emails
$timestart = (Get-Date -f "HH:MM")

$emailFrom = "<senders address>"
$emailTo = "<recipients address>"
$subject = "[$vm - Backup Started]"
$body = "Backup Details
-------------
VM Name:",$vm,"
Clone Name:","$vm-$date","
Target Datastore:", $customer.BackupDS,"
Time Started:", $timestart

$smtpServer = "<smtp server>"
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($emailFrom,$emailTo,$subject,$body)

Send Completed Email Script

#Set Date format for emails
$timecomplete = (Get-Date -f "HH:MM")

$emailFrom = "<senders address>"
$emailTo = "<recipients address>"
$subject = "[$vm - Backup Complete]"
$body = "Backup Details
-------------
VM Name:",$vm,"
Clone Name:","$vm-$date","
Target Datastore:", $customer.BackupDS,"
Time Started:", $timestart,"
Time Completed:", $timecomplete
$smtpServer = "<smtp server>"
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($emailFrom,$emailTo,$subject,$body)

Content of CSV

The content of the csv file is very simple. This is what mine looks like:

MasterVM,BackupDS
VM1,BackupDataStore
VM2,BackupDataStore

Summary

So as you can see, it’s simple, but very effective. Let me know your thoughts/ideas on this could be improved.

Sources:

• Creating a linked clone from a snapshot point in PowerCli

© Simon for Gestalt IT, 2010. | PowerCLI: A Simple VMware Backup Script
Read more posts categorized as Featured, Server Virtualization