The great battle of enterprise storage is on!

The time has come to take sides on the core question of storage for virtual servers: Do you want storage intelligence to live in the hypervisor or the array? Most administrators are already lining up on one side or the other, unintentionally casting their vote while the rest flounder. But the storage industry must wake up and embrace the divide.

Hypervisor Huggers Unite!

Do you “heart” virtualization?

VMware’s vSphere dominates the world of enterprise server virtualization and has become the core element of the modern open systems datacenter. Microsoft recognizes this but has been unable to field a competitive hypervisor ecosystem for the virtual datacenter. Today, vSphere is the state of the art and nowhere is this more apparent than in storage.

In just a few years, VMware has delivered and updated a host of advanced storage functionality, from provisioning to migration and load balancing to backup and data protection. vSphere 5 includes an advanced and scalable storage virtualization layer, delivering everything a datacenter needs. VMFS sculpts basic block storage into a shared resource for virtual machines, with snapshots, policy-based layout and movement, and flexible allocation and thin provisioning.

Most VMware administrators are “server guys” and relish these features. They have never experienced an automated “storage service” like this, and the enterprise storage world has never been able to construct anything remotely as flexible, user-friendly, and functional. And Hypervisor Huggers don’t need complex enterprise storage arrays to do it: They can use basic iSCSI or Fibre Channel devices to provide performance and capacity and let VMware do the rest!

Storage DRS is exemplary of the new virtual datacenter world. Introduced in vSphere 5 (and restricted to the pricey Enterprise Plus license), Storage DRS uses the core technology of Storage vMotion to dynamically balance I/O and capacity across a diverse pool of storage. Storage DRS even uses Policy-Driven Storage and VASA to enforce tiered storage and data placement strategy. This kind of virtualization has been a “holy grail” quest for the enterprise storage industry, but they’ve never delivered on their promises.

Cheers for Storage Stalwarts!

Do you wish all this virtualization nonsense would just go away?

But not every IT environment wants be 100% vSphere focused, and many aren’t convinced that dumb storage is the smartest place for their data. These Storage Stalwarts want smarter and better-integrated storage arrays, and VMware is innovating here as well.

VMware’s Storage APIs for Array Integration (VAAI) technology is squarely aimed at this market. VAAI allows vSphere to hand off heavy storage operations to the high-end storage arrays from the major players. It works transparently, too, coordinating cloning without the kind of scripting and hair-pulling that used to require. VAAI in vSphere 4.1 also includes block zeroing support and something called “atomic test and set” which we’ll get to in a moment. Microsoft announced their own cloning integration, ODX, but it won’t ship until Windows Server 8 appears sometime next year.

But cloning is just the tip of the iceberg when it comes to Hypervisor-to-array integration. The rising army of NAS users have seen the glory of more-complete array integration for a while, and they’re not quiet about it. They love that VMware’s NFS protocol support makes storage “disappear” in vCenter, becoming just another resource with integrated thin provisioning and flexible allocation and movement.

VMware is moving aggressively to please their Storage Stalwarts, adding more VAAI support for block and file in vSphere 5. But, as the company laid out at VMworld 2011, neither access method is ideal for virtual servers. So VMware has been pushing the enterprise array vendors for ever-greater integration. They see a future where a VAAI-based protocol enables arrays to de-multiplex I/O streams from the hypervisor and intelligently handle per-VM data.

Stephen’s Stance

You can spot a Hypervisor Hugger by their big LUNs: They would rather treat storage as a bulk commodity, and array vendors should be lining up to get their business. Storage Stalwarts will jump on each new VMware innovation, finally making use of the capabilities they have spent over a decade paying for but not utilizing. The only untenable stance is trying to keep a foot in both worlds: It’s foolish to buy an enterprise array and use it as bulk storage!

© Stephen Foskett for Gestalt IT, 2011. | Are You a Hypervisor Hugger or a Storage Stalwart?
Read more posts categorized as Featured, Server Virtualization, Storage

After checking if I was able to ping the interface, I logged on to the machine to see if there were any obvious errors. The vCloud Director daemon was still running and so was the database, but a netstat did not show any listeners on the vCloud Director IP. So, after going over the vCloud Director log files, there was a pretty obvious error in the vcloud-container-info.log:

ORA-28001: the password has expired

So, you now stop your vCloud Director daemon and switch to your Oracle user to see what was going on inside of the DB:
sqlplus "/ as sysdba"

Now, list all the users to see if they have an expired password:
select username,ACCOUNT_STATUS,EXPIRY_DATE from dba_users;

Or display just the specific user:
select username,ACCOUNT_STATUS,EXPIRY_DATE from dba_users where username='VCLOUD';

And guess what came up:
USERNAME ACCOUNT_STATUS EXPIRY_DA
-------- -------------- ---------
VCLOUD EXPIRED 17-MAR-11

Expired is something that you don’t want to see for a user that is being used actively. So, let’s set the password again and unlock the user:
alter user VCLOUD identified by replace_this_with_your_password;
alter user VCLOUD account unlock;

So, once that is done, let’s check one more time:
SQL> select username,ACCOUNT_STATUS,EXPIRY_DATE from dba_users where username='VCLOUD';
USERNAME ACCOUNT_STATUS EXPIRY_DA
-------- -------------- ---------
VCLOUD OPEN 26-SEP-11

Now, start your vCloud Director daemon again, and in the log file you should no longer see the error, and the web interface should be working normally again.

© Bas for Gestalt IT, 2011. | Shorts: VMware vCloud Director not displaying the web portal
Read more posts categorized as Server Virtualization

For a great overview of the entire U1 release check out Rick Vanover’s post vSphere 4.1 update 1 released from his Rickatron Blog and via his Servers and Storage Column/Blog at TechRepublic.

The rest of this post contains cut and pastes from the Release Notes and some commentary about them from me. I want to stress again that these are issues that have now been fixed!

Finally, I’ll point out the one huge VM backup issue (that I can think of right now) that still does not appear to be resolved.

• Cannot take quiesced snapshots of Microsoft Windows Server 2008 R2 virtual machine running vCenter Server 4.1 When creating a snapshot of a Microsoft Windows Server 2008 R2 virtual machine that has vCenter Server 4.1 installed, the snapshot operation might fail to complete. This issue occurs on Microsoft Windows Server 2008 R2 virtual machines when the ADAM database is installed. The issue is resolved in this release.

The change from 2008 to 2008 R2 has caused quite a few applications to stumble, so it is good to see that VMware users can now install vCenter on the latest and greatest OS from Microsoft running as a virtual machine without having to worry about various hassles, not to mention coming up with a separate backup or business continuity solution for arguably one of the most important VMs in the environment.

• Creation of large .vmdk files on NFS might fail. When you create a virtual disk (.vmdk file) with a large size, for example, more than 1TB, on NFS storage, the creation process might fail with an error: A general system error occurred: Failed to create disk: Error creating disk. This issue occurs when the NFS client does not wait for sufficient time for the NFS storage array to initialize the virtual disk after the RPC parameter of the NFS client times out. By default the timeout value is 10 seconds. This fix provides the configuration option to tune the RPC timeout parameter using the esxcfg-advcfg -s <Timeout> /NFS/SetAttrRPCTimeout command.

Although specific to users of NFS storage, this one was definitely a head scratcher. Possible scenarios where this problem could rear it’s ugly head include full VM restores and even VM replication. Add the possibility of dedicating a large .vmdk as the backup repository for your backup server VM.

Let me know if I missed any other resolved issues specific to VM backups.

The issue still unresolved

Unfortunately, nothing in vSphere 4.1 U1 appears to address the loss of connectivity when committing snapshots for VMs using Change Block Tracking (CBT) while running on NFS storage.  This leaves VMware admins stuck between a rock and hard place – disable CBT or move VMs off of NFS. More about the problem can be found in VMware’s KB Article:

Virtual machine freezes temporarily during snapshot removal on an NFS datastore in a ESX/ESXi 4.1 host

© Rich for Gestalt IT, 2011. | vSphere 4.1 U1 Released. Fixes Specific For VM Backups
Read more posts categorized as All, Server Virtualization

Most people would have heard of the two guys (Duncan Epping and Frank Denneman) who wrote this book and most of you will know that these guys are smart, real smart. This is something that has drawn me to read this book. Being a regular reader of their blogs I know that they are very “straight to the point” people. There will be no bullshit. Just good, complicated, technical content.

So far, I’ve read it on the bus, train, plane, taxi and tram. To be honest, any spare time I’ve had, I’ve picked it up and read a little. Usually, with me this wouldn’t happen. I’ve often found many IT books to be too large/heavy. So much so, I don’t carry it around with me. This book is a good size, 220 pages. It’s really easy to read, it’s like reading a long blog post. The language is informal, there is a little humor in there to. The chapters are short so you don’t always need loads of spare time to read a section. It could even used as a Toilet Book, you could get a chapter read in a single visit.

I like to think I know quite a bit about virtualisation, I work for VMware so I really should.  I know how to setup/configure HA and DRS and I’ve done it many many times. Select this setting, click a few drop-down menus and hey presto, HA and DRS does it’s thing – Feet up, relax. It’s the same for Reservations/Share/Limits, we all have a good idea of what they do, but do you fully understand the impact of the changes you are making? Are you making the right decisions? Read this book and you will be able to answer Yes to both of those questions.

In addition to the in-depth technical information, there are also Pro’s, Con’s and recommendations for various design configurations helping you to make decisions on what is best for you and your environment.

It’s one of those books where you can take as much or as little as you like. If you want a better basic knowledge or HA and DRS, then this book explains everything in a very clear and easy to understand context. If you really want to know how the DRS algorithm calculates the Priority Level of a Migration Recommendation this book will also service you well.

This book is available NOW from the following:

• Amazon
• CreateSpace

© Simon for Gestalt IT, 2010. | vSphere 4.1 HA and DRS Technical Deepdive – Book Review
Read more posts categorized as Server Virtualization

EMC EBC Cork

I spent some days in Cork, Ireland this week presenting to a customer. Besides the fact that I’m now almost two months in to my new job, and I’m loving every part of it, there is one part that is extremely cool about my job.

I get to talk to customers about very cool and new technology that can help them get their job done! And while it’s in the heart of every techno loving geek to get caught up in bits and bytes, I’ve noticed one thing very quickly. The technology is usually not the part that is limiting the customer from doing new things.

Everybody know about that last part. Sometimes you will actually run in to a problem, where some new piece of kit is wreaking havoc and we can’t seem to put our finger on what the problem is. But most of the time, we get caught up in entirely different problems altogether. Things like processes, certifications (think of ISO, SOX, ITIL), compliance, security or just something “simple” as people who don’t want to learn something new or feel threatened about their role that might be changing.

And this is where technology comes in again. I had the ability to talk about several things to this customer, but one of the key points was that technology should help make my life easier. One of the cool new things that will actually help me in that area was a topic that was part of my presentation.

Some of the VMware admins already know about this technology, and I would say that most of the folks that read blogs have already heard about it in some form. But when talking to people at conventions or in customer briefings, I get to introduce folks over and over to a new technology called VAAI (vStorage API for Array Integration), and I want to explain again in this blog post what it is, and how it might be able to help you.

So where does VAAI come from?

Well, you might think that it is something new. And you would be wrong. VAAI was introduced as a part of the vStorage API during VMworld 2008, even though the release of the VAAI functionality to the customers was part of the vSphere 4.1 update (4.1 Enterprise and Enterprise Plus). But VAAI isn’t the entire vStorage API, since that consists of a family of APIs:

• vStorage API for Site Recovery Manager
• vStorage API for Data Protection
• vStorage API for Multipathing
• vStorage API for Array Integration

Now, the “only API” that was added with the update from vSphere 4.0 to vSphere 4.1 was the last API, called VAAI. I haven’t seen any of the roadmaps yet that contain more info about future vStorage APIs, but personally I would expect to see even more functionality coming in the future.

And how does VAAI make my life easier?

If you read back a couple of lines, you will notice that I said that technology should make my life easier. Well, with VAAI this is actually the case. Basically what VAAI allows you to do is offload operations on data to something that was made to do just that: the array. And it does that at the ESX storage stack.

As an admin, you don’t want your ESX(i) machines to be busy copying blocks or creating clones. You don’t want your network being clogged up with storage vMotion traffic. You want your host to be busy with compute operations and with the management of your memory, and that’s about it. You want as much reserve as you can on your machine, because that allows you to leverage virtualization more effectively!

So, this is where VAAI comes in. Using the API that was created by VMware, you can now use a set of SCSI commands:

• ATS: This command helps you out with hardware assisted locking, meaning that you don’t have to lock an entire LUN anymore but can now just lock the blocks that are allocated to the VMDK. This can be of benefit, for example when you have multiple machines on the same datastore and would like to create a clone.
• XSET: This one is also called “full copy” and is used to copy data and/or create clones, avoiding that all data is sent back and forth to your host. After all, why would your host need the data if everything is stored on the array already?
• WRITE-SAME: This is one that is also know as “bulk zero” and will come in handy when you create the VM. The array takes care of writing zeroes on your thin and thick VMDKs, and helps out at creation time for eager zeroed thick (EZT) guests.

Sounds great, but how do I notice this in reality?

Well, I’ve seen several scenarios where for example during a storage vMotion, you would see a reduction in CPU utilization of 20% or even more. In the other scenarios, you normally should also see a reduction in the time it takes to complete an operation, and the resources that are allocated to perform such an operation (usually CPU).

Does that mean that VAAI always reduces my CPU usage? Well, in a sense: yes. You won’t always notice a CPU reduction, but one of the key criteria is that with VAAI enabled, all of the SCSI operations mentioned above should always perform faster then without VAAI enabled. That means that even when you don’t see a reduction in CPU usage (which is normally the case), you will see that since the operations are faster, you get your CPU power back more quickly.

Ok, so what do I need, how do I enable it, and what are the caveats?

Let’s start off with the caveats, because some of these are easy to overlook:

• The source and destination VMFS volumes have different block sizes
• The source file type is RDM and the destination file type is non-RDM (regular file)
• The source VMDK type is eagerzeroedthick and the destination VMDK type is thin
• The source or destination VMDK is any sort of sparse or hosted format
• The logical address and/or transfer length in the requested operation are not aligned to the minimum alignment required by the storage device (all datastores created with the vSphere Client are aligned automatically)
• The VMFS has multiple LUNs/extents and they are all on different arrays

Or short and simple: “Make sure your source and target are the same”.

Key criteria to use VAAI are the use of vSphere 4.1 and an array that supports VAAI. If you have those two prerequisites set up you should be set to go. And if you want to be certain you are leveraging VAAI, check these things:

• In the vSphere Client inventory panel, select the host
• Click the Configuration tab, and click Advanced Settings under Software
• Check that these options are set to 1 (enabled):
  • DataMover/HardwareAcceleratedMove
  • DataMover/HardwareAcceleratedInit
  • VMFS3/HardwareAcceleratedLocking

Note that these are enabled by default. And if you need more info, please make sure that you check out the following VMware knowledge base article: >1021976.

Also, one last word on this. I really feel that this is a technology that will make your life as a VMware admin easier, so talk to your storage admins (if that person isn’t you in the first case) or your storage vendor and ask if their arrays support VAAI. If not, ask them when they will support it. Not because it’s cool technology, but because it’s cool technology that makes your job easier.

And, if you have any questions or comments, please hit me up in the remarks. I would love to see your opinions on this.

© Bas for Gestalt IT, 2010. | What is VAAI, and how does it add spice to my life as a VMware admin?
Read more posts categorized as Server Virtualization, Storage

Think about how you deploy virtual machines. Usually you will deploy one machine at a time, which is a good thing if you only need one server. But usually in larger environments, you will find that applications or application systems are not based on a single server. You will find larger environments that consist of multiple servers that will segregate functions, so for example, your landscape could consist of a DB server, an application server, and one or more proxies that provide access to your application servers.

If you are lucky, the folks installing everything will only request one virtual machine at a time. Usually that isn’t the case though. Now, this is where vCloud Director comes in. This will allow you to roll out a set of virtual machines at a time as a landscape. But it doesn’t stop there, since you can do a lot more because you can pool things like storage, networks and you a tight integration with vShield to secure your environment. But this should give you a very rough idea of what you can do with the vCloud Director. For a more comprehensive overview, take a look at Duncan’s post here.

Anyway, let’s dig in to the technical part.

There are plenty of blog posts that cover how to set up the CentOS installation, so I won’t cover that at great length. If you are looking for that info, take a peek here. If you want to install the Oracle DB on CentOS, take a look here to see how it’s done.

Here are some tips that might come in useful during the install:

• Use the full path to the keytool. There is a slight difference between /usr/bin/keytool, /usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre/bin/keytool and /opt/vmware/cloud-director/jre/bin/keytool. Be sure to use one of those, and if the commands to create and import your self-signed certificates are not working for some reason be sure to try a different one.

If you just simply create a database and browsed through the installation guide, you might have a hard time once you install the binary. Basically you run the “dbca” tool to create an empty database. If you by any chance forget to create the database files and run the installation binary (or the vCD configuration tool for that matter), you will receive an error while running the .sql database initialization scripts under /opt/vmware/cloud-director/db/oracle. The error message will tell you that there was an error creating the database.

Well, if only you had read the installation guide properly. Bascially what you do is start up the database:

sqlplus "/ as sysdba"
startup


Make sure that the path you use in the “create tablespace” command actually exists. If they don’t you need to perform “mkdir $ORACLE_HOME/oradata” first. Then create the tablespaces and corresponding files:

Create Tablespace CLOUD_DATA datafile '$ORACLE_HOME/oradata/cloud_data01.dbf' size 1000M autoextend on;
Create Tablespace CLOUD_INDX datafile '$ORACLE_HOME/oradata/cloud_indx01.dbf' size 500M autoextend on;

Now create a seperate user that we will give right for the database. The password for the user is the thing you type after “identified by”:

create user vcloud identified by vcloud default tablespace CLOUD_DATA;

Make sure that you give the user the correct rights to perform all the DB operations:

grant CONNECT, RESOURCE, CREATE TRIGGER, CREATE TYPE, CREATE VIEW, CREATE MATERIALIZED VIEW, CREATE PROCEDURE, CREATE SEQUENCE, EXECUTE ANY PROCEDURE to vcloud;


Now run the setup script, or run the configure script and you should be set to go.

© Bas for Gestalt IT, 2010. | Shorts: VMware vCloud Director installation tips
Read more posts categorized as Cloud Computing, Server Virtualization

The following screenshot shows the mysterious snapshot as I found it on my lab domain controller

In my case, the scheduled job that needs to create and then normally commit the snapshot is a Veeam Backup and Replication job. At some point in the life of my lab I did run out of space, and although it’s not an issue anymore, the snapshot still exists even when the jobs aren’t running! I was a bit surprised that backup jobs could even complete with this snapshot in place.

VMware KB article 1003302 explains that the Consolidate Helper snapshot is actually created when trying to commit (delete all) a snapshot.

From Delete All snapshot operation results in a Consolidate Helper snapshot when a datastore has insufficient disk space:

“If you try to initiate a Delete All snapshot for a virtual machine using Snapshot Manager, and if that virtual machine is on a datastore that does not have sufficient space for the snapshot, the following message displays in VMware Infrastructure (VI) Client:

msg.hbacommon.outofspace: there is no more space for the redo log of <VMname>-0000xx.vmdk.

You are given the option to abort or retry.

• If you choose Abort, the virtual machine is powered off, the snapshot is aborted, and a Consolidate Helper snapshot is created. The Snapshot Manager UI displays that Consolidate Helper snapshot. You can delete the Consolidate Helper snapshot after you have made space available.
• If you click Retry, the Snapshot Manager returns to Consolidate Helper snapshot mode unless you have made more disk space available.”

Personally, I never saw the VI/vSphere Client message.

The KB Article suggests deleting the snapshot when your datastore has enough free space and even offers instructions for extending your datastore.

Luckily, my snapshot was only consuming 29.14 KB of space so it went under the radar for about 8 days. I’ve blogged before about how unnoticed ESX snapshots can be dangerous.

Clicking the Delete All button from the Snapshot Manager Window (previous screenshot) after making sure there were no active backup jobs removed my snapshot painlessly this time.

This apparently is one of a few new Changes to the ESX Snapshot mechanism.

© Rich for Gestalt IT, 2010. | Consolidate Helper Snapshot Appears On vSphere VM
Read more posts categorized as Server Virtualization

However, I’ve now had time to reflect and one of the most interesting areas I covered this week which was the new storage features in vSphere 4.1. I had the chance to cover these in multiple sessions, see various demo’s and talk about it with the VMware developers and engineers responsible. There are two main features I want to cover in depth as I feel they are important indicators of the direction that storage for VMware is heading.

SIOC – Storage I/O Control

SIOC had been in the pipeline since VMworld 2009, I wrote an article on it previously called VMware DRS for Storage, slightly presumptuous of me at the time but I was only slightly off the mark. For those of you who are not aware of SIOC, to sum it up again at a very high level let’s start with the following statement from VMware themselves.

SIOC provides a dynamic control mechanism for proportional allocation of shared storage resources to VMs running on multiple hosts

Though you have always been able to add disk shares to VM’s on an ESX host, this only applied to that host, it was incapable of taking account of VM I/O Behaviour of other VMs on other hosts. Storage I/O control is different in that it is enabled on the datastore object itself, disk shares can then be assigned per VM inside that datastore. When a pre-defined latency level is exceeded on a VM it begins to throttle I/O based on the shares assigned to each VM.

How does it do this, what is happening in the background here? Well SIOC is aware of the storage array device level queue slots as well as the latency of workloads.  During periods of contention it decides how it can best keep machines below the predefined latency tolerance by manipulating all the ESX Host I/O Queues that affect that datastore.

In the example below you can see that based on disk share value all VM’s should ideally be making the same demands on the storage array device level queue slots.  Without SIOC enabled that does not happen. With SIOC enabled it begins throttling back the use of the second ESX host’s I/O queue from 24 slots to 12 slots, thus equalising the I/O across the hosts.

Paul Manning (Storage Architect – VMware product marketing) indicated during his session that there was a benefit to turning SIOC on and not even amending default share values.  This configuration would immediately introduce an element of I/O fairness across a datastore as shown in the example described above and shown below.

So this functionality is now available in vSphere 4.1 for Enterprise Plus licence holders only.  There are a few immediate caveats to be aware of, it’s only supported with block level storage (FC or ISCSI) so NFS datastores are not supported. It also does not support RDM’s or datastores constructed of extents, it only supports a 1:1 LUN to datastore mapping. I was told that extents can cause issues with how the latency and throughput values are calculated,  which could in turn lead to false positive I/O throttling, as a result they are not supported yet.

It’s a powerful feature which I really like the look of. I personally worry about I/O contention and the lack of control I have over what happens to those important mission critical VM’s when that scenario occurs. The “Noisy Neighbour” element can be dealt with at CPU and Memory level with shares but until now you couldn’t at a storage level. I have previously resorted to purchasing EMC PowerPath/VE to double the downstream I/O available from each host and thus reduce the chances of contention.  I may just rethink that one in future because of SIOC!

Further detailed information can be found in the following VMware technical documents

• SIOC – Technical Overview and Deployment Considerations
• Managing Performance Variance of applications using SIOC
• VMware performance engineering – SIOC Performance Study

VAAI – vStorage API for Array Integration

Shortly before the vSphere 4.1 announcement I listened to an EMC webcast run by Chad Sakacc.  In this webcast he described EMC’s integration with the new vStorage API, specifically around offloading tasks to the array. So what does all this mean, what exactly is being offloaded?

So what do these features enable? Let’s take a look at them one by one.

Hardware assisted locking as described above provides improved LUN metadata locking.  This is very important for increasing VM to datastore density.  If we use the example of VDI boot storms, if only the blocks relevant to the VM being powered on are locked then you can have a more VM’s starting per datastore.  The same applies in a dynamic VDI environment where images are being cloned and then spun up; the impact of busy cloning periods, i.e. first thing in the morning is mitigated.

The full copy feature would also have an impact in the dynamic VDI space, cloning of machines taking a fraction of the time as the ESX host is not involved. What I mean by that is when a clone is taken now, the data has to be copied up to the ESX server and then pushed back down to the new VM storage location.  The same occurs when you do a storage vMotion, doing it without VAAI takes up valuable I/O Bandwidth and ESX CPU clock cycles. Offloading this to the array prevents this use of host resource and in tests has resulted in a saving of 99% on I/O traffic and 50% saving on CPU load.

In EMC Labs a test of storage vMotion was carried out with VAAI turned off, it took 2 mins 21 seconds.  The same test was tried again with VAAI enabled, this time the storage vMotion took 27 seconds to complete. That is a 5x improvement, and EMC have indicated that they have had a 10x improvement in some cases. Check out this great video which shows a storage vMotion and the impact on ESX and the underlying array.

There is also a 4th VAAI feature which has been left in the vStorage API but is currently unavailable, Mike Laverick wrote about it here. Its a Thin Provisioning API and Chad Sakacc explained during the group session that its main use is for Thin on Thin storage scenarios. The vStorage API will in the future provide vCenter insight into array level over provisioning as well as the VMware over provisioning.  It will also be used to proactively stun VM’s as opposed to letting them crash as currently happens.

As far as I knew EMC was the only storage vendor offering array compatibility with VAAI. Chad indicated that they are already working on VAAI v2 looking to add additional hardware offload support as well as NFS Support. It would appear that 3Par offer support, so that kind of means HP do to, right? Vaughan Stewart over at NetApp also blogged about their upcoming support of the VAAI, I’m sure all storage vendors will be rushing to make use of this functionality.

Further detailed information can be found at the following locations.

• What does VAAI mean to you? – Chad Sakac EMC
• EMC VAAI webcast – Chad Sakac EMC

Storage DRS – the future

If you’ve made it this far through the blog post then the fact we are taking about Storage DRS should come as no great surprise.  We’ve talked about managing I/O performance through disk latency monitoring and talked about array offloaded features such as storage vMotion and hardware assisted locking. These features in unison make Storage DRS an achievable reality.

SIOC brings the ability to measure VM latency, thus giving a set of metrics that can be used for storage DRS.  VMware are planning to add capacity to the storage DRS algorithm and then aggregate the two metrics for placement decisions.  This will ensure a storage vMotion of an underperforming VM does not lead to capacity issues and vice versa.

Hardware Assisted Locking in VAAI means we don’t have to be as concerned about the number of VM’s in a datastore, something you have to manage manually at the moment.  This removal of limitation means we can automate better, a storage DRS enabler if you will.

Improved Storage vMotion response due to VAAI hardware offloading means that the impact of storage DRS is minimised at the host level. This is one less thing for the VMware administrator to worry about and hence smoothes the path for storage DRS Adoption.  As you may have seen in the storage vMotion video above the overhead on the backend array also appears to have been reduced, so you’re not just shifting the problem somewhere else.

For more information I suggest checking out the following (VMworld 2010 account needed)

• TA7805 – Tech Preview – Storage DRS

Summary

There is so much content to take in across all three of these subjects I feel that I have merely scratched the surface.  What was abundantly clear from the meetings and session I attended at VMworld is that VMware and EMC are working closely to bring us easy storage tiering at the VMware level.  Storage DRS will be used to create graded / tiered data pools at the vCenter level, pools of similar type datastores (RAID, Disk type). Virtual machines will be created in these pools; auto placed and then moved about within that pool of datastores to ensure capacity and performance.

In my opinion it’s an exciting technology, one I think simplifies life for the VMware administrator but complicates life for the VMware designer. It’s another performance variable to concern yourself with and as I heard someone in the VMworld labs comment “it’s a loaded shotgun for those that don’t know what they’re doing”.  Myself, I’d be happy to use it now that I have taken the time to understand it; hopefully this post has made it a little clearer for you to.

© Craig for Gestalt IT, 2010. | Whats new in vSphere 4.1 Storage: SIOC, VAAI, and DRS
Read more posts categorized as Featured, Server Virtualization, Storage

The following post was contributed by Dwayne Lessner (@DLink7).

While going through the release notes for vSphere 4.1 I noticed one of my favourite vSphere tools be might be going away. vCenter Update Manager (VUM) 4.1 and its subsequent update releases are the last releases to support scanning and remediation of patches for Windows and Linux guest operating systems.

I think it would be a mistake not to continue on with VUM. The tool can scan and remediate both powered on and powered off virtual machines (VMs). You can easily tell from a single pane of glass which VM’s are compliant or not. With a large virtual desktop Infrastructure (VDI) environment it’s fast and easy to update your templates and linked clones. You can even automatically take a snapshot before you apply the updates in case there is a problem. This tool has shaved off hours on monthly change windows. If I had to do all this work manually I would have to hire extra staff which in today’s market is not going to happen.

Shavlik, the company behind the patch database that VUM relies on, seems to have a good working relationship with VMware. Last year when VMware announced VMware GO, a free web-based service that will allow a customers to set up ESXi, Shavilk was a major partner. It’s hard to believe they would just drop each other but I believe there is a cloud play. Shavlik will be providing cloud-based IT management and patching through their OPsCloud strategy. I believe with VMware’s Redwood around the corner the two companies will offer the proper hooks to each others infrastructure in the form of the appropriate cloud API’s.

It’s likely that the same tools will exist but in different forms and under different names. I am sure we will have another reason to spend the money on another upgrade.

© guest for Gestalt IT, 2010. | Is My Favorite vSphere Tool Going Away?
Read more posts categorized as All, Exclusive, Server Virtualization

Side note – I hate saying “V A A I”. IMO it’s quite a tongue twister. I have adopted saying “V double A I” because it is much easier to get out.

VAAI Introduced 2 years ago

Although it does not exist in vSphere 4.0 today, believe it or not VAAI was first introduced at VMworld 2008 when the vStorage API was announced.

VMware Unveils New vStorage Technologies to Deliver Leading Efficiency and Manageability for the Virtual Datacenter Operating System

New Capabilities Deliver Deeper Integration with Storage Partner Functionality
vStorage enables intelligent integration of storage products within the Virtual Datacenter OS through the new vStorage APIs.

vStorage APIs deliver tight integration of advanced capabilities from storage partners with the Virtual Datacenter OS from VMware.  vStorage APIs for array integration will enable customers to leverage array-based capabilities, such as snapshots, provisioning, replication and restore, directly with individual virtual machines in conjunction with the clustering and pooling capabilities of VMware Infrastructure.  vStorage APIs for multi-pathing will provide customers the ability to integrate advanced load balancing capabilities provided by leading partners’ multi-pathing software with their virtual environment.

If you are like me you were probably so overwhelmed by the news of the VCD-OS (That’s what VMware was calling the hybrid Cloud at the time … I guess?) that you missed (or were not capable or ready to comprehend) how complete of a strategy already existed for building Infrastructure as a service (IaaS) with the vStorage API. For me, looking back 2 years later it is amazing to me it was all there – at least in concept.

Don’t confuse VADP and VAAI

VAAI is only one part of the vStorage API. VADP (vStorage API for Data Protection) is another, but the these two seem to have been easily confused lately. VADP is enabling access to files on VMFS without VCB, but VAAI has nothing to do with that today. Will it in the future? Maybe if the process of creating and committing snap shots can be handled by the storage device processor, but that’s the key point and difference. VAAI will offload functionality to the storage processor and thus reduce compute load on the hypervisor. VADP allows third party technology partners to perform full image virtual machine backups.

What vSphere functions can be offloaded with VAAI?

Once again I’ll link to a post from VMworld 2008. Chad Sakac (@sakacc) wrote about what VAAI back then when it was first announced. In So, what does vStorage really mean? Sakac lists several functions that a VAAI aware storage device could do. I’ve pasted and edited for length Sakac’s original list. Quoting Chad:

• Acceleration via storage offloads using SCSI driver primitives [omitted] - think “I/O” dedupe (not to be confused with Storage Dedupe).  This can have a HUGE impact on storage network utilization – much more efficient, and very important for lower bandwidth storage networks like 1Gbps iSCSI.
• Thin Provisioning Integration
• The ESX server tells the array what BLOCKS makeup a VM, so svmotion can be offloaded to the array as an alternative to the file-copy method now, snapshots can potentially be done on VMs even for block devices.  Again, this uses SCSI driver primitives

How will VAAI do this?

Much of this still remains to be seen, but according to Sakac and the others I’ve spoken with about VAAI recently, you’ll be able to buy a storage device with processors that are VAAI capable. That means a hardware list that will probably start out small and grow quickly. Sakac likens VAAI integration in the future to buying servers with Intel VT-x or AMD-V technology today. You just don’t buy servers for vSphere without these on chip technologies.

In my own terms, a VAAI  aware storage device will be able to understand commands from vSphere and perform storage functions with it’s own compute resources.

When will we see this new hardware? Honestly I’m not sure, but I imagine we are getting close.

So, what does VAAI mean for vSphere virtualization?

Simply put, VAAI will allow more hypervisor compute power to remain available for running virtual machines. This means greater VM to host ratios without more physical resourcesand even larger workloads in a single VM. This also means faster VMotions, Storage VMotions, cloning, VM deployment and more. I’m guessing that this intelligent “pillow talk” between vSphere and storage will ultimately lead to safer disk over allocation and dynamic adjustments to storage workloads never before possible. Those are just a few possibilities off the top of my head.

© Rich for Gestalt IT, 2010. | What Is VAAI And What Does It Mean For Virtualization?
Read more posts categorized as Featured, Server Virtualization, Storage