Stepping out of a subway car is an entirely different matter when it's moving!

Mobility of virtual machines is a sticky wicket. As open systems infrastructure becomes increasingly virtualized, administrators and managers wish to use the technology to balance workload, ease migration, and provide better availability. Although technology is improving, actually moving virtual machines is not always a piece of cake. Let’s lay down a baseline of information so we may begin a discussion on the true nature of virtual machine mobility.

Mobility of What?

Let us consider first the question of what exactly is being moved. Systems administrators often focus on “the machine”, which encompasses the operating system and configured state of the virtual machine itself. But the true “mass” of the system is its stored data. Hypervisor vendors have come up with different techniques of moving these two essential elements, reflecting the unique characteristics of each.

• The virtual machine is an instance of an operating system along with its state and configuration. Mobility of virtual machines requires all of this to be preserved, along with any I/O channels. Live migration of virtual machine requires that any active network sessions be maintained, along with RAM content, registers and buffers, and so many other elements.
• The virtual machine image (commonly referred to as “storage”) is the static content addressed by a virtual machine. Typically a VMDK or similar virtual disc image, it must be accessible to the virtual machine at all times. Live migration of a virtual machine image is tricky, but perhaps not quite as complex as live migration of a running operating system.

VMware, Microsoft, and others recognize these 2 distinct elements to be migrated, and have come up with a variety of complementary technologies for each:

• vMotion is VMware’s virtual machine migration solution, and has continually evolved with each iteration of the hypervisor. DRS leverages vMotion to automate mobility. VMware has also created Storage vMotion and Storage DRS as complements to handle mobility of virtual machine images.
• Microsoft Hyper-V Live Migration is conceptually similar to vMotion, though newer and less full-featured. With Hyper-V 3.0, Microsoft will introduce Storage Live Migration as a complementary technology akin to Storage vMotion. Most other virtual machine managers also support some form of live migration, though live migration of storage is less common.

Mobility in What State?

One of the key benefits of virtual machine technology is the ability to “run anywhere” on dissimilar hardware. From the very beginning, hypervisors have provided the ability to create a universal virtual machine image that would run on a variety of supported platforms.

This leads to one of the key values of server virtualization in the data center: Disaster recovery. The ability to take a virtual machine image and system state and bring it online after a disaster is a true revolution for open systems IT. The benefits of the single usage of server virtualization technology easily justify the investment to many businesses.

But this sort of “cold” migration seems passé when compared to the live or “hot” migration possible with technologies like VMware vMotion. Live migration is much more difficult, since active client sessions must be preserved in activity must not be greatly interrupted.

This is the second great question that must be asked when considering virtual machine mobility: In what state will the virtual machine be moved? Will it be a cold, powered down image of the system? A suspended or paused operating system image? Or a full, running machine?

Mobility to Where?

Once we have decided whether we are discussing virtual machine migration or movement of storage resources, we must consider the scope of the movement. The ability to move a virtual machine from one member of the cluster to another has now become fairly common. But what about systems that are not related in a cluster? Or that are spread over great distances?

• The nice thing about clusters is that they share resources before and after a virtual machine is moved. It is practical to move the running virtual machine, its storage, or both independently and to expect that performance will not dramatically suffer as a result. The cluster can also preserve network connections, and even I/O state, without much impact on clients or other external elements.
• It is a bit more difficult to move systems within a data center, since one must maintain the I/O connections that might be interrupted. It is fairly trivial to configure an IP network and storage array to allow multiple machines to access the same iSCSI or NFS storage resources. It is a little more difficult to configure Fibre Channel (and, by extension, FCoE) SAN’s to handle this sort of dynamic movement, but it is not impossible. Although moving a running machine from one network port to another could cause client access to be interrupted, technologies like VXLAN allow these sessions to continue, and improved network switching technology should reduce performance impact.
• Moving the machine to a different data center is another matter entirely. Stretching a layer-2 Ethernet LAN or Fibre Channel SAN across a metro or greater distance, while possible, will always be problematic. IP routing is flexible, but it takes time for changes to propagate when live machines are moved. And it is difficult to keep storage in sync over long distances due to the amount of time it takes for information to transit. Again, all of these challenges are being addressed in various ways, but they’re still hard!

Stephen’s Stance

Moving cold virtual machine images from system to system, or even across great distances, is one of the main selling points of server virtualization. But it becomes much more difficult to manage movement of virtual machines that are still running, especially outside cluster or across WAN links. When talking about virtual machine mobility, it is important to consider what is being moved, the state it is in, and where it is going.

Note: This discussion is part of “Building Virtual Infrastructure”, my new seminar series with Truth in IT.

© sfoskett for Stephen Foskett, Pack Rat, 2012. |
Virtual Machine Mobility: Of What, and to Where and in What State?


This post was categorized as Enterprise storage, Features, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2012. | Virtual Machine Mobility: Of What, and to Where and in What State?
Read more posts categorized as Storage

How could a company as mighty and diverse as HP have had so many issues with executive management?

HP stumbled mightily in 2011, and it had nothing to do with product or people. Even sales remained strong, though the PC business is changing. HP’s mighty stumble was a crisis of confidence due to a chain of shenanigans at the very top. This culminated with the short reign of Léo Apotheker, leaving HP to reassure the market of its strategy.

HP And the Enterprise IT Industry

Sometimes, it’s hard to get a sense of scale when talking about very large things. How big is the solar system? How far away is the nearest star? The same is true of earthly things, exemplified by popular misconceptions about the global financial crisis. It’s difficult for people to understand just how much money a trillion dollars is.

In my little world of enterprise storage, it’s difficult to reconcile “big storage” players like EMC and NetApp with “big storage and everything else” players like HP, Dell, Oracle and IBM. Sure, EMC and NetApp lead the pack in terms of market share, but they’re nowhere near as large as the integrated players. HP has more than 7 times the revenue of EMC, which makes 3 times more than NetApp.

This old slide might need updating, but you get the picture…

HP is an incredibly diverse company, dominant in the PC, printing, and blade server market and top 5 just about everywhere else, including networking, services, and enterprise storage. And HP has nearly 325,000 employees, all working to move the company forward in one direction or another.

NetApp is a motorcycle, with one drive wheel pushing it forward at high speed; HP is more like NASA’s shuttle crawler-transporter, a 16 motor mammoth. Single-purpose companies can be agile, but they can also be derailed by market downturn or technological shift. Storage specialists like NetApp continually try to innovate and acquire to keep themselves vital, while larger companies like Cisco and EMC try to diversify while maintaining market leadership. HP doesn’t need to try; it is diverse.

HP Is a Very Large Thing

I watched HP's rise in the server market of the 1990's

HP has long been synonymous with innovation, high-technology, and silicon Valley. I have been an HP customer as long as I have been in IT, and watched as they integrated technology from Apollo and Convex in the 1990’ s. The server products that resulted became the dominant UNIX platform, but HP’ s innovative storage concepts didn’ t take the market by storm.

After HP merged with Compaq (which brought Tandem and Digital Equipment Corporation), the company vaulted ahead in the Wintel market and also gained valuable storage expertise. Throughout the last decade, HP was firing on all cylinders and dominant in nearly every arena. The company owns half the blade server market, is tied for first in servers, leads in PCs and printers, and is a contender in networking and storage. It’s simply impossible to say what HP is in a single sentence.

HP storage has an extremely broad product range, which management is working to reconcile. Acquisitions of LeftHand, Ibrix, and 3PAR gave HP storage shot in the arm to be sure. An injection of startup mojo has energized the marketing and product groups within HP just when the company needed it. HP’ s market share has grown somewhat as a result, though not as much as the hyper-focused NetApp. HP networking similarly took on 3Com, bedeviling Cisco in the Ethernet switch market.

The Executive Soap Opera

It takes a lot to bring a mammoth to its knees, but a shot between the eyes usually does the trick. While the many engines of HP push it forward, the company’s upper management has seemed, at times, suicidal. Business schools could design an entire curriculum around the folly of Patricia “I spy” Dunn and Mark “penny-pinching” Hurd. Who would think that HP management could top this?

From August 2010 through September 2011, HP dominated IT headlines in completely the wrong way. The board wanted a change, and selected Léo Apotheker to transform HP. But it was a soap opera from the very start, with Oracle hiring Mark Hurd and sending Apotheker into hiding among accusations of corporate espionage while at SAP.

The new CEO didn’t seem to understand HP at all, though he was intent on steering it in a new direction. Apotheker set about dismantling HP’s consumer and end-user businesses, killing Palm/WebOS and threatening to sell off the PC business. The company was to focus instead on enterprise computing, but these drastic moves spooked the entire industry.

It wasn’t long before HP’s board struck again, with a shake up at the hands of Ray Lane and former eBay CEO Meg Whitman brought in to replace Apotheker. The first order of business for the new HP executive team appears to be reassuring the entire world that management has not gone completely insane.

Stephen’s Stance

To an outsider like me, the most disappointing thing about HP’s mighty stumble is that it has nothing to do with the people who really make the company what it is. I have met many creative, hard-working individuals in HP’s storage, server, networking, and printer groups, and they could not be more different from the executive soap opera. I only hope that this new board and CEO will bring some stability and let HP cruise forward once again.

Disclaimer: HP has sponsored the Tech Field Day events which I organize, and has on occasion invited me to attend events at their expense.  But I do similar work with nearly every company in the IT industry, and this piece is my own opinion.

© sfoskett for Stephen Foskett, Pack Rat, 2012. |
HP’s Mighty Stumble


This post was categorized as Computer History, Enterprise storage, Gestalt IT. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2012. | HP’s Mighty Stumble
Read more posts categorized as Storage

I Promise Not To Quote That Old Annoying Dell PC Slogan

I’ve been buying Dell computers for decades, but not really because I loved them. Sure, my XPS laptop was awesome, but it burned out its motherboard and I never really touched the RMA replacement, having bought a MacBook Pro in the meantime. Enterprise buyers seem to have the same ambivalence about Dell. They buy it, but I’m not sure they really “buy” the company as an IT partner.

I’m sure you’ve heard the same comments as me: “Dell just assembles off-the-shelf components and sells them in volume” or “Dell’s a follower, not an innovator.” There seems to be a great deal of respect for Dell’s ability to produce competitive products and sell them at reasonable cost. Truly, most of their competitors would love to have this kind of reputation. But most of their competitors also have a reputation for partnership, innovation, and solution selling.

Dell Is Making An Effort

It seems clear that Dell would like to change this attitude, and they are investing serious resources to make it happen. While acquisitions like Compellent and Force10 raised eyebrows in storage and networking, it is the activity I see behind the scenes that paints the clearest picture. Dell isn’t just buying into new markets, they’re investing to change the company.

When Dell acquired EqualLogic in 2008, many assumed it was a tactical investment to increase margins over the (resold) EMC storage equipment the company was then pushing. Pundits were similarly dismissive of the acquisition of Perot Systems in 2009, calling it a “me too” effort after HP acquired rival EDS. Regardless of the motivations, however, Dell was becoming more of a serious challenger to HP and IBM every day.

After failing to acquire 3PAR in 2010, then picking up Compllent shortly after, accusations that Dell was “mini me” to HP were rampant. But HP stumbled mightily in 2011, and many in IT quickly lost confidence in that company’s management. All the while, Dell moved forward, increasing in-house IP and expanding enterprise offerings.

What Is The Result?

Today, one sees a very different landscape than just last year. Dell’s acquisitions focused on some of the ripest spots in storage and networking, and no one would disagree that the company has the ability strongly to push these products. Compellent and Force10 went from interesting startups to serious contenders overnight.

Dell really pulled out all the stops to tell us they are "Fluid by Design"

More importantly, Dell has retained much of the innovation these companies offered, from employees to support programs. Last week, I attended the Dell Storage Forum in London, an event initiated by Compellent prior to the acquisition. At the event, I talked to many Dell employees who came to the company through acquisition but had now been given power to challenge the status quo in their respective areas.

If Dell really intended only to push product, why retain marketing personnel? Why invest in the Dell Storage Forum? Why continue Compellent’s beloved Co-Pilot support program?

Then there are the products. Dell leveraged its investment in Ocarina Networks to create a deduplicating backup appliance, the new DR4000. They salvaged file system startup ExaNet and are beginning to bring scale out technology to market. The latest revision of the Compellent software finally brings it to parity in terms of VMware support. And Dell is really working to sell their DX Object Store.

This is the sort of activity one would expect from a contender, not a “box pusher”.

Stephen’s Stance

In the words of Malcolm Reynolds, my days of not taking Dell seriously are certainly coming to a middle. Dell is investing in product IP, innovative marketing and PR events, customer support, and personnel. This does not mean that Dell is instantly a player in the enterprise storage and networking markets, or that all this work will pay off. But I don’t laugh when I hear Dell boast that they intend to be a “top three” enterprise storage company in a few years. It could happen.

Disclaimer: Dell sponsored two Tech Field Day events in 2011, paid me as a speaker at two DX events, and paid for my trip to Dell Storage Forum in London. But no one can buy a post on this site, and I did similar business with IBM, HP, Cisco, and many other companies. This is my opinion.

© sfoskett for Stephen Foskett, Pack Rat, 2012. |
Why Should Anyone Take Dell Seriously in Enterprise Storage?


This post was categorized as Computer History, Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2012. | Why Should Anyone Take Dell Seriously in Enterprise Storage?
Read more posts categorized as Storage

It's not going to be this easy to bridge Fibre Channel and Ethernet!

Before the holidays, I posed a question on Google+ that generated quite a bit of interest and feedback. Now that it has settled down a bit I’d like to summarize the unresolved elements to make FCoE truly a world-class storage interconnect.

Setting the Stage

FCoE has been a controversial topic in both storage and networking, and for good reason. No one would deny that Ethernet is not an ideal transport mechanism for block storage I/O. “Porting” Fibre Channel to run on Ethernet networks has been a supreme technical challenge, and many companies and individuals have labored long and hard to make FCoE a reality.

Now that FCoE is specified in the standard and has been deployed in production environments, the question turns to its future. Will it take off and seize the mantle of dominance currently held by what I like retroactively to call “Fibre Channel over Fibre Channel?” Will they coexist for the next decade, with FCoE mainly deployed in “block” environments such as Cisco UCS? Or will FCoE ultimately fail to catch on, displaced by some other storage protocol like plain FC, iSCSI, NFS, or something entirely different?

The data center needs a flexible new protocol to meet the needs of virtual environments, and convergence of storage and data networking makes a great deal of sense in these environments. This was the root of my question, and I ask it in all earnestness.

My question: What elements remain unresolved to make FCoE truly world-class? What should the vendors be prioritizing? Here are the answers I received.

Technical Considerations

Link Aggregation on CNA’s

Converged network adapters (CNA’s) allow multiple protocols to access a single Ethernet connection, but some also include multiple ports that can be aggregated. In traditional Ethernet networks, link aggregation is a respectable approach for performance and availability. But storage networks have traditionally relied on host-based MPIO software, and these features are mutually exclusive. The zeitgeist seems to be a recommendation to avoid link aggregation on CNA’s that are used for storage networks.

How Do You Handle Virtual Machine Mobility?

As I described recently, virtual machine mobility is a major technical challenge for existing networks. The VMware proposal, the VXLAN, seems to be gaining traction right now. But this is only a solution for data networking. How will FCoE SANs handle virtual machine mobility? This remains unresolved as far as I can tell, though Ethernet switch vendors have come up with their own answers. Brocade demonstrated just such a solution at Networking Field Day 2, and I know that others have answers as well. But will there be an interoperable industry solution?

How Should FCoE Be Implemented Over Longer Distances?

Fibre Channel has traditionally relied on routers and other protocols (FCIP and iFCP) to span distances, but FCoE raises the possibility of native traversal. While it is certainly possible to span distances with FCoE, this is definitely not a recommended or supported idea. Without TCP/IP, or any routing mechanism, it’s just a bad idea. But I imagine that it won’t be long before vendors decide to give it a go anyway.

Implementation Considerations

Is TRILL Required for FCoE Networks?

This has been one of my own questions since the very beginning. Clearly, edge only FCoE works just fine without TRILL. But as networks become more complicated, and virtual machines move, it seems an awfully good idea to have some protocol to alleviate East-West routing concerns. I feel much better with TRILL (or some similar Ethernet fabric technology) in a complicated FCoE network.

Should All Switches Be Full FC Forwarders?

There are number of ways to implement FCoE on Ethernet network, and not all involve building a full Fibre Channel stack in each switch. While many (including myself) assumed that FCoE implied Fibre Channel forwarding in all switches, this is clearly not the direction taken by vendors, at least initially. Perhaps the current “Ethernet forwarding” approach is only a stepping stone, or perhaps it will emerge as the dominant FCoE standard.

How Will OpenFCoE and LoM Be Used?

OpenFCoE is a software solution allowing FCoE to be run without a CNA. If this became popular, it wouldn’t be long before data center architects began looking at LAN on Motherboard (LoM) and even 10GBASE-T as a potential SAN alternative. Will this be used in the long run? It could happen, but it’s certainly not something that’s here at the moment. But OpenFCoE is a real player, especially with Intel’s backing.

How Will Technologies like Zoning Interoperate?

Many networkers are just now beginning to see the true complexity of Fibre Channel SANs. Although interoperability of higher-level Fibre Channel functions between vendors has never been a priority in “FC over FC” SANs, Ethernet could change things. I would not be at all surprised to see a groundswell of customer support demanding greater levels of interoperability from FCoE than from FC, and zoning and VSAN is the likely first beachhead.

The Big Question: When Will We See the “Killer App” For FCoE

Just about everyone agreed that the real challenge for FCoE is market acceptance. Customers aren’t yet demanding FCoE, and vendors are finding it hard to articulate a compelling case to move from “tried-and-true” FC. Convergence, cost savings, and performance have all been put forth, but customers aren’t biting. Perhaps they just need a little time and a little more proof.

This post relies extensively on feedback from a number of people, including Ivan Pepelnjak, Tony Bourke, J Metz, Dmitri Kalintsev, Derick Winkworth, David Hardaker, Juan Lage, and Corey Hines.

© sfoskett for Stephen Foskett, Pack Rat, 2012. |
Eight Unresolved Questions About FCoE


This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2012. | Eight Unresolved Questions About FCoE
Read more posts categorized as Storage

Windows 8 server editions will include a filter driver for NTFS for data deduplication

The next version of Microsoft Windows Server includes integrated data deduplication technology. Microsoft is positioning this as a boon for server virtualization and claims it has very little performance impact. But how exactly does Microsoft’s de-duplication technology work?

Introducing Windows 8 Deduplication

Let’s make one thing clear right from the start: Microsoft started from a clean sheet and invented their own deduplication technology. This is not a licensed, cloned, or copied feature as far as I can tell. There are some clever aspects to it, along with a few head scratchers for folks like me who’ve seen lots of different deduplication approaches.

Microsoft’s deduplication is layered onto NTFS in Windows 8, and will be a feature add-on for Server users. It is implemented as a filter driver on a per volume basis, with each volume a complete, self describing unit. It is cluster aware, and fully crash consistent on all operations. This is a pretty neat trick: As is typical for Microsoft, deduplication will be a simple, transparent feature.

Now let’s talk for a moment about what Windows 8 deduplication is not.

• It is a server-only feature, like so many of Microsoft’s storage developments. But perhaps we might see it deployed in low-end or home servers in the future.
• It is not supported on boot or system volumes.
• Although it should work just fine on removable drives, deduplication requires NTFS so you can forget about FAT or exFAT. And of course the connected system must be running a server edition of Windows 8.
• Although deduplication does not work with clustered shared volumes, it is supported in Hyper-V configurations that do not use CSV.
• Finally, deduplication does not function on encrypted files, files with extended attributes, tiny (less than 64 kB) files, or re-parse points.

Some Technical Details on Deduplication in Windows 8

Microsoft Research spent 2 years experimenting with algorithms to find the “cheapest” in terms of overhead. They select a chunk size  for each data set. This is typically between 32 KB and 128 KB, but smaller chunks can be created as well. Microsoft claims that most real-world use cases are about 80 KB. The system processes all the data looking for “fingerprints” of split points and selects the “best” on the fly for each file.

After data is de-duplicated, Microsoft compresses the chunks and stores them in a special “chunk store” within NTFS. This is actually  part of the System Volume store in the root of the volume, so dedupe is volume-level. The entire setup is self describing, so a deduplication NTFS volume can be read by another server without any external data.

There is some redundancy in the system as well. Any chunk that is referenced more than x times (100 by default) will be kept in a second location. All data in the filesystem is checksummed and will be proactively repaired. The same is done for the metadata. The deduplication service includes a scrubbing job as well as a file system optimization task to keep everything running smoothly.

Windows 8 deduplication cooperates with other elements of the operating system. The Windows caching layer is dedupe-aware, and this will greatly accelerate overall performance. Windows 8 also includes a new “express” library that makes compression “20 times faster”. Compressed files are not re-compressed based on filetype, so zip files, Office 2007+ files, etc will be skipped and just deduped.

New writes are not deduped – this is a post-process technology. The data deduplication service can be scheduled or can run in “background mode” and wait for idle time. Therefore, I/O impact is between “none and 2x” depending on type. Opening a file is less than 3% greater I/O and can be faster if it’s cached. Copying a large file can make some difference (e.g. 10 GB VHD) since it adds additional disk seeks, but multiple concurrent copies that share data can actually improve performance.

Stephen’s Stance

Although I am intrigued by Microsoft’s new deduplication technology in Windows 8 server, I still have many questions about its usefulness and impact on performance. Concentrating duplicate data in the system volume makes sense from a technical perspective, but could lead to an I/O hotspot on the disk. This is especially true for external caching storage systems, since there is no integration between Microsoft deduplication and storage array features. I am particularly concerned about the use of deduplication with VHD files in Hyper-V, since it could eat up valuable system RAM and impact I/O performance.

If you would like to try Microsoft deduplication for yourself, I am happy to report that it is included in the developer preview of Windows 8 that is available on Dev Center. Here are a few commands to get you started, and read Rick Vanover’s post too!

Import-Module ServerManager
Add-WindowsFeature -name FS-Data-Deduplication
Import-Module Deduplication
Enable-DedupVolume E:
get-dedupvolume

Note: I am a Microsoft MVP and Microsoft briefs me on upcoming technologies under NDA. This post is based on a Microsoft briefing from November which was said at the time not to be covered by any NDA. All of this information could be gleaned by experimenting with the Windows 8 developer preview, but it’s much easier to just go to the source.

© Stephen Foskett for Gestalt IT, 2012. | Microsoft Adds Data Deduplication to NTFS in Windows 8
Read more posts categorized as Server Virtualization, Storage

It isn’t always easy to get where you need to go!

Consider the following situation: You go to lunch with your good friends, John and Mary. Halfway through a rousing discussion of the latest Hollywood movie, Mary starts talking about the fantastic action sequences while John criticizes the romantic angle. You realize something mine-bending has happened: John now has Mary’s personality, and vice versa. It’s like they have switched brains or something!

This truly weird situation isn’t likely to happen in person, but occurs all the time in the data center. Virtualization of server, network, and storage services illuminates the link between physical resources and functional applications. A running virtual machine can instantly move from one server, network adapter, HBA, or LUN to another. And when it happens, traditional components have no idea how to react.

The Challenges of Mobility

Mobility is perhaps the “killer app” of virtualization, but it is also the killer of traditional IT systems. Let’s consider the challenges of this “Twilight Zone” moment.

• The operating system expects a consistent hardware environment, which is exactly what the hypervisor creates
• The LAN must be prepared to redirect all network traffic instantly and seamlessly to one or more new physical interfaces
• The SAN similarly must be able to reroute all I/O to a new pair of HBA’s without missing a beat
• The storage array must be able to re-present capacity to a new physical device, and must maintain snapshots and other configurations
• The backup system must also be able to maintain consistency over time even as machines relocate to different server and storage locations

All of this must be done while maintaining quality of service (QoS), access control, reporting, and appropriate segmentation at all levels. This is an incredibly challenging task, and no conventional protocol (IP, Ethernet, NFS, SCSI, Fibre Channel, etc.) is anymore ready then you are when you’re good friends switch personalities.

Two Paths

So much of the development that is currently taking place in IT focuses on accommodating this “mobility issue”. Two key approaches have emerged to take on this challenge:

• “In a vacuum” technologies (like VXLAN) assume that no other changes will be made, so the focus is on maintaining complete compatibility in front and behind
• “Clean sheet” technologies (usually from startups) take a different approach, throwing out compatibility in favor of technical elegance

Both of these approaches have merit. Attempting to maintain compatibility only works so far (just ask a Windows API programmer), but it leverages the existing environment and recognizes that most people are not ready for wholesale change. Clean sheet designs always make more sense, but they rarely attain mass acceptance. Nearly every technology we rely on today is full of bolt-ons in the name of compatibility. Some, like Ethernet and x86, actually work pretty well, too.

The Stack of Lies

The difference between virtualization and cloud computing is exactly this same distinction. Hypervisors, NPV, NAT, thin provisioning, and so many other virtualization technologies exist mainly to maintain compatibility in a vacuum. In contrast, true cloud computing dispenses with the entire stack and creates a new platform for applications.

This is, perhaps, the reason that cloud computing is not taken off in the enterprise. Simply put, IT is not prepared to ditch everything they have ever used even in the face of a demonstrably superior alternative. Currently, the highest use of cloud is behind gateways and virtualization engines that bring it back down to earth.

This brings us to the stack of lies called server virtualization. Any “modern” virtualized data center is built on lie after lie, with each level telling the other what it wants to hear. The volume manager lies to the operating system, hypervisor lies to the volume manager, and the storage array lies to the hypervisor. The same sad state of affairs allows networking and even memory to function in a virtual world.

But these shaky stacks of lies have difficulty adapting to motion, since no level truly “knows” the reality of the world around. The depressing truth is that a bowl of spaghetti like VXLAN is perhaps the highest form of art we can expect in a virtual data center.

Stephen’s Stance

As a techie, I am always drawn to clean sheet designs that offer technical elegance along with functionality. But I know that, realistically, products that assume nothing about the world around them and bend over backward to maintain compatibility are more likely to succeed. Still, I maintain hope that the issues of virtual machine mobility will be solved in an elegant way, rather than adding to the “stack of lies”.

© Stephen Foskett for Gestalt IT, 2012. | The Terrifying True Story Of Virtual Machine Mobility
Read more posts categorized as Server Virtualization, Storage

The great battle of enterprise storage is on!

The time has come to take sides on the core question of storage for virtual servers: Do you want storage intelligence to live in the hypervisor or the array? Most administrators are already lining up on one side or the other, unintentionally casting their vote while the rest flounder. But the storage industry must wake up and embrace the divide.

Hypervisor Huggers Unite!

Do you “heart” virtualization?

VMware’s vSphere dominates the world of enterprise server virtualization and has become the core element of the modern open systems datacenter. Microsoft recognizes this but has been unable to field a competitive hypervisor ecosystem for the virtual datacenter. Today, vSphere is the state of the art and nowhere is this more apparent than in storage.

In just a few years, VMware has delivered and updated a host of advanced storage functionality, from provisioning to migration and load balancing to backup and data protection. vSphere 5 includes an advanced and scalable storage virtualization layer, delivering everything a datacenter needs. VMFS sculpts basic block storage into a shared resource for virtual machines, with snapshots, policy-based layout and movement, and flexible allocation and thin provisioning.

Most VMware administrators are “server guys” and relish these features. They have never experienced an automated “storage service” like this, and the enterprise storage world has never been able to construct anything remotely as flexible, user-friendly, and functional. And Hypervisor Huggers don’t need complex enterprise storage arrays to do it: They can use basic iSCSI or Fibre Channel devices to provide performance and capacity and let VMware do the rest!

Storage DRS is exemplary of the new virtual datacenter world. Introduced in vSphere 5 (and restricted to the pricey Enterprise Plus license), Storage DRS uses the core technology of Storage vMotion to dynamically balance I/O and capacity across a diverse pool of storage. Storage DRS even uses Policy-Driven Storage and VASA to enforce tiered storage and data placement strategy. This kind of virtualization has been a “holy grail” quest for the enterprise storage industry, but they’ve never delivered on their promises.

Cheers for Storage Stalwarts!

Do you wish all this virtualization nonsense would just go away?

But not every IT environment wants be 100% vSphere focused, and many aren’t convinced that dumb storage is the smartest place for their data. These Storage Stalwarts want smarter and better-integrated storage arrays, and VMware is innovating here as well.

VMware’s Storage APIs for Array Integration (VAAI) technology is squarely aimed at this market. VAAI allows vSphere to hand off heavy storage operations to the high-end storage arrays from the major players. It works transparently, too, coordinating cloning without the kind of scripting and hair-pulling that used to require. VAAI in vSphere 4.1 also includes block zeroing support and something called “atomic test and set” which we’ll get to in a moment. Microsoft announced their own cloning integration, ODX, but it won’t ship until Windows Server 8 appears sometime next year.

But cloning is just the tip of the iceberg when it comes to Hypervisor-to-array integration. The rising army of NAS users have seen the glory of more-complete array integration for a while, and they’re not quiet about it. They love that VMware’s NFS protocol support makes storage “disappear” in vCenter, becoming just another resource with integrated thin provisioning and flexible allocation and movement.

VMware is moving aggressively to please their Storage Stalwarts, adding more VAAI support for block and file in vSphere 5. But, as the company laid out at VMworld 2011, neither access method is ideal for virtual servers. So VMware has been pushing the enterprise array vendors for ever-greater integration. They see a future where a VAAI-based protocol enables arrays to de-multiplex I/O streams from the hypervisor and intelligently handle per-VM data.

Stephen’s Stance

You can spot a Hypervisor Hugger by their big LUNs: They would rather treat storage as a bulk commodity, and array vendors should be lining up to get their business. Storage Stalwarts will jump on each new VMware innovation, finally making use of the capabilities they have spent over a decade paying for but not utilizing. The only untenable stance is trying to keep a foot in both worlds: It’s foolish to buy an enterprise array and use it as bulk storage!

© Stephen Foskett for Gestalt IT, 2011. | Are You a Hypervisor Hugger or a Storage Stalwart?
Read more posts categorized as Featured, Server Virtualization, Storage

The Micron RealSSD P320h is more than a new form-factor for the company; it also introduces their RAIN technology

This morning, Micron announced their “RealSSD” P320h SSD, bringing them into the PCIe storage market for the first time. Already a leading supplier of both consumer and enterprise SATA SSDs, as well as the world’s leading supplier of NAND flash in partnership with Intel, this move puts Micron in direct competition with existing suppliers like media darling Fusion IO.

Introducing the RealSSD P320h

The RealSSD P320h SSD can be seen as a PCIe companion to the existing P300 series of enterprise SSDs, but it is much more than that. Micron brands their entire SSD line “RealSSD”, from the consumer grade C300 and C400 (which Crucial sells as the M4) to the enterprise P300. All include Micron’s own SSD controller ASICs and flash memory modules, offering greater levels of integration and profitability for the company.

The P320h seems similar to the recently introduced P300 in terms of componentry, but the PCIe interface puts it into a different realm of performance. Although Micron’s RealSSD line is lauded for its 6 Gb SATA interface, an internal PCI express card like that P320h blows the doors off any disk interface. Micron’s presentations show the P320h delivering over many times the IOPS of their already speedy P300! As demonstrated already by numerous competitors, there is no substitute to the low latency I/O performance of a PCIe card.

The RealSSD P320h will initially be offered in two models, both in the full-height half-length PCIe form factor. The 350 GB model offers slightly lower random write performance than its 700 GB big brother, but both boast massive performance numbers. Both use 34 nm SLC flash chips and are built on a 32-channel design.

Massive Performance

The P320h delivers astonishing performance, at least in Micron’s PowerPoint illustrations. My own C300 can push 250 MB per second of sequential writes, and the enterprise class P300 is maybe a bit faster. But the P320h is rated at 2 GB per second sustained sequential write performance, nearly an order of magnitude faster. It would be difficult find an application, let alone a server, that could sustain this kind of throughput for long.

Micron boasts industry-beating throughput

Even more important than throughput is I/O performance. SSDs like my C300 excel at servicing I/O requests, and are rated at 30,000 to 45,000 random 4K write IOPS. Again, the P300 is able to match this level of performance while providing five-year reliability for enterprise applications. But the P320h delivers nearly 350,000 4K write IOPS, besting the fastest and most expensive enterprise storage arrays in existence.

Micron’s IOPS claims are astonishing

Note that all numbers in these charts are from the manufacturers’ own specification sheets: Micron P320h, Fusion I/O ioDRIVE DUO, TMS RamSan-70, LSI WarpDrive SLP-300, OCZ Z-Drive R2 e88, Virident tachIOn. I have included two of Micron’s RealSSD SATA drives for comparison purposes: Micron P300, Micron/Crucial C300.

Benchmarks should always be taken with a grain of salt, and manufacturer spec-sheet claims are doubly dubious. But Micron makes these claims, and it won’t be long before these devices are independently benchmarked.

Digging Deeper Into the Hardware: RAIN Reliability

The P320h is a first for Micron in a number of ways. Along with being their first PCIe card, it also is the first SSD to feature Micron’s so-called RAIN architecture and RealSSD Manager software. The P320h includes optimized drivers for Windows and Linux to further improve performance, but Micron is leaving it to partners to integrate the SSD with operating systems, hypervisors, or applications.

RAIN is Micron’s answer to concerns about SSD reliability in the enterprise. Although already using ultra-reliable SLC chips, the P320h introduces a raid like technology Micron calls Redundant Array of Independent NAND, or RAIN for short. Reminiscent of SandForce’s Redundant Array of Independent Silicon Elements (RAISE), RAIN arranges flash memory chips in a 7+1 arrangement to improve reliability and recoverability in the event of a failure.

Micron claims that this combination of intelligent controllers, RAIN, and SLC memory chips delivers top-notch reliability. The company measures reliability in terms of the number of full drive fills that can be sustained per day for five years. They claim that enterprise customers demand SSDs that can sustain 30 fills per day for five years, or an incredible 54,750 drive fills. That’s 25 PB of data written for 350 gig unit or 50 PB for the big 750 GB P320h!

Putting things another way, if the 700 GB P320h was pounded with sequential data at its rated maximum of 2 GB per second, it would take over nine months to wear out this drive. Just in case the customer expects to hammer on the drive constantly, they may use the RealSSD Manager software to throttle performance and ensure reliability to a given date.

Stephen’s Stance

PCIe SSDs like Micron’s new P320h offer mindbending performance and enterprise class reliability. Although expensive, these devices are in an entirely different league from any other storage option. Micron promises to bring the PCIe P320h to market at nearly $15 per gigabyte, a substantial discount over other PCIe SSD competitors. But the card will still cost more than $5000, making it an expensive add-on for most servers.

The challenge in enterprise storage is not delivering absolute capacity or performance any longer. Today’s challenge is making that capacity and performance available to applications and, ultimately, and users in the form of improved satisfaction or profitability. Micron is moving the ball forward on the hardware front, but my focus remains on software providers that will deliver this performance in a usable form for applications, hypervisors, and operating systems.

And what’s next from Micron? I expect a SAS HDD-form factor SSD shortly, and perhaps a line of PCI Express Mini Cards or “Blade” SSDs might follow.

© Stephen Foskett for Gestalt IT, 2011. | Micron Bursts Into the PCIe SSD Market
Read more posts categorized as All, Storage

Ekahau and MetaGeek are working together to bring Wi-Fi site surveying and spectrum analysis to the masses in a portable format

The most interesting products and companies at Interop Las Vegas 2011 were found around the edges of the show floor. Companies like NEC, Synology, Ciphertex, and Endace may have gone unnoticed in the shadows of towering booths of the industry titans but deserve attention. One such pairing was two Wi-Fi analysis companies, MetaGeek and Ekahau. Both work together to enable spectrum analysis and site surveying on portable devices – smart phones and tablets.

MetaGeek Makes Spectrum Accessible

One of my personal favorite presentations at Wireless Field Day in March was the awesome startup tale told by Ryan Woodings, founder of MetaGeek. While working on a wireless mouse dongle, Ryan and company noticed RF interference and turned lemons into lemonade: They repurposed the USB dongle as a spectrum analyzer and started a company to make this formerly esoteric technology accessible.

Although MetaGeek’s Wi-Spy is nowhere near as full-featured as the big guys, it’s far more accessible at 1/10 the cost. It’s cheap enough that a home hobbyist could pick one up as a way to learn about Wi-Fi and 2.4 and 5 GHz wireless spectrum. And the Windows software is easy enough that even a storage guy like me could figure out that my home Wi-Fi router was on the wrong channel.

At Tech Field Day, Ryan wowed the crowd with an early peek at an iPad app to interact with Wi-Spy captures. I ran into Ryan at Interop, and he showed me a more-polished version of that app, promising it would hit the App Store soon. He also hinted that an iPhone version would follow, and showed off email and Dropbox integration.

Ekahau Site Survey: Laptop to Android

MetaGeek had a guest in their booth from another company I’d heard of from my Wi-Fi engineer friends: Ekahau makes site survey products for laptops and was talking about an Android version as well. Ekahau’s Mobile Survey supports a variety of Android devices, including the popular Samsung Galaxy Tab and Motorola Droid.

Stephen’s Stance

Wireless Field Day exposed me to a whole new world of enterprise IT. The wireless engineers there spend days roaming through corporate facilities doing site surveys, planning Wi-Fi access point installations, and troubleshooting connectivity and interference. Most use Windows laptops, but it’s exciting to think that Apple iPhones and iPads and Android phones and tablets may also be used in the future. These devices are much more portable, with great battery life and interactive screens. And it looks like MetaGeek and Ekahau are leading the way. I can’t wait to get an update from these companies at Wireless Field Day 2 in early 2012!

© Stephen Foskett for Gestalt IT, 2011. | MetaGeek and Ekahau: Wi-Fi Analysis To Go
Read more posts categorized as Featured, Networking

I’ve got a new video podcast up and running: Raising the Floor is a series of discussions about the future of enterprise IT. I kicked the series off talking about one of my favorite topics: Cloud storage.

I invited two excellent guests to join me for this conversation:

1. Josh Goldstein, VP of Marketing, Cirtas
2. Andres Rodriguez, Founder and CEO, Nasuni

It was a pretty broad discussion, all packed into less than half an hour. I urge you to check out the podcast (and subscribe in iTunes, rss, or email) but I wanted to share a few excerpts. You can also read the entire transcript over at Foskett Services: I’ll be posting it as a series of articles over the next week!

Andres Rodriguez: I think one of the things that hurt StorageNetworks at the time was the fact that they couldn’t deploy the equipment in an efficient, multi-tenant way. And so, if you look at the new cloud architectures, (places like Nirvanix, places like Amazon) those systems are designed from the get-go to be shared among many, many users, and make very efficient use of the equipment and the software running it across that user base.

…

Josh Goldstein: Today, the cloud is built on top of process that are very difficult for most companies to replicate on their on. So, the price you’re paying to your cloud provider includes not just storing your data but also keeping multiple replicas of that data spread across different geographical sites.

You’re highly protected against not only a disk drive failure, but also an entire array failure or even an entire site failure where your information’s still is survived those kinds of events and is remaining accessible to your when you need it.

That’s something that for most organizations to engineer that level of reliability is extremely expensive and difficult for them. The cloud providers have been able to do that at scale and still deliver the capacity to you with that type of protection at a price point that’s really pretty amazing.

…

Andres Rodriguez: I’ll start there and I said anything that traditional storage world, we have file based systems. We have block based systems and that is exactly where Cirtas and Nasuni stand. We are the equivalent of say a company named EMC and NetApp for the cloud world. The approaches are complimentary. And they are both trying to solve the same problem. I’ll start on the file side but Josh can take on the block. But on the file level is really, look, if you want to have something that behaves very much like a file server, say like a NetApp box. It means you are going to have a file system, you want to have a protocol to export it locally on so something like CIFS, you are now going to have access to directory integration so that you can have access control. This is what makes a file server useful in the datacenter.

…

Andres Rodriguez: One of the really interesting things about that is that Josh is essentially describing thin provisioning in the classic sense. But unlike doing thin provisioning and this is the cloud is automatically provisioned. So he is essentially promising the volume. But then the volume is growing gradually, smoothly without any administrative interference. Without you having to worry about it; the volume is filling in its data as it goes. And that is one of the things we talked about in the beginning.

The cloud really allows you to build a different type of storage system, because automatic provisioning is really thin provisioning should be but isn’t. When you’re talking about physical linked arrays that are limited by actual hard drives, you know, running, spindles running in your datacenter.

Again, you can watch the video of the whole conversation right now over at Foskett Services, or you can read the transcripts that will be posted there over the next few days.

Disclaimer: I worked at StorageNetworks, and Nasuni is a client of Foskett Services.

© Stephen Foskett for Gestalt IT, 2011. | Talking Cloud Storage Gateways With Nasuni and Cirtas
Read more posts categorized as All, Storage, Tech Field Day Roundtable Podcast