So we can thin-provision, de-dupe and compress storage; we can automate the movement of the data between tiers; now one single array may not have all these features today but pretty much every vendor has them road-mapped in some form or another. Storage Efficiency has been the watch-word and long may it continue to be so.

All of these features reduce the amount of money that we have to pay for our spinning rust; this is mostly a capital saving with a limited impact on operational expenditure. But there is more to life than money and capital expenditure; storage needs to become truly efficient through-out its life cycle; from acquisition to operation to disposal. And although some operational efficiencies have been realised, we are still some distance from a storage infrastructure that is efficient and effective throughout its life-cycle.

Storage Management software still arguably is in its infancy (although some may claim some vendor’s tools are in their dotage); the tools are very much focused at the provisioning task. Improving initial provisioning has been the focus of many of the tools and it has got much better; certainly most provision tasks are point and click operations from the GUI and with thin and wide provisioning, much of the complexity has gone away.

But provisioning is not the be all and end all of Storage Administration and Management and it is certainly only one part of the life-cycle of a storage volume.

Once a volume has been provisioned, many things can happen to it;

i) it could stay the same

ii) it could grow

iii) it could shrink

iv) it could move within the array

v)  it could change protection levels

vi) it could be decommissioned

vii) it could be replicated

viii) it could be snapped

ix) it could be cloned

x) it could be deleted

xi) it could be migrated

And it is that last one which is particularly time-consuming and generally painful; as has been pointed out a few times recently, there is no easy way to migrate a NetApp 32-bit aggregate to a 64-bit aggregate; there is currently no easy way to move from a traditional EMC LUN to a Virtual Provisioned one; and these are just examples within an array.

Seamlessly moving data between arrays with no outage to the service is currently time-consuming and hard; yes, you can do it, I’ve migrated terabytes of data between EMC and IBM arrays with no outage using volume management tools but this was when large arrays were less than 50 Tb.

We also have to consider things like moving replication configuration, snapped data, cloned data, de-duped data, compressed data; will the data rehydrate in the process of moving? Even within array families and even between code levels, I have to consider whether all the features at level X of the code are available at level Y of the code.

As arrays get bigger, I could easily find myself in a constant state of migration; we turn our noses up at arrays which are less than 100 Tb which when we are talking in estates which are several petabytes is understandable but moving 100s of Tb around to ensure that we can refresh an array is no mean feat and will be a continuous process. Pretty much once I’ve migrated the data, it’s going to be time to consider moving it again.

There are things which vendors could consider; architectural changes which might make the process easier. Designing arrays with migration and movement in mind; ensure that I don’t have to move data to upgrade code levels; perhaps consider modularising the array, so that I can upgrade the controllers without changing the disk. Data-in-place upgrades have been available even for hardware upgrades; this needs to become standard.

Ways to export the existing configuration of an array; import it onto new array, perhaps even using performance data collected from existing array to optimise layout and then replicate the existing array’s data to enable a less cumbersome migration approach. These are things which will make the job of migration more simple.

Of course, the big problem is…..these features are not really sexy and don’t sell arrays. Headline features like De-Dupe, Compression, Automated Tiering, Expensive Fast Disks; they sell arrays. But perhaps once all arrays have them, perhaps then we’ll see the tools that will really drive operational efficiencies appear.

p.s I know, very poor attempt at a Tabloid Alliterative Headline

© Martin for Gestalt IT, 2010. | Managing Migration Makes Martin Mad!
Read more posts categorized as Storage

EMC caused a major stir on April 14 as they announced the next-generation Symmetrix enterprise storage array, the V-Max. Since that time, many of the features have been discussed and dissected on various blogs at the same time as EMC moves forward with sales of the new system. But one question remains: When can end-users actually purchase and use the V-Max system as described? And in particular, When does the V-Max get the most desirable and hyped Fully-Automated Storage Tiering (FAST) and scale-out features?

So let’s say a customer decided to purchase a new V-Max system from EMC today. What would they get?

• They would get a freshly-designed cabinet, controllers, and firmware with more cache RAM, faster CPUs, larger disk drives, and support for all of their existing Symmetrix DMX features. Beta customers actually had both components late last year, and customer orders are proceeding. So it is safe to say that the V-Max clustered array itself is currently shipping. Note that the maximum configuration is 8 engines, each with a quad-core CPU, 128 GB of cache RAM, and 16 each of host and drive channels.
• Like the DMX before it, the V-Max supports a variety of disk drive types, from flash to Fibre Channel to Serial ATA. The larger so-called enterprise flash drives (EFDs) also appear to be shipping this quarter.
• Automatic provisioning with templates and management is available now as well.
• Virtual provisioning (thin provisioning and non-disruptive relocation) are enabled and shipping, but they’re mutually exclusive (see more below).
• PowerPath/VE and vSphere Storage plug-ins are shipping, even though VMware vSphere 4 was just released.

So our V-Max customer would be able to buy and deploy a solid, scalable, next-generation enterprise storage array that does everything they expect from existing systems and a little more. This is more than many new or re-engineered storage products can boast.

But our customer does not get everything that has been talked about regarding the V-Max architecture. What don’t they get?

• Future protocols like 10 GbE iSCSI and FCoE and 8 Gb Fibre Channel are not shipping yet. No surprise there.
• One of the most important components of V-Max is its fully-automated storage tiering, called FAST by EMC, which will not be fully realized for years. This technology is a major departure for the Symmetrix, which has long had the capability to manually move volumes but lacked the sub-LUN granularity needed to truly take advantage of the newest storage technologies like flash SSD. Although EMC plans to release “FAST v1″ during the second half of this year, this functionality merely automates the existing LUN relocation capability of all current Symmetrix arrays (for non-thin-provisioned LUNs) and is nowhere near the promise of FAST. Full FAST will not ship until 2010, and we suspect it might be well into that year.
• True Virtual Matrix mode, where multiple V-Max array “clusters” link together virtually to form a system larger than 8 engines. EMC has also talked about extending this mode over distances, creating a geographically dispersed array. None of this is coming any time soon.

So the answer to the core question of whether the EMC Symmetrix V-Max is shipping is yes. Customers can go out and buy one. But they will not get all of the functionality discussed and presented by EMC and its representatives last month.

Not So FAST

Fully-Automated Storage Tiering (FAST) is one of the most significant capabilities of the V-Max platform. As described, FAST will not be available for a year or more. This is not a secret or a scoop – EMC is the first to admit that FAST was pre-announced and is “Coming Next”!

V-Max's FAST technology "automates movement and placement of data based on changing needs" (illustration courtesy of EMC)

What is FAST? It is an automated tiered storage technology that can place data on different tiers of storage based on requirements. High-performance data on flash, bulk data on SATA, and so forth. EMC V-Max marketing materials, and the launch content, spent much time focusing on this technology, telling us that it automatically adjusts tiering of data, not just volumes, and reacts as needs change, rather than statically placing data. We’ve reported some confusion about just how FAST accomplishes this in the past, but we all agree that this would be an awfully nice way for an array to manage data.

At the core of FAST is EMC’s recently-introduced virtual provisioning technology. Why is FAST so much better than VP, and other competing automated storage tiering technologies? According to EMC blogger, Barry Burke, FAST has five key benefits:

1. Change RAID protection types as LUNs and volumes are relocated
2. Relocate a large number of LUNs and volumes concurrently
3. LUN and volume relocation is very quick
4. LUN and volume relocation has low performance impact on the array, and on the applications using the data being moved
5. LUN and volume relocation is done without disrupting replication

Let’s give EMC some credit for its FAST vision, as well as the pieces that have already been shipped. But we’re still quite a way off from seeing this FAST vision realized in production technology:

1. Current DMX and V-Max hardware allows customers to use multiple drive types, including flash SSD EFD, for over a year. Many other vendors are just now getting around to shipping flash disks. No one would claim that this is “FAST”.
2. Current DMX and V-Max also allows thin provisioning (a component of EMC’s Virtual Provisioning). This isn’t FAST either.
3. Current V-Max software allows customers to non-disruptively relocate volumes between disk and RAID types (another component of Virtual Provisioning). This is very nice to have. But this is not fully automated storage tiering. It’s just easier storage tiering. And it doesn’t apply to thinly-provisioned volumes.
4. FAST V1, which EMC plans to ship in the second half of 2009, will automate the selection and migration of volumes to different disk and RAID types. Although this is technically fully automated storage tiering, it does not quite live up to the promise of FAST as described by EMC at the V-Max launch. And it still won’t support thin (Virtual Provisioned) volumes!
5. Finally, EMC claims that they will fully realize the FAST vision with FAST V2, which will ship in 2010. This would enable granular (sub-volume) tiered storage for thinly-provisioned volumes.

Why does it matter that EMC won’t ship FAST V2 until next year? Simply put, anything less than FAST V2 will still not effectively utilize the expensive EFD capacity paid for by EMC customers. Even FAST V1 will take up expensive high-performance flash capacity with the entire contents of a LUN, including empty space! Customers wishing to conserve EFD capacity will use virtual provisioning to only store “used” data on flash. But these cannot be easily relocated if demand does not meet expectations, and not all of that used data requires high-speed flash storage. EMC also lacks thick-to-thin (or thin-to-thick) technology, though we suspect they’re working on that.

So customers have a choice: They can either use their EFDs more efficiently with thin volumes, or move data between EFDs and disks as needs change. They cannot have both.

An open question is how much EMC will charge for FAST V1 and V2. Assuming they are like Apple with the iPod Touch, and their history affirms this, EMC will charge for FAST as each major new capability is delivered. EMC has always charged for major new features. This is due to accounting rules which prohibit a company from charging customers ahead of releasing major features. The only way around this would be for EMC to recognize V-Max revenue as recurring over time, as in the case of the iPhone at Apple. Fat chance of that! Note that EMC is also talking about bringing FAST to CLARiiON and Celerra, but those timetables are not disclosed.

How Big Is Your Virtual Matrix?

Many have wondered about EMC’s choice of a name for this new storage system, some speculating that since the “V” stands for “virtual”, the V-Max was designed for server virtualization environments. EMC CEO Joe Tucci’s comments at the V-Max launch seemed to indicate this as well.

However, while it certainly is targeted at VMware vSphere and similar systems, the “V-Max” name actually has a very specific, and very different, meaning. “V-Max” is actually a reference to EMC’s new “Virtual Matrix” architecture, which allows V-Max engines to combine into a scale-out array with a virtual matrix of interconnects. Like FAST, the Virtual Matrix Architecture is the real vision of revolutionary storage from EMC. And like FAST, this vision is nowhere near being realized.

V-Max is so far being sold as a stand alone machine, not a scale-out cluster. At current GA, EMC is only offering 8 engines that are part of the same system cabinet (physically connected through MIBE and SIB), but there is nothing available that would connect 2 of these system cabinets, where instead of 8 engines you are now running 16 or 32 engines, forming a real V-Max. 

The V-Max name refers to the Virtual Matrix architecture of the array cluster (illustration courtesy of EMC)

EMC’s “vision” clearly extends well beyond 8 engines and a single V-Max.  EMC’s design allows for a federated set of V-Max machines that can all be managed as a single entity, and the company alluded to this in their product introduction.  They suggested that these separate V-Max machines wouldn’t even have to be in the same place, physically.  

Although the vision articulated at the product’s introduction talked about massive scaling in multiple locations, EMC’s maximum current configuration for the V-Max is 8 engines with up to 128 Xeon CPU cores, 1 TB of RAM, and 128 each front-end and back-end storage interfaces linked to 2,400 disk drives. This is no tiny system, to be sure, easily doubling the current DMX-4 array’s scalability as well as besting the competition. In fact, EMC can rightly claim that a V-Max can be configured today to be the largest integrated enterprise storage array available.

But 8 engines linked by local RapidIO connections in a single data center isn’t a revolution. The revolution happens when EMC realizes its vision of linking “dozens” of engines, thousands of ports, and tens of thousands of drives into a geographically dispersed system. Former EMC executive, Dave Donatelli, now controversially linked to HP, claimed that the V-Max can support up to 256 engines in a product launch video that has now been removed. IDC analyst, Benjamin Woo, talked about EMC’s ability to support a “single namespace” across geographies during his launch coverage. EMC’s Chuck Hollis says that “the overall architecture can eventually grow into literally thousands of ports, thousands of processing cores, tens of thousands of disks, and many terabytes of cache memory — all operating as a single, tightly-coupled array.” EMC’s Barry Burke also suggests that future revisions to the Virtual Matrix architecture could scale well beyond anything hinted here.

Again, in that same post, EMC’s Hollis tells us that the architecture delivers four key benefits in terms of scalability (bulleted for your pleasure):

1. “The difference between the two?  The V-Max can grow and grow from that modest starting point — still capturing the economics of mid-tier arrays and linear cost scaling.
2. Second, the V-Max array isn’t limited to a single cabinet with short wires.  Think in terms of multiple cabinets, separated by many meters or — in the future — longer distances using optical connections – all behaving as a single, giant array.
3. Third, the scaling is utterly linear and modular.  As more building blocks are added, the array gets bigger and bigger, and there’s no need to replace the frame, buy a bigger cabinet, etc. — nothing gets thrown away.
4. Fourth, there’s no assumption that all the building blocks are exactly the same.  Some may be big, some may be small, some may be old, some may be new.”

So a reader would look at this and think to himself, “this new V-Max starts as cheap as a competing midrange array and can scale out with different building blocks to span my entire data center, or even my entire enterprise.” This is a great promise, and awfully near to the holy grail of enterprise storage.

The problem is that none of this is possible today.

1. The V-Max starts at “about $250,000″ for a single engine. This is much, much more expensive than competing midrange arrays. It has to be, since a single V-Max engine has a lot more horsepower and componentry than any midrange array!
2. All V-Max connectivity is currently done with RapidIO connections in “a single cabinet with short wires.” Perhaps you could define “cabinet” differently and claim that the V-Max uses multiples, but no one (especially an EMCer who wants to keep his job) would suggest that the V-Max engines can be “separated by many meters or … longer distances.” When Chuck says this is “in the future” he means it!
3. V-Max scales to 8 engines, and initial reports are that scaling is indeed fairly linear. So this is a fair statement. But what if EMC replaces RapidIO with InfiniBand or datacenter Ethernet? Would there really be no need to “throw away” components?
4. There are no “big”, “small”, or “old” V-Max building blocks. They’re all new, and they’re all identical, and you can have up to eight of them. EMC says you will be able to mix and match once new building blocks come out, and we expect them to live up to this promise.

So even today’s V-Max system does use the Virtual Matrix architecture. It’s got a virtualized matrix interconnecting the engines, after all. But it’s nowhere near the scale-out promise of the Virtual Matrix concept. That’s quite a ways off still.

Conclusion: V-Max Is Not Overtaking The Future Yet

It is important to note that EMC sales literature does not claim capability beyond what’s currently on the truck. The V-Max data sheet, for example, makes no mention of FAST or scalability beyond 8 engines. And EMC’s Dave Graham is characteristically straightforward listing what’s in the box. In fact, EMC’s official web site and literature probably errs on the too-cautious side: A casual visitor might well ask himself, “where’s the beef?”

Why might an average storage buyer think there is more “beef” to be had? One only needs to watch EMC’s V-Max launch, and the surrounding coverage. Indeed, EMC’s own theme for the launch was “Overtake the Future!” EMC’s customers expected a revolution, and a revolution is what the company promised.  This isn’t anything new, unexpected, or even out of place. Companies often tout the theoretical future capabilities of their new products but don’t deliver on those promises for some time. Many have accused Microsoft of using this tactic to combat Windows competitors, for example. And let’s not forget the industry’s fascination with legendary vaporware products!

But the company’s boosters seem to want to have their cake and eat it, too. They’ve poured on the verbiage describing FAST, and used this future capability to declare victory in the battle against the likes of HDS, 3PAR, and Compellent. EMC’s Hollis tells us that “there’s nothing remotely like V-Max in the marketplace today”, and other EMC folks have made similar claims. As Wikibon points out, “EMC’s pre-announcement of Fully Automated Storage Tiering (FAST) is an admission that without FAST the answer is essentially no, EMC can’t compete on cost.” In other words, EMC was forced to prematurely unveil their FAST vision to keep their customer base from jumping ship to competitors already offering similar functionality. The same is true of the scale-out capability of the Virtual Matrix architecture.

For now, EMC has merely introduced a solid next-generation enterprise storage array. V-Max without FAST and scale-out is a nice-to-have refresh of the Symmetrix DMX-4. EMC should be especially respected for leaving all of the core Engenuity features like TimeFinder and SRDF intact, since this must have been no small feat. V-Max with FAST V1 is a more attractive proposition, but does not go that far beyond the capabilities of Symmetrix Optimizer. The same can be said of the 8-engine V-Max limit – it’s larger, but nothing to get too excited about.

V-Max with full-on FAST V2 and multi-site heterogeneous scale-out is the real revolution in EMC storage. If everything plays out, you could have several V-Max systems in the same data center, or perhaps even different data centers, all being managed as a single entity.  Then layer FAST on top of that (at the block level), and you have something unlike pretty much anything else on the market today. The question is, when can they achieve this, and will some of the other vendors catch up in the meantime?

This is what observers, analysts, and customers were looking for. And this is what they might think they saw. But in reality, we’ll all just have to wait another year or two and see what develops.

© Gestalt for Gestalt IT, 2009. | EMC Symmetrix V-Max: When Does It Get FAST and Virtual?
Read more posts categorized as Exclusive, Featured, Storage, Top Story

Despite working for a vendor, David Merrill has a habit of posting some very good entries full of common sense; I find myself nodding in agreement with much of what he posts. His latest couple of entries here and here had me nodding in agreement; it’s not just the vendors who are guilty of some dubious voodoo economics, I’m sure that most of us have put together business cases which if were really scrutinised, don’t really stack up.

We often talk about trying to make capital acquistions cost neutral in less than eighteen months; a reduction in Opex to offset the capital cost. Vendors are often complicit in this, as I mentioned in my previous entry, inflated maintenance costs mean that is often cheaper to refresh and take the bundled maintenance offered with a new system than to continue to pay maintenance on the legacy kit.

However, if I examine the failures that we tend to have; it is generally the moving parts which fail; you know those things which spin at speed? Yes, the spinning rust. And if there is one thing which has fallen in cost; it is spinning rust.

Okay; with the very much older disks, vendors simply can’t get new drives that small but I assume that most of you are aware that a large number of maintenance replacements are not actually new components? They can be previously failed and reconditioned components or perhaps pulled from arrays which have been migrated to the latest and greatest technology.

Maintenance in the IT industry is a fantastic example of Voodoo Economics…but hey it’s green, well they are recycling and re-using! But remember, there is a third part to that; REDUCE!

Vendors don’t have any real incentive to reduce maintenance costs; it firstly enables a constant upgrade treadmill because if you really had to evaluate the value of the new features, life would be a lot more complex but if you don’t upgrade, maintenance is a very nice and high margin activity.

Actually EMC should be thanking companies like HDS and IBM; it enables people to keep their legacy arrays around for a lot longer and hence keep paying EMC high maintenance! And no I’m not saying that EMC’s maintenance charges are especially high, there are much worse offenders out there!

© Martin for Gestalt IT, 2009. | Maintenance Madness
Read more posts categorized as All, Storage

I sat in a meeting today where the subject of how often you refresh your storage infrastructure came up. I know that many companies are working on a three-five year model but we were discussing whether this should be increased to seven and what needs to happen to make this so.

There are a few reasons why we were coming to this conclusion; firstly spinning rust in the Enterprise is probably at it’s peak and actually anything over the current maximum size of a spindle has potentially limited use i.e anything over a 1-2 terabyte drive is not especially useful for shared storage infrastructure. Please note, I say shared storage infrastructure!

Larger drives may still have a place to play in your archive tier but even that is debatable. And if you look at most Enterprise end-user desktops; they often have rather small local drives. It is the home-user and their insatiable demand for storage which really drives the size of spindles now.

We also know that the performance of the spinning rust is probably not going to improve dramatically. So what does change? Well, yes we have the introduction of SSDs and a couple of things mean that a four-five refresh cycle for that technology is probably sensible. And then there are the storage controllers themselves; these don’t especially wear out but technology does move on. 

But the current designs of arrays mean that when we refresh; we are forced to refresh the lot. We are also forced to refresh by overly inflated maintenance costs. Let’s be honest; most refreshes are justified by cost savings on the OpEX i.e maintenance. Even if I go to a virtualised infrastructure as espoused by HDS or IBM; these maintenance costs still mean it is often more attractive to refresh rather than sweat the asset.

However the current economic climate means that we are now more closely beginning to examine the model of keeping things for longer and examining our maintenance budgets very carefully. Dropping maintenance for software which is now stable and at terminal releases; potentially talking to third-party maintenance organisations who are much more willing to support legacy kit at a reasonable cost.

And we are considering strategies which enable us to continue to make use of kit for longer. VMWare’s announcements today allowing replication and thin-provisioning at the hypervisor layer for example.  So funnily enough, EMC have come round to external storage virtualisation; you just buy it from VMWare as a software product.

It’ll be interesting to see what other traditional storage related functionality makes its way into the hypervisor. And at what point EMC realise that they are actually selling ‘traditional’ storage virtualisation but as a software product and at which point that they do become a software company.

Funny old world, as EMC slowly catalyzes into a software butterfly selling storage virtualisation, Oracle becomes a hardware grub. And in the space of a week; EMC ‘kill’ DMX with V-MAX, then they kill V-MAX with vSphere. Now that’s what I call progress!

© Martin for Gestalt IT, 2009. | Investment Strategies and Virtualisation
Read more posts categorized as All, Storage

Whilst HDS and EMC throw rocks at each other with regards to whether it is better to build custom parts or take things off the shelf and just use custom when you require (I’m expect the other Barry to sit on his hands but there are good reasons why the SVC team decided to build out of commodity parts and I suspect that they are very similar reasons to EMCs). I think we should look beyond the hardware and look at what is coming down the line to us.

The most important thing roadmapped is FAST, Fully Automated Storage Tiering. FAST changes things; it takes a whole bunch of ideas from a whole bunch of places and runs with them. If you are another vendor and you feel aggrieved that EMC have stolen your idea; just take heart, it won’t be the first time in history that this has happened and it won’t be the last.

The foundation is Wide-Striping* using a model which splits your data into chunk(let)s and spreads it across spindles. Once these chunks are distributed, you can monitor the characteristics of the I/O at an individual chunk level; this allows us to do tiering at a sub-LUN level. A hot chunk of data can be moved to a higher tier and a cooler chunk of data down into a lower tier.

In the past we have been limited to moving a whole LUN (with the exception of Compellant); this has always been a time consuming job, identifying what needs to move and then moving it. Yes, technologies have come along to make this easier but to sweat the asset and especially to make best use of SSDs; we needed to move individual ‘blocks’ as in a given file-system , it is possible that only some blocks are hot and frequently accessed. Traditionally if you could, you would hold these in cache but if SSDs are expensive, cache is yet more so. This approach will allow some cache to be replaced by SSDs and for some cache unfriendly workloads, to all intents and purposes, you have massively increased the amount of cache available. You might not want to hold a terabyte or so of real cache for that evil 100% random read app but with SSDs; this becomes viable and not at a huge utilisation hit.

But there are going to be issues with the FAST approach; firstly, where do you put a new workload? If you simply assign it some disk and let the array decide, what the hell is it going to do with the workload? It could put it on the slowest tier possible and then migrate up; it could stick it on the fastest tier and migrate down. Both of these approaches have significant risk, so I suspect we are going to have to give the array some clues and we are going to have to understand more about the whole system we are putting in. The difference in performance between the top tier and the bottom tier is going to be large.

No longer will the Storage Admin be a Lun Monkey; they are going to need to really understand their workloads and the applications. They are going to need to learn to talk the application developers and understand workloads, they are also going to have understand business cycles.

For example applications which spend 11 months of the year pretty idle may suddenly at year end need a lot of performance. What happens if all your applications demand stellar performance once a year? Perhaps you need a way of warning the array that it needs to prefetch a load of data. A badly written end-of-year reporting extracting which generates thousands of random read IOPs. A badly written user-generated SQL; in the past, this just crippled the application; with FAST, this could cripple the whole array as it tries to react.

The FAST approach is potentially the thin-provisioning of IOPs. This going to need a lot of thinking about. Potentially you will have to domain storage to protect applications from the impact of one another. We are going to need to know more about the whole system than we have before if we are to truly benefit from FAST,

Building rules which suit your applications; sure, V-MAX will come with its own canned rules for things like VMware and known applications. Indeed EMC will probably be leveraging all the performance data that they have been gathering over the years to help us write the rules. Storage Templates as described by Steve Todd here are just the start.

So although at one level, the Storage Admin’s job could get alot easier; the Storage Manager’s job has got a whole lot harder. Yes Barry, I asked for FAST and now you’ve given it to us; now we’ll have work out what this all means!

I have some really ‘interesting’ ideas as to where EMC could take V-MAX but they’ll have to wait for another time as I’m still supposed to be on leave from Enterprise IT.

* It’s Wide Striping not Wide StripPing as I keep seeing written; Wide Stripping is what happens on a Rugby Tour after a good night out!

© Martin for Gestalt IT, 2009. | FAST and Furious
Read more posts categorized as All, Storage

© Chris for Gestalt IT, 2009. | EMC Announced Next Generation V-Max Architecture
Read more posts categorized as All, Storage

I found David Merrill’s blog entry here on Squeezing (Easily) into Tight Jeans amusing. David is talking about a couple of his customers who were using various capabilities to reduce the amount of storage they needed; I suspect using techniques such as thin-provisioning and the ability of the USP-V to consolidate islands of storage into a usable pool of storage.

And then they were going to decommission a whole bunch of arrays and reduce the amount of storage on the floor. I think David was surprised that they were choosing to decommission the storage as opposed to simply use the reclaimed storage for growth.

But sitting on this side of the fence, the customer side; this is no big surprise at all. Depending on the age of the arrays and depending on the software sitting on the arrays and especially if the arrays were out of warranty periods; the maintenance costs are generally so high that it simply does not make economic sense to keep them around.  

Software maintenance on all of the Enterprise class arrays is just plain expensive. If you then factor in that if you are trying to sweat an asset for a couple of extra years; that is another couple of years of what are often power and space inefficient arrays and you are going to be looking at another migration effort in fairly short order, it does not make a huge amount of sense.

The situation is actually a lot less clear on mid-range arrays as the maintenance costs are often considerably lower but if you have got aging Enterprise arrays; get them out if you can.

© Martin for Gestalt IT, 2009. | Economic Realities
Read more posts categorized as All, Storage

Lets have a Recap

So Cisco announced its server today. A few quick bullet points if you didn’t pay attention:

1. Cisco announced something that is more like a fabric of servers, or a network of servers. I think they run VMware only and they have very large CPU/Memory/IO bus architectures but standard Intel for all that.
2. They have shoehorned a big arse network blade into the chassis that connects to network and storage. Plus it is configuration mechanism. That is, a central configuration fabric for clusters of bare metal servers to act in concert and become a single or many virtual instances. Then VMware runs on top of that. One term I heard is Processing Area Network which seems to sum it up.
3. Lots of storage vendor love from NetApp and EMC.
4. Same for the usual, but large, crowd of “me too’s” like Accenture for consulting, Emulex etc for specific accessories
5. Unusually, Microsoft came out supporting the strategy.

That’s probably enough. If you to geekdive into some infromation check out Colin McNamara who has obviously be playing with this at Cisco. He has some good details there. Recommended.

References

Chuck Hollis at EMC expressed serious love in his blog.
Marketwatch got the first press release, its a good enough round up.

My Top Five Takeaways

Number 5 – Cisco outplayed IBM & HP at their own game

“Cisco and VMware today announced a comprehensive, strategic original equipment manufacturer (OEM) agreement which will incorporate product engineering and integrated sales” – by getting VMWare on the same team and acting as their OEM, HP & IBM can hardly get too pissed. Cisco got a better position and outmanoevered them.

HP, IBM and to a lesser extent, Dell, have failed to make significant commitments to virtualisation because they think it means less sales. Which is stupid, it means less sales of bigger hardware with more profit.I’m betting that boneheads inside IBM/HP/Dell cannot really understand the difference between volume and profit. Not really, not enough to actually do anything anyway.

On the other hand, Cisco is not taking HP/IBM/Dell head on so they can stay “frenemies”. This isn’t taking away desktops or SME servers or even traditional servers. But is does steal their leadership on the next generation of technology and thus kills their growth. Even if HP/IBM/Dell pull some technology out of the hat that isn’t from Intel (and that is very unlikely), it will be too late as the Cisco Markenaut will have stifled any opportunity.

For a marketing triumph that nails the coffin shut, Cisco has brought Microsoft (can you believe that), plus EMC, NetApp and raft of other vendors onto their team.

Number 4 – Huge Professional Services Opportunity

Bifurcation of Professional Services Delivery – as usual. On one hand Cisco is partnering with their Data Centre partners (accenture, Tata, et al) but also ” Cisco will also offer virtualization consulting services to help customers accelerate the deployment of Unified Computing Systems across their datacenters…”.

And you when accenture lines up to endorse something you know that consulting is in the mix. They can sniff it out and latch on quicker than a vampire can get to blood in a blood bank.

So Cisco has the resellers on side, and those Cisco reseller partners that signed up for Data Centre specialisation will have huge grins today.

Number 3

Infrastructure Orchestration – thats like the One Ring in Lord of the Rings. One interface to configure all the resources, storage, networking, CPU, memory inside the server frame/network. More importantly they are fully abstracted from the OS, and this includes VMWare. This can be accomplished with a spare “bare metal” server where a new host can be created on the fly, all the way down-do the same NIC configuration as the original server.

Folks, this is grade A, number one goodness. Dollars are going leap out of Enterprise pockets to grab hold of this. Lets pretend that server people will actually understand the concepts here (and many will not), but my understanding suggests that I can create hardware architectures that look exactly like my current servers and then drop my current OS/Application software directly on top.

Imagine porting that CiscoWorks server you built three years ago directly onto emulated hardware, on emulated virtual machine, in virtual platorm. It needs no changes and no migration.

Dream or reality but that’s the marketing message today.

And make no mistake, this is the Enterprise product for building private clouds. None of the namby pamby pretending that public clouds will work and save the known universe from all ills. Public clouds are fifteen to thirty years away when the applications will be ready. This is ready now.

This product allows us to spin private Enterprise clouds. I call this Misty Computing. Its close, on the ground, and somewhat tangible.

I’m still not convinced about FCoE, but it doesn’t look like there much choice to date. I’m still hopeful though (Hi Doug and Omar )

Marketing – oh boy, it’s gonna be hard

Sadly, we are going to see an enormous marketing push from Cisco. I don’t know if I can stand it, but, lets face it, most people are not going to understand this technology and will need to be pushed, cajoled, shoved, encouraged and led to water before there will be uptake.

And I’m tired of that. Just tell me the truth and let me get on.

Shiny – its really shiny. And I Like SHINY.

Told you it was SHINY.

Conclusion

It’s interesting that, in a way, it looks as if Cisco has produced a network of servers. Compared to the current methodology of putting servers into your network, this makes sense now that virtualisation means that OS, and applications in the future, will be fully mobile (not bound to single silicon instance).

Whether customers are ready for this product remains to be seen, but so far, it’s hard to believe it won’t be successful. I suspect that HP/IBM/Dell just lost a significant portion of the Enterprise Cloud market.

© Etherealmind for Gestalt IT, 2009. | 5 Things About the Cisco Announcement and a Wrap Up (Post Prandial)
Read more posts categorized as Cloud Computing, Featured, Networking

CC-by-SA 2.5 image "Golf Bunker" by Ken123

Today, many companies are considering a compete migration from a physical to a virtual infrastructure. Based on the promises of cost savings, administrative efficiencies, and improved resource utilization, virtualization is seen as the technology that makes it possible to do more with less. However, there are many pitfalls to consider when virtualizing server infrastructure. This article suggests planning decisions to be considered by the CIO, IT Director, design architect, and IT Manager that, if overlooked, could ruin the total cost of ownership (TCO) and the return on on investment (ROI) expected from this virtual infrastructure.
�

Migration pitfalls

Implementing virtualization is not a simple process of building a server. Too often, management makes the mistake of expecting that the administrators and architects of the server team will be solely responsible for virtualization decisions. In reality, the impact of a move to virtualization should be viewed exactly as if a company were moving their physical servers to a new data center. Storage, networking, security, Active Directory, messaging, web presence, and all of the systems and services necessary to the organization must be considered together. Some of these groups should be more involved than others in the actual planning and ultimate design decisions, but virtualization strategy affects everyone. Key members from all of these focus areas must be trained and given panning responsibilities to achieve a collaborative plan and design.

A potentially larger “gotcha” once everyone is involved in the planning process is the temptation to include all of the upgrade and service redesign projects that have been “in the queue” for months (or longer). Although the migration may seem like a great opportunity to upgrade to the latest operating system, separate instances from poorly performing database servers, or build the new CRM system, these added activities bring complexity and could pose road blocks to the core goal of the virtualization initiative: consolidating hardware. These projects should be tabled and taken on later, after the new infrastructure is in place. In fact, the flexibility of a virtual infrastructure will actually make these projects simpler.

Performance pitfalls

Condensation of numerous physical servers into a virtual environment is intended to increase the effective utilization of processing power and other system resources, but it is easy to overshoot the target and overload the new system. Although the average load of a number of servers might be within the capabilities of a new virtual infrastructure, resource spikes can be a serious problem. Consider that periods of high demand tend to cluster across servers through the day and week, as employees begin the workday, the backup process begins, or a software build is kicked off. If performance is not measured and correlated carefully before migrating to a virtual infrastructure, these resource spikes can make the new infrastructure unacceptably slow.

One often overlooked performance challenge is the randomization of demands introduced by virtualization. Traditional computer design assumed that processes would proceed through their steps sequentially, with the system devoting resources from start to end. Multitasking operating systems challenged this assumption, but the allocation of memory, I/O, and storage remained clustered. But highly-utilized virtual servers upset this predictable flow, rapidly switching from one task to another entirely different one. In short, virtualization randomizes accesses that were once sequential and transforms large requests into many tiny ones. CPU manufacturers have responded with clever techniques to save and restore registers and handle memory mapping, but this blender effect is far from solved. I/O and storage systems are particularly affected, as they are optimized to stream large sequential operations rather than small random ones.

Storage pitfalls

Although one might think that virtualization reduces storage requirements as systems are consolidated, the opposite is often true. Many of the functions that help drive down CAPEX and OPEX costs in virtualized environments require the use of shared networked storage like SAN or NAS. Data that had once been scattered around the data center on internal drives is consolidated on these networked storage devices. Thousands of redundant copies of operating system files, for example, end up sitting on the storage array. The ease of creating virtual machines from templates tends to lead to virtual “server sprawl” far worse than in the physical world. Storage array capacity is also needed for swap space or paging files as well as popular enhancements like snapshots, DR copies, and backup images.

The result is a flood of redundant data that must be taken into account when planning capacity needs. Although storage is typically not the top item in an IT budget, virtualization can cause it to rise and chew up some or all of the savings from the project. It is difficult to avoid these issues, but some techniques can help mitigate the problem. For example, deduplication of primary storage and thin provisioning can help reduce the storage footprint of the virtualized infrastructure.

Administration pitfalls

Virtual sprawl can cripple infrastructure support teams if it is not managed. The challenge of administering this new virtual environment is often ignored in the implementation rush. Rather than managing 3,000 servers, many wake up and find that they are managing 30,000 virtual machines. This is partly because server build times are reduced from days to minutes, and partly because server consolidation on virtual infrastructure is so effective that idle guests are barely noticed. If left unmanaged, the “gasoline fire” spread of VMs could create a burden in the form of permissions, backups, upgrades, patches, and monitoring than what existed in the physical infrastructure.

To combat these scenarios, IT departments need to start with process and policies. Change control becomes paramount. Second in importance is a new server request and approval process. Build standards need to be created and adhered to. Finally, virtual servers must to be audited for activity and then powered off or removed if idle. The good news is that there are numerous life cycle management and automation tools now available for helping an organization provision, maintain, report, and decommission virtual machines.

Backup/Restore pitfalls

Backup is another area where TCO and ROI, as well as performance, can be negatively impacted by virtualization without appropriate planning. As mentioned above, virtual server hardware is often sized based on the average server load. For example, consider the impact of combining ten servers with average CPU utilization of just 10% onto a single physical device. The ROI would appear to be excellent, but the large spike in utilization that happens during backup can cause unacceptable performance issues or even application failures. The performance hit during backup can be mitigated through off-machine backup using like VMware VCB or storage snapshots, but the cost and effort of implementing these features must be taken into account when planning the new system. The conventional approach to backup, loading backup agents on each system and backing them up on a schedule, does not translate in the virtual world.

Disaster recovery pitfalls

One of the primary driving factors for moving to a virtualized environment is its potential positive impact on disaster recovery (DR). Although the DR benefits are real, the requirements for the storage system can become a pitfall. Extra DR data is pushed onto storage arrays and then often replicated to another location. All of this extra data requires increased storage capacity, array features, and WAN bandwidth. The capacity issues can potentially be addressed using primary storage deduplication and thin provisioning, and WAN optimization appliances can reduce bandwidth requirements.

While virtualization does make servers more portable, thus making DR easier, the disaster recovery copy process is a major challenge. Some vendors now have tools like VMware Site Recovery Manager (SRM) to assist in this orchestration, but these tools are somewhat immature, and care should be taken when planning to use them. For example, most do not address failback, so administrators may find themselves doing a lot of work to return to operation at their primary data center after a failover.

© Gestalt for Gestalt IT, 2009. | Planning for Virtual Infrastructure: Avoid the Pitfalls
Read more posts categorized as Exclusive, Featured, Server Virtualization, Top Story

© Chris for Gestalt IT, 2009. | Data Migration Strategies – Part III
Read more posts categorized as All, Storage