EMC FAST

EMC FAST (Fully Automated Storage Tiering) is actually a suite made of of two different products. the first, called FAST Cache operates by keeping a copy of “hot” blocks of data on SSD drives. In effect it acts as a very fast disk cache for data that is currently being accessed while the data itself is being stored on either 15K SAS or 7200 RPM NL-SAS (SATA) drives.

FAST Cache provides the ability to improve the performance of SATA drives, as well as to turbo charge the performance of fiber channel and SAS drives as well. In general, this kind of technology helps to divide performance from spindle count, which helps drive down the number of drives required for many workloads, thus driving down the cost of storage, and the overall TCO of storage.

The other product in the FAST suite is FAST Virtual Pool.  This is the product that most people associate with FAST since it is the one that leverages  three different disk technologies, SSD, high speed drives such as 15K RPM SAS, and slower high capacity drives such as 7200 RPM NL-SAS. By placing only data that requires high speed access on the SSD drives, data that is receiving a moderate amount of access on the 15K SAS drives, and putting the rest on the slower, high capacity disks EMC FAST is able to drive the TCO of storage downward.

NetApp FlashCache

NetApp approaches the overall issue of improved performance while simultaneously driving down the TCO of storage in a different way. NetApp believes that using fewer disks to store the same amount of data is the best way to drive down TCO. Therefore NetApp has spent a significant amount of time developing storage efficiency tools to help their customer’s store more data in less space.  For example, they developed a variant of RAID-6 called RAID-DP which provides the protection and performance of RAID-10, while utilizing significantly less space. NetApp has also developed block level de-duplication which can be utilized with primary production data.

However, as with many technologies of this type there could be a performance penalty paid for it’s utilization. Therefore, Netapp needed to develop a way to improve the performance if it’s arrays while also supporting it’s storage efficiency technology. With the advent of Flash memory, Netapp found a way to do this without any need for significant changes in the architecture of it’s arrays. Thus was born FlashCache.

FlashCahce provides a secondary read cache for hot blocks of data. This proves a way to separate performance from spindle count,  and thus not only allows workloads intended for Fiber Channel or SAS drives to potentially run on SATA drives, but it also addresses some of the performance issues with the storage efficiency technologies that NetApp developed. For example, with FlashCache utilized in a virtual desktop environment Netapp de-duplication allows many individual Windows images to be represented in a very small footprint on disk. However a problem arrises when a large numer of desktops all try to access their Windows image at once. However with the addition of FlashCache, most, if not all of the Windows image would end up being storage in Flash memory, thus avoiding the performance issue of a boot storm, virus checking storm, etc.

Conclusion

Both EMC and Netapp have developed ways to help both improve the performance, and drive the TCO of storage downward. the two vendors approached the problem is somewhat different ways, but in the end they have both solved the problem in unique and effective ways.

The NetApp technology requires that the user buy-in completely to the NetApp vision of storage efficiency. If the user ignores the advantages of de-dupication in particular, or has data or workloads  that simply don’t allow for the application of the NetApp storage efficiency technology then the TCO saving that NetApp promises will not be achieved. Utilizing FlashCache to seperate performance from spindle count is also critical in maintaining the performance of the array. This separation of performance from spindle count also in and of itself drives dwn the number ofd drives needed to support a workload, and thus also drives down the TCO.

The EMC technology requires a very good understanding of your application workloads, and careful planning and sizing of the different tiers of storage. EMC could do more to make the two sub-products work together so that a single solution could provide both the TCO and the performance improvements at the same time. However, EMC FAST is a product that provides the TCO improvement promised, and doe it with a clean and elegant solution.

Finally, a little on the future. With the cost of Flash memory coming down 50% year over year, it will soon reach the same price point that we currently see 15K HDD’s at. Once that happens one has to wonder what role 15K HHDs will fill? If 15K HDDs are, indeed, squeezed out of existence by this reduction in the price of Flash memory, what purpose will 3 tiered automated storage tiering fill? Or, will the future simply be 2 tiers of storage, one that provides bulk capacity, and one that accelerates the performance of this bult capacity? if that predication is correct, then FAST VP will have a limited life, and FAST Cache and FlashCache will be the longer surviving technology.

© Joerg Hallbauer for Gestalt IT, 2011. | EMC FAST and NetApp FlashCache a Comparison
Read more posts categorized as All, Featured, Storage, Top Story

Wandering the EMC World show floor was very revealing. Along with EMC’s new VPLEX, Chris Mellor and I spotted the unannounced SDRAM-based ZeusRAM SSD from STEC. While getting some hands-on time with Iomega’s new 12-drive storage array, I spotted an exciting but unannounced feature: The ix12-300r includes native Avamar backup client!

Native Avamar support in the ix12-300r? That’s new!

Chocolate and Peanut Butter

EMC’s Avamar backup product deserves much more press than it gets. Global deduplication is the secret sauce, allowing high efficiency for backups, both in terms of capacity and bandwidth usage. This means that Avamar is very well-suited for small business and remote office use.

Iomega’s new ix12 is similarly well-suited in this environment. It includes flexible storage allocation, both iSCSI SAN and NAS, and is bursting with features, from replication to webcam support. The 12-drive array fits perfectly into the small-business and remote-office markets with a low $5k-$10k price tag.

Avamar’s backup deduplication technology and Iomega’s “just right” small-business storage is a real “chocolate-and-peanut butter” combination. Both target the same markets and mixing them together is even better than selling them apart. Integrated Avamar client support makes the ix12 even more compelling, and will likely drive Avamar sales as well.

I talked to the Iomega execs at EMC World and they confirmed that this support will be included in the shipping ix12 software but was not yet supported by the company. They expect Avamar qualification to come later in the year, but I suspect that customer interest could accelerate this timetable. If you are interested in the combination, let Iomega and Avamar know!

ix12 Impressions

My release-day review of the ix12-300r was based on press materials and a briefing, but getting a hands-on look at the device revealed details beyond the Avamar support. I was quite pleased by the build quality of the array: It is solidly constructed and the connectors, fans, and power supplies looked to be of high quality. The designers seem to have paid attention to cooling and airflow, important in a box containing 12 spinning hard disk drives. The dual power supplies are hot-swappable and seemed fairly robust as well.

I also spotted two empty PCI Express slots at the rear of the ix12. One wonders what use Iomega will put these to. One can imagine adding additional gigabit or 10 gigabit Ethernet controllers, but solid-state storage (SSD) would be much more interesting. The ix12’s LifeLink software has been enhanced fore more-flexible RAID configuration; I wonder if it also includes the ability to tier storage to SSD. A mini array with EMC’s FAST technology would be compelling, but the use of SSD as a cache (FAST Cache?) might be simpler to implement.

Clearly, the ix12-300r is much more than a SATA-expanded ix4-200r. Iomega is adding real valuable business features to the system and moving it ever closer to the EMC and Dell CLARiiON products. Yet the price is still well below those systems. Chocolate and peanut butter, anyone?

© Stephen Foskett for Gestalt IT, 2010. | EMC Mixes Avamar Into Iomega ix12-300r
Read more posts categorized as Storage

In order for FAST to work, it needs to gather and react to a lot of information from the array itself. In fact for FAST to be truly useful, it needs to gather, react and store alot of information about what is going on the array.

Take a typical corporate accounting application; most of the time it can be pretty quiet and non-performance intensive but at certain times of the year, it will be a very intensive workload. During these times, you might want it all to be on the fastest, most performant tier; now FAST will react to a sudden increase in workload and move the application when it sees the demand increase but will FAST be able to move this quickly enough? So perhaps, we need to give the array some hints as to when to prime the load?

These sort of peaks are very predictable and we know when they will happen but not all peaks are quite as predictable; or at least we don’t think they are. FAST will be gathering stats all the time and by analysing this data; it might be able to do the predictive analysis a lot quicker and spot things that we can’t or at least don’t have the time for. It may pick up on relationships between applications, application X runs hot at a certain time which causes application Y to become busy at some period later; for example, certain types of activity may cause a reporting job to be run at a later date.

You see from our storage infrastructure, we can start to gather a lot of information about our whole estate. But EMC could go further, they have things like nLayers and Smarts to leverage; they could start to pull information from VMware and do a whole lot of analysis on this. NetApp have SanScreen; HP have a zillion tools as do IBM.

Once you’ve got that information, you need to start turning that into something the business understands so that you can sit with the business and do what-if modelling, show conflicts and clashes where multiple services are demanding the same high-performance infrastructure at the same time. Perhaps the business owner needs to prioritise or purchase more infrastructure. Perhaps they need less, perhaps they can shift some stuff into the Public Cloud and just pull it back when they need too.

So FAST could be rather more than just a way optimising your storage infrastructure; if you data-mine this in the same way Google data-mine statistics, you can find out a lot of stuff which you didn’t realise and probably completely change the way you look at your infrastructure.

So when EMC talk about FAST being a foundational technology, they aren’t wrong…actually, like Virtual Provisioning, it is so important….it should be Free! Actually they could fund this by getting rid of half their account managers; FAST could literally sell itself.

© Martin for Gestalt IT, 2009. | Google for the Infrastructure
Read more posts categorized as Storage

FAST (FULLY AUTOMATED STORAGE TIERING)

FAST made a debut in the storage market yesterday (12/08/09). Finally after the market buzz we got a preview of the product in terms of its features, functionality, characteristics, possible shortcomings and use cases.

This blog post focuses on the features, the drawbacks and some applications around FAST.  By no means is this a comprehensive or an exhaustive list of the above.

After all, FAST makes a debut, (The previous post on FAST).

..

NOTE: Out of the box thinking by EMC, imagine the flexibility one would have with a large infrastructure and FAST moving data based on policy. Its all about the big picture.

..

..

Here are some features, highlights of FAST and how it operates.

• FAST will perform data movement based on IOPS, average I/O size and write percentage. This is currently true for the Symmetrix V-Max, Clariion CX4 and Celerra NS.

• Three elements that define FAST: Storage Type, FAST policies and Storage Groups.

• FAST is based on user defined – configuration policies.

• The configuration of FAST is typically done through FAST wizards (Symmetrix Management Console) on Symmetrix V-Max, FAST LUN Migrator for Clariion CX4 and Rainfinity File Management Appliance or VE for Celerra NS.

....

..

..

• User defined analysis period for FAST. That will enable FAST to recommend or perform data analysis and then a data move based on policy.

• FAST created policies will associate with Storage Groups.

• FAST policies will be configurable at a LUN / drive / drive type / speed etc level.

• Data movement will take place based on a time of the date policy called “COLD” move or on usage policy called “HOT” move.

• Data movement will take place between various drive types, various LUN types and LUN sizes. (LUN types, LUN sizes will need to be same for the source and destination LUNs). For example a 9GB FBA LUN being migrated from Fibre to FLASH will need similar source and destination LUN properties.

• FAST data analysis will be performed in the background.

• For Symmetrix V-Max platform FAST will perform all analysis without the use of Symmetrix Performance Analyzer. Understanding is there will be some sort of API plugin available on the Service Processor of the V-Max that will enable Symmetrix Management Console (FAST plugin) to interface with the Symmetrix through the SYMAPI interface.

• For Clariions, the performance data of the array will be monitored and collected by Navisphere Analyzer.

• Based on the source LUN analysis, FAST will recommend the user to move the data either a faster speed drives or a slower speed drives.

• FAST will enable roll back based on user preferences (automated).

• FAST will be configurable by Symmetrix Management console wizards or SYMCLI

• FAST configurable by Clariion Naviseccli and installed on a host connected to Clariion, Host software called FAST LUN Migrator.

• FAST configurable by Rainfinity File Management Appliance GUI or CLI for Celerra NS “out-of-box data” movement. This enables the Celerra to move data to another Celerra or Centera or Atmos.

• FAST configurable by Rainfinity File Management / VE (Virtual Appliance) for Celerra NS “in-the-box” data movement.

• FAST can be installed non-disruptively on all platforms.

• FAST will operate both at an FBA and CKD level supporting open systems and mainframes for V-Max.

• FAST will operate at a LUN level on Symmetrix V-Max

• FAST will operate at a LUN level on Clariion CX4

• FAST will operate at a file level on Celerra NS

• FAST v1 users will be able to purchase a FAST v2 upgrade when it’s released in second half of 2010.

• FAST can be purchased as a FAST suite or part of an ATSM (Advanced Tiering Storage Management) suite with bundled discounting prices.

• For a unified storage system like a front end NAS (Celerra) with a backend SAN (Clariion), FAST can coexists at both levels. But it is not recommended to deploy FAST at a Celerra LUN level.

• FAST integrates with Symmetrix Management Console and with Rainfinity GUI (Celerra) for simple management. Though Clariion implementation will need one to specialize in CLI.

.

FAST introduction by EMC

..

..

FAST Drawbacks

Here are some drawbacks of FAST as I see it today.

• Does not support Virtual provisioning. So the Virtual provisioned LUNs will not be FAST enabled. Sub-LUN expected in second half of 2010

• FAST is not free and is charged based on RAW CAPACITY of the Storage Array.

• FAST will only work with similar LUN types (example FBA LUNs can be migrated to FBA) and LUN sizes (9GB LUN can be migrated to a 9GB LUN) only.

• FAST will require Professional Services and is not recommended for customer self implementation at least for the Clariion and the Celerra platforms. EMC is making a claim that FAST can be self provisioned on the Symmetrix V-Max platform.

• FAST works only on current generation systems like EMC V-Max Enginuity 5874, Clariion CX4 Release 29 and Celerra NS.

• With Clariion CX4 and FAST implementation, requirement is to have Navisphere Analyzer.

• With Celerra NS and FAST implementation, requirement is to have Rainfinity File Management /VE or Appliance.

• With Symmetrix V-Max FAST implementation, requirement is to have Symmetrix Management Console (Not free any more starting with the V-Max)

• For the Clariions, FAST only analyzes Fibre drives and LUN movement has to initiate from Fibre channel to FLASH or Fibre channel to SATA drives only. Movement from FLASH to Fibre or SATA to Fibre has to be initiated manually.

• A customer implementing Virtual Provisioning on Clariion CX4 and Symmetrix V-Max that is using 50% Thick Provisioning and 50% Virtual Provisioning. FAST will not work with Virtual Provisioned LUNs. But the customer will probably pay for 100% of Storage Raw Capacity when it comes to paying for FAST either non-bundled or bundled in an ATSM (Advanced Tiering Storage Management).

• FAST does not support IBM System I platform currently.

..

..

Some real world FAST applications

• Acadia Service Model can now add granularity with offerings around on-demand resources using FAST at its core.

• On Demand application and workload needs can be met for OLTP, Data Warehousing, Mainframe compute and Virtual compute using FAST driven policies.

• Multi-tenancy with Private Clouds

• Move high demand data on faster drives while rarely used data goes on slower drives.

• Sell ITaaS based on SLA’s. Higher SLA’s can mean higher price. All automated processes controlled by policy.

• Higher transactions typically mean low overall cost; mean higher efficiency means higher profits all achievable through FAST policy engine.

Need to see some real world implementations of FAST now. In theory and on paper, FAST looks pretty compelling but practically will it do the magic.

..

..

Some unanswered questions today

• How does it prevent from LUNs jumping between Fiber, FLASH and SATA if the application has un-expected performance that day?

• Manageability of FAST interfaces.

• Usability of FAST interfaces.

• Granular configuration policies associated with analysis.

• Future upgrades.

• Current implementations and how long and how effective are those.

• Can you improve performance without using SSD’s. Example with Fibre and SATA drives only.

• With future upgrades from FAST v1 to FAST v2 what happens with current user defined policies.

• Would Symcli scripts change with FAST implementation?

• During lockout periods (Where Symmetrix configuration cannot change, eg before a BIN file change) would FAST still operate.

..

To read some of the answers to the questions, as commented by Barry Burke on the StorageNerve Blog

FAST is a bit of a new subject for me. Any experts please feel free to correct me if my understanding of FAST is incorrect at any level.

Stay tuned for a series of FAST posts over the next few days talking about various topics and how FAST plays within those areas.

© Devang for Gestalt IT, 2009. | FAST: Features, Drawbacks, Applications and some Questions
Read more posts categorized as Storage

I spent some time today looking at the release notes for Enginuity code 5874.207.166, which presumably is the one that brings the much lauded Fully Automated Storage Tiering (FAST) into general release on V-Max.  Just above the FAST paragraph I found the following:

Symmetrix Virtual Provisioning Space Reclamation reduces capacity requirements and total cost of ownership by automatically reclaiming chunks (768 KB track groups) that contain all zeros. This is most effective when used on volumes after thick-to-thin migration or replication.

So, it seems that V-Max now supports features previously only seen on 3Par InServ, HDS USP V and HP XP – that is the ability to reclaim empty “zeros” of data from LUNs – otherwise known as Zero Block Reclaim.

I don’t remember EMC mentioning this little fact as part of their big FAST announcement.  In fact, looking back over Barry B’s posts, here’s a link to a post from July in which Barry indicates (quoting again);

I cannot confirm nor deny that VP will support one or more unused space reclamation approaches in the future.

So do EMC just see ongoing space reclamation as a BAU activity, not worthy of an announcement?  I’m surprised that this would be the case.  Reclamation of “empty” storage is incredibly important when migrating from thick->thin storage environments.  Hitachi quote around 40% savings from using ZPR after a migration to thin provisioning on USP V.

Perhaps EMC don’t want us to know that migrating to V-Max can actually reduce the amount of storage in use.  After all, its not good for hardware sales, is it?

By the way, EMC, please feel free to comment on this new feature and how easy it is to use.  I’d be interested to discover how it is implemented.

© Chris for Gestalt IT, 2009. | Enterprise Computing: Has EMC Slipped Zero Block Reclaim Into V-Max?
Read more posts categorized as Server Virtualization, Storage

So EMC has proved critics like me wrong and have introduced EMC FAST (Fully Automated Storage Tiering) as an offering in Dec 2009. There were many skeptics like me that saw this product release being stalled because of various reasons, here.

Truly this was one of the most awaited products of 2009 from EMC after the initial announcement by EMC back in April 2009 along with the release of Symmetrix V-Max.

I have dedicated a separate deep-dive section for EMC FAST on the StorageNerve Blog, here

Along with FAST, EMC has also introduced some new enhancements to the EMC Symmetrix V-Max, Clariion CX4 and Celerra NS platforms. Currently FAST will be available on the above 3 platforms at debut and will provide automated storage tiering “in-the-box” for Symmetrix V-Max, “in-the-box” for Clariion CX4 and “out-of-box” for Celerra NS platforms.

As expected EMC has not made this feature free but rather offers a pricing model based on bundled software. FAST will be an available feature within ATSM: Advanced Tiering Storage Management bundle and will be charged based on RAW Capacity of the Array (Symmetrix, Clariion and Celerra)

FAST will not be supported on EMC DMX-4 and Clariion CX3. In essence FAST is only compatible with EMC Symmetrix V-Max Enginuity Microcode 5874.xxx.xxx and Clariion CX4 Flarecode Release 29.

FAST is a software only feature and integrates with existing hardware / software on the associated platforms.

The following are some of the features FAST will support at GA on the EMC Symmetrix V-Max, Clariion CX4 and Celerra NS platforms.

EMC Symmetrix V-Max

• Volume / LUN based data movement (Automated Storage Tiering) for open systems and CKD – mainframe volumes.

• Management of FAST through Symmetrix Management Console or SymCLI

• Data Movement can be accomplished between FLASH, fibre channel and SATA drives within the V-Max platform. Data can move in any direction and on any type of the drives based on policy.

• Data movement within a single frame or serial number only.

• FAST suite can be purchased as a standalone software suite but will be available at a discounted price based on a bundled option with Symmetrix Optimizer, DCP: Dynamic Cache Partitioning and SPC: Symmetrix Priority Controls.

• Symmetrix Performance Analyzer is not required for FAST to operate on the Symmetrix V-Max platform.
  

EMC Clariion CX4

• LUN based data movement (Automated Storage Tiering)

• Management of FAST will be enabled through CLI only. Not supported as an integrated part of Navisphere today.

• Will work with Flarecode Release 29 (Clariion CX4) only.

• Unlike the Symmetrix, the automated data movement will only be supported between fibre channel drives to FLASH or fibre channel drives to SATA. There is no automation related to data movement from FLASH to fibre channel or FLASH to SATA or SATA to FLASH or SATA to fibre Channel drives.

• Data movement within a single frame or serial number only.

• FAST suite can be purchased as a standalone software suite but will be available at a discounted price based on bundled option with Navisphere Analyzer and NQSM: Navisphere Quality Service Manager.

• Navisphere Analyzer is required for FAST to operate.

EMC Celerra platform

• File based data movement (Automated Storage Tiering)

• Management of FAST can be accomplished through Rainfinity file management appliance GUI or CLI. Customers also have an option to purchase Rainfinity File management/VE (Virtual Appliance), which has some limitations.

• Data movement can be enabled to another tier “in-the-box” or to another Celerra or Centera or Atmos.

• Celerra FAST is most interesting as it enables out the system data movement, like to another Celerra or Centera or Atmos.

• FAST suite can be purchased as a standalone software suite but will be available at a discounted price based on a bundled option with Rainfinity File Management Appliance or Rainfinity File Management /VE.

Based on EMC heat index charts, a before and after picture of a FAST implementation would look like this.

A FAST implementation video

Here is a post, back from August 2009 on Gestalt IT describing how EMC’s Unified Storage vision and federation may work. It is good to see, some of those things come to fruition now, and at least FAST with Celerra proves it.

Stay tuned for a series of FAST posts over the next few days talking about various other topics and how FAST plays within those areas.

© Devang for Gestalt IT, 2009. | After all, FAST makes a debut
Read more posts categorized as Storage, Tech

As you might have read in my earlier blog post, EMC has announced the release of the first version of their product called “Fully Automated Storage Tiering” or in short “FAST”.

Now, to describe the purpose of this technology in a very simple form, we are talking about the performance of your storage and some logic that will help you put those things that need performance on the fastest bits available in your storage environment.

And that’s about as far as we can go with the concept of simple. Why? Because if this technology is to add value, you need it to be really clever. You would almost need it to be a bit of a mind reader if you will. You will want it to know what your application is going to do, and you will want to know where it does that on the most granular level of your storage, namely the blocks on the disks. Or more simply, you don’t want it to react, you want it to behave proactively.

So, let’s start with some mixed news:

As some of you will notice these three platforms have something in common. EMC tried to get rid of using custom ASICs in favor of using commodity x86 based hardware for as much as they could. In the new V-Max you will only find a custom ASICs that resides on the Virtual Matrix Interface controller, and is responsible for the coordination of local and remote memory access.

This swap to x86/x64 and a 64 bit architecture was done on all three mentioned platforms. On its own this is a good thing, but it would also be a good explanation why EMC as of now is not supporting older arrays. EMC is bound to get requests for this new technology for their older arrays like the CX3 or the DMX4. There are two likely options there:

1: It’s not going to happen.

Porting the code to a different hardware platform is a pain. The logic behind it is still the same, but the question is, up to where would you backport it? DMX3? DMX2? Where would you draw the line? Combine that with the fact that not all the newer features are available on the older machines and you can probably imagine that it would be easier to just not make these features available on older arrays.

2: They are almost done and will release it sooner than anyone thought.

EMC has a lot of developers. Chances are they were also working on FAST for the other platforms and will be releasing it in the not too far future.

Since we will be seeing arrays being removed from the product purchase portfolio, my money is on option number one. You won’t have the option of buying a DMX3 within the next half-year. And you can also replace half a year with 1.5 year for the DMX4. Sure, you can get extended support which will add four or five years to the life cycle of your array, but implementing new features for arrays which will not be sold anymore in the near future? I find that sort of unlikely.

As explained before, normally your application won’t be updating the data on the entire LUN. Usually you have a few so-called “hot zones” which are just blocks of data are being accessed by reads and/or writes more frequently. An excellent graphical example of this fact is something called a “heat map”. This heat map is created by an (unfortunately) internal EMC application called SymmMerge but fortunately fellow blogger Barry Burke, a.k.a. “The storage anarchist” allowed me to use some images from his blog.

So, this would be the situation in a fairly common environment:

Note that in this image we are talking about actual disks, but the image will also work if we just simply replace the word “drives” with “blocks”. The green blocks are doing fine, but the red and orange blocks are the ones that are being accessed a lot.

The ideal solution would normally be to put the red and orange blocks on a faster medium. EMC would normally tell you that the ideal medium for these kind of blocks would be EFDs or “Enterprise Flash Drives”. And you could put the green blocks on a medium that might not need quite as much performance or the same response times as regular fiber channel drives or perhaps even cheaper SATA drives for bulk storage. Each class of drive (EFD, FCD, SATA) is called a tier, hence the term “Tiering”.

After a redistribution your image would look something like this, where all blocks would be on a storage class that suits their individual performance needs:

Now, probably one of the biggest pain points for a lot of people is that this version of FAST is not capable of doing this on a block level. Version 1 is only capable of moving data on to a different tier on a LUN level. But your database/CRM/BW/etc. normally does not read and/or write to the entire LUN.

So with this version of FAST you actually put a lot more data on a faster tier than you would actually need to. On the other hand EMC stated that the key value for FAST version 1 is not so much in the fact that you move your LUNs to different tiers, but in the fact that you can set user policies to have the system do this for you. It takes some of the effort involved and handles things for you.

Now, you can create up to 256 different tiers which in its current version allow you to define tiers based on RAID levels, the speed of a drive and the drive type. It should be noted that the tier definitions will differ when using dynamic or static tiering. Currently disk size and rotational speed are not considered when you create a dynamic tier, so a dynamic tier may contain disks of differing performance characteristics, but a tweet from Barry Burke stated that FAST is actually aware of the RPMs, and knows the latency impacts of contention and utilization. Or at least “it will be in the future“

Now, you can create a policy for a storage group, which is basically a group of disks that are managed as a set, and have that policy associate a storage group with up to three tiers, depending on the tiers you actually have in place. Now, combine that with setting limits for the percentage of capacity that is on a single tier and you will see that you could for example say that you want 80% of you capacity to reside on SATA disks and 20% on EFDs.

Fast will now apply your policy and, depending on the choice you made, automatically move the data around across those tiers or give you a recommendation on what would be a sensible choice. It can even relocate you data to a different RAID type on the other tier, and your SymmDev ID, your WWN, your SCSI ID and all external LUN references will remain unchanged during the move. If you have replication set-up, that stays active as well.

Now since this all stuff that might have a performance impact if done during peak loads on your box, the default is that all the moves are performed as lowest priority jobs during time slots or windows that you as the end-user can define. Just keep in mind that you are limited to 32 concurrent moves and to a maximum of 200 moves per day.

Prices start at $5,000 USD for the entry-level systems, and will set you back $22,000 USD for the Symmetrix V-Max. But that is the starting price, and the unbundled price. You could also consider a bundle called the “Symmetrix FAST Suite” that includes the optimizer and priority control & cache partitioning. All to be delivered as a “firmware upgrade” for your array.

Well, I’ve got mixed feelings on that point. I can see how this first version can add some value to your environment, but that will depend on your environment. People who only use one tier might not have as much value, and adding the cost of new disks in to the equation will not make it any easier. Especially when we take the release of FAST v2 into account that is “planned for GA in 2nd Half 2010” and will also provide support for thinly or virtual provisioned LUNs and be able to move stuff around at the block level.

I know there is value in this release for some customers that are actually using the V-Max. The automated tiering can at least help you meet a certain service level, but that added value is highly dependent on your environment. Personally, I’d probably wait for the release of version 2 if possible. On the other hand, EMC needs to gain traction first and they were always open about the fact that they would release two versions of FAST, and stated that version 1 would not have all the features they wanted, and that the rest of the features were planned for version 2. I have somewhat of a hard time with some of the analysts who are now complaining that FAST v1 is actually that what EMC said it would be. Did they just ignore previous statements?

It’s the same story as usual. Every storage vendor seems to agree on the fact that automated storage tiering is a good thing for their customers. Some have different opinions whether or not the array should be the key in this automation, because you are at risk of the array making a wrong decision.

EMC started off their journey with some steps towards automated tiering, but they delivered just that, the first steps toward an automated tiering vision. If we would remove the argument of a price tag, I would be almost positive I’d recommend version 2 too any possible customers. For version 1, the answer is not that simple. You need to check your environment and see if this feature makes sense for you, or adds value to your setup.

Besides FAST we’ve also seen some new cool features being introduced with the new “firmwares” that were released for the various arrays, such as thin zero reclaim and dedupe enhancements. Look for coming posts that will go in too more detail on the new Flare 29 and Enginuity 5874.

© Bas for Gestalt IT, 2009. | EMC’s FAST, take 1. Action!
Read more posts categorized as Server Virtualization, Storage

Yesterday, EMC announced Fully Automated Storage Tiering (FAST), their much hyped and much anticipated storage feature enabling the automated moving of data between tiers of storage on a policy basis.  However the most notable missing feature in the EMC announcement was the lack of support for legacy DMX-3 and DMX-4 platforms.  This to me sends a message loud and clear that despite continuing to sell it, the DMX3/4 legacy monolithic hardware is dead.  If that’s the case, why bother buying from EMC any more?

Discounting EMC in the storage array market may seem like a naive and perhaps foolish comment to make.  After all, recent IDC numbers show EMC top of the pile at nearly a quarter of all external storage arrays sold, depending on which figure you choose to use.  However, take a moment to look at the EMC briefing pages on FAST (you can find them here).  There you will see Intel co-branded with EMC, highlighting many previous messages that monolithic architectures are dead and commodity modular boxes are the way of the future.  We’ve seen that this year already with the release of Atmos.

To my knowledge, FAST is the first “innovation” of the new V-Max product line, but it isn’t unique.  In fact, I don’t think any features of V-Max are unique; the architecture is found in many other products.  There’s a whole raft of mid-range storage arrays from IBM (XIV), 3Par, Compellent, Pillar, Dell/Equallogic and HP (Lefthand) with the last two being acquisitions of successful companies.  I expect in the next 12 months we’ll see enterprise modular releases from Hitachi/HP and a revamped EVA.  Most of the products mentioned here have been designed from scratch to remove the legacy encumberances of the past that products such as V-Max still retain.

So what’s my point?  Well, simply this; EMC have legitimised the enterprise modular architecture characterised by V-Max.  This accepts that the future is commodity-based hardware with differentiation in software.  However, EMC are no longer the leaders in this field and are having to play catch up.

 There’s never been a better time to look wider than the Big 4 (EMC/Hitachi/HP/IBM) and see if the features you need can be found elsewhere.

© Chris for Gestalt IT, 2009. | Enterprise Computing: Is There Any Point Buying From EMC?
Read more posts categorized as Server Virtualization, Storage

Though there are two distinct groups of people, one favoring automation and the other half possibly thinking this technology brings no value-add in terms of how storage is utilized and managed today. This game was initially started by Compellent (Compellent Data Progression technology) about 4 years ago, then joined by Pillar Data Systems and now other OEM’s (including EMC, HDS, IBM) are starting to catchup on policy based automated storage tiering.

With private clouds in the near future and then hybrid clouds (a mesh of private and public clouds) in the horizon, automation, workload distribution, SLA’s, QoS will need to be monitored and managed to optimally run IT Infrastructures. Policy based management will create a new wave of storage management, automation and will act as a principle ingredient of hybrid clouds.

Generation 1 of policy based storage tiering works within a single storage subsystem.
Generation 2 in the near future should work across heterogeneous storage subsystems (by the same manufacturers).
Generation 3 over the next year or two will work across storage platforms irrelevant of the manufacturers.
Generation 3 of policy based management will include the entire stack of management. These products will be capable of not only managing the Storage, but also interact through policies at the Virtualization, Networking, Application, OS, Middleware and other layers in the stack of Infrastructure management..

We should see an up-rise of new emerging technologies that will create these external policy based engines for data movement automation. All infrastructure components including Storage, Virtualization, Networking, Application, OS, Middleware will provide the necessary API’s for these external engines to interact and enable data automation and workflow automation in hybrid clouds (irrelevant of the manufacturers).

Here are a few articles from the past month related to the topics of Policy, Automated Storage Tiering, Workloads, SLA’s and QoS.

Pillar

http://blog.pillardata.com/pillar_data_blog/2009/10/autotiering-of-data.html

EMC

http://flickerdown.com/2009/09/why-policy-is-the-future-of-storage/

http://flickerdown.com/2009/10/why-policy-is-the-future-of-storage-part-2/

http://stevetodd.typepad.com/my_weblog/2009/10/greenfield-monitoring-of-a-private-cloud.html

http://stevetodd.typepad.com/my_weblog/2009/09/federation-and-private-cloud.html

Compellent (Partner Blog)

http://blogs.cinetica.it/cinetica/2009/10/19/dear-mike/

http://blogs.cinetica.it/cinetica/2009/08/25/tiered-storage-and-new-features-for-the-rest-of-us/

Ocarina Networks

http://onlinestorageoptimization.com/index.php/tiered-storage/

HDS

http://blogs.hds.com/hu/2009/09/ilm-revisited-intelligent-tiered-storage-for-file-and-content-data.html

Independents

http://www.storagemonkeys.com/index.php?option=com_myblog&show=the-end-of-history-or-just-the-beginning-.html&Itemid=136

http://thestoragearchitect.com/2009/10/18/enterprise-computing-do-we-need-fast-v1-emc/

http://www.theregister.co.uk/2009/09/22/emc_fast/

http://itknowledgeexchange.techtarget.com/storage-soup/hp-drops-roadmap-nuggets-at-storageworks-techday/

http://itknowledgeexchange.techtarget.com/storage-soup/spinnaker-founders-bring-avere-out-of-stealth/

http://breathingdata.com/2009/10/18/can-and-when-will-ssds-sata-replace-fcsas/

http://gestaltit.com/featured/top/gestalt/emc-unified-platform-storage-tiering/

http://storagenerve.com/2009/10/14/enhancements-to-emc-symmetrix-v-max-systems/

Some other posts include

http://www.compellentblog.com/Posts/2008/9/Patent-Sense-Automated-Tiered-Storage-and-Continuous-Snapshots.aspx

http://www.compellentblog.com/Tags/automated-tiered-storage.aspx?page=2

http://blogs.computerworld.com/compellent_ilm

Your thoughts always welcome!!!

cheers
@storagenerve

© Devang for Gestalt IT, 2009. | Policy! Policy!! Policy!!!
Read more posts categorized as Featured, Storage

So, here’s my rash statement from Twitter last night: “If FAST isn’t free, I don’t want it!  All it’s doing is automating process I could script/do manually”.  It’s a bold statement, I know, so is FAST really offering something better than what could be achieved today using EMC’s Symmetrix Optimizer?

Hot Spots

EMC’s Symmetrix architecture (18 years old and counting, I believe) uses the concept of disk hypers to present LUNs.  Each physical disk is carved into a number of slices, which are then recombined to create LUNs to present to a host.  A mirrored (RAID-1) LUN uses two hypers, a RAID-5 (3+1) LUN uses 4.  EMC ensure general performance by setting standards on how LUNs are created from hypers and that’s reflected in a “binfile” layout.  However despite this sensible planning, it is possible (especially as hard drives are now much larger and contain many more hypers) that two hypers on a single physical disk could be highly active and so contend against each other – in other words “hot spots” on disk.

Optimizer helps alleviate the issue of hot spots by exchanging the high I/O hypers with low I/O ones, distributing busy LUNs across more physical spindles.  This is classic load balancing where resources are distributed across the available infrastructure in order to obtain better overall generic performance.  EMC have now rebranded Optimizer as part of Ionix for Storage Resource Managment, but it’s still effectively the same product.  Hyper swaps can be managed automatically, based on historical performance data.  They can also be user-defined – a manual swap at the users request.

Although tedious (and not as well automated as Hitachi’s HiCommand Tiered Storage Manager), in theory Optimizer could be used to manually move workload between storage tiers.  In fact, Optimizer is already aware of a tiered storage infrastructure.  Here’s a quote directly from the ControlCenter 6.1 manual:

“Optimizer is also aware of physical drives that operate at different speeds, as well as location of the data on the physical media, which influences the I/O rate. This information is used when determining which logical devices to move.”

So with a little bit of knowledge on the layout of data on a Symmetrix array, it would be possible today to use Optimizer to perform LUN-based FAST.

Load-Balancing Versus Policy

Unfortunately, simple load-balancing of I/O across a storage array doesn’t offer what should be seen as the next generation of storage tiering.  Where Storage Tiering 1.0 was about offering multiple layers of storage within the same physical infrastructure and manually placing or moving LUNs to the appropriate tier, Storage Tiering 2.0 will be about establishing policies that determine more service-based measurements of the performance and availability customers receive.

A policy-based approach would allow rules to be established on how data at the application layer moves between tiers.  This is a critical distinction from the load-balancing  methodology earlier described.  As an example, where an application was known to require higher performance at a certain time of day or day of the week, data could be moved proactively to a faster tier of storage, returning later once the high I/O workload had completed.  Whilst achievable using Optimizer, there’s no doubt the process of application migration would be tedious and time consuming.  I expect the v1.0 implementation of FAST will simply package up Optimizer into a tool that automates the migration of related data between tiers.  Don’t forget, other vendors have been offering this feature for some time – for example Hitachi and Tiered Storage Manager.

Increasing Granularity

Now LUN-based migration has its benefits.  Where large numbers of disks exist in an infrastructure, application data can be placed or moved to the most appropriate location as required.  However with the introduction of solid state disks (SSDs), a more granular approach is needed as the number of SSDs deployed in an array is likely to be low due to their excessive cost.  Moving an entire application (or even LUN) to SSD will be undesirable unless that application can take full use of the SSD hardware.  There are very few, if any, applications that require high-intensity read/write activity from every piece of application data all the time.

Block-level tiering offers a higher level of granularity to the placement of data.  A LUN can be split into blocks and placed across multiple layers of storage technology including traditional HDDs and faster SSDs.  Selective placement will ensure the more efficient use of expensive SSD media by placing only the highly active data onto it.

All of a sudden with increased granularity we’re back to Storage Tiering 1.0 where data is being placed on faster technology purely based on increasing overall system performance.  This is a feature Compellent have been offering for some time.  Data is migrated up or down the tier hierarchy on a daily basis, subject to performance figures over a 12-day period.  This level of granular performance management is possible because data is stored in a block-based structure.  Unfortunately for EMC, the hyper design legacy represents a technical challenge in making FAST version 2 a reality.

Patent Rights

As just mentioned, Compellent already offer block-based data migration in their products.  At a recent dinner in London with the Compellent team, they highlighted their strong position in the market, protected by patents covering block-level data migration between tiers.  You can find the filed patent here.  Compellent use the term “Data Progression” to describe how blocks are moved between tiers based on I/O activity.  As I/O activity is monitored over time, it is possible to determine the most appropriate tier of storage to use when expanding capacity.  Typically these are lower tier SATA drives, as initial performance requirements are usually over-estimated.  This metholodogy is very much Storage Tiering 1.0 discussed earlier.

Compellent aren’t the only people claiming rights to block-level tiering within a storage array.  I’ve also found the following patent application from IBM, filed by Barry Whyte, Steve Legg and others.  If IBM and Compellent both claim to have invented the FAST concept, how does that position EMC?  Do they have an earlier patent which trumps these two?

Summary

Storage Tiering 1.0 provides performance management of storage arrays.  Storage Tiering 2.0 extends this to offer policy-driven optimisation offerings.  Both of these technologies are available today from existing vendors in one format or another.  EMC will simply be playing catchup with these vendors once FAST 1 & FAST 2 are released.  I’d like to be surprised and see EMC offer something the competition currently don’t.  I’m not holding my breath…

© Chris for Gestalt IT, 2009. | Do We Need FAST v1, EMC?
Read more posts categorized as Featured, Storage