With Automated Tiering solutions now rapidly becoming de rigeur, well in announcement form anyway; perhaps it’s time for us to consider again what is going to go wrong and how much trouble the whole thing is going to cause some people.
Fully Automated Tiering solutions are going to cause catastrophic failures when they are deployed without understanding the potential pitfalls. I keep going on about this because I think that some people are sleep-walking into a false nirvana where they no longer need skilled people to manage their infrastructure estates; in fact, automated solutions means that you are going need even more skilled people; you will probably need less monkeys but you are going to need people who can think and learn.
Fully Automated Tiering solutions cannot be fully automated and be safe for most large businesses to run; they work on the premise that data gets aged out to slower and less performant disk, this reduces the amount of expensive fast disk that you require and can save you serious money. So why wouldn’t you do this, it’s obvious that you want to do this, isn’t it?
Actually in most cases, it probably is; unstructured file data almost certainly is safe to shift on this basis, actually most unstructured data could be stuck on the lowest tier of disk from day one and most people wouldn’t notice. In fact you could write most of it to /dev/null and not that many people would notice.
But structured data often has more complex life-cycle; accounting and billing systems for instance, the data can be very active initially eventually entering a period of dormancy when it basically goes to sleep. However, then something will happen to wake it up; periodic accounting runs, auditing and then the data will be awake and almost certainly required pretty sharpish. If your automated array has moved all the dormant data to a less performant tier; your periodic accounting jobs will suddenly take a lot longer than expect and potentially cripple your infrastructure. And that is just the predictable periodic jobs.
Automated Tiering solutions also encourage poor data management policies; actually, they don’t encourage but they certainly reinforce a lot of bad practise which is in place at the moment. How many people have applications which have data which will grow forever? Those applications which have been designed to acquire data but do not age it out? Well, with Automated Tiering solutions, growing your data forever no longer attracts the cost that it once did; well, certainly from a storage point of view and it might even make you look good; as the data continues to grow, the amount of active data as a percentage of the estate will fall and hence the amount of data which sits on the lower tiers increases and you could argue that you have really good storage management policies in place. And you do!
However, you have really poor Data Management policies in place and I think that one of the consequences of Automated Tiering, is the need for Data Management; your storage might well be self-managing, your data isn’t. If you do not address this and do not get a true understanding of your data, its life-cycle, its value; you are looking down the barrel of a gun. Eventually, you are going to face a problem which dwarfs the current storage-management problems.
What kind of problems?
- Applications that are impossible to upgrade because the time taken to upgrade data-in-place will be unacceptable.
- Applications that are impossible to recover in a timely fashion.
- Application data which is impossible to migrate.
- Data corruptions which down your application for weeks
And I’m sure there are a lot more. It’s hard enough to get the business to agree on retention periods for back-ups; you need to start addressing Data Management now with both your Business and your Application teams.
The good news is that as Storage Management becomes easier, you’ve got more time to think about it; the bad news is you should have been doing something about it already.
I think you are making a big deal about nothing. A company’s data and applications tend to grow alongside the expertise of their IT staff. A company with a novice storage manager almost certainly also has very modest storage needs in terms of both capacity and speed. Sure they may have structured data inappropriately setup on automatically tiering storage causing valuable data to migrate down to tier 3 right before a big report wakes it back up, but in the end the amount of data touched by that report is going to be small enough as to be insignificant. This seems considerably more intelligent to me than allowing a novice storage admin to purchase a new SAN without tiered storage technology and just choosing several trays of the fastest and most expensive disks they can get their hands on since they have no clue if the cheap stuff will work for them.
If you are a company large enough to have structured data that has real need for RAID 10 fibre channel disks in order to be useful, then chances are you also have the budget for a proper storage manager, and if you don’t then you’ll almost certainly be hiring one after that first end of quarter report falls on its face. Or, more likely, your novice storage manager will figure it out and fix it, thus gaining a bit more knowledge and growing as a storage manager. As long as you have a SAN that handles tiered storage appropriately it will be no big deal to learn from your mistakes and to force your important structured data LUNs back up to tier 1 permanently.
Really this is all a matter of scale. The environments that you are describing where tiering on structured data are going to have disastrous effects are also the environments where you have IT staff heavily pigeon holed in to departments and areas of expertise. You’ll have your DBAs demanding the best disk performance because DBAs always believe they deserve the best. You’ll have storage admins who spend all day in front of their SAN consoles and monitoring the performance of their various LUNs. These are not the sort of environments where you run in to “set it and forget it” storage policies and jack-of-all-trades IT guys managing all aspects of the environment.
As for ever expanding application data going unchecked… well.. What do you propose? I don’t understand your argument here. At least I’ve never encountered an environment where a business application’s data growth is slowed down because IT says it should, but I have certainly seen companies blow fortunes on buying more and more crazy expensive disks in order to continue providing storage to critical business apps because their SANs didn’t give them any choice.. Because at the end of the day management will almost always feel it is cheaper to buy more space at $4000 a TB than it is to try to remove business data and risk losing something of real value. So how can you fault a savvy IT department for trying to get the per TB cost down in order to manage these applications without breaking the bank. If you want to be helpful then you should be talking about the “4th tier” of storage: on-line archival to tape. Really, saying that inexpensive storage for growing application data is a ding against storage tiering is one of the most ridiculous arguments I have heard yet.
I’ve been managing a SAN with automatic tiering for 6 years now and have never had a problem. This is not complicated business. It just requires IT coming out of their offices and talking to production and executive staff in order to understand the value of the data they are storing and how it is utilized. Easily 99% of my storage happily tiers automatically without a problem. That remaining 1% I manage manually to ensure the best performance. It isn’t a big deal.