Much of what we do in Storage Management can be considered living on a prayer; this is not just a result of the parlous state of the storage management tools that we use but also due to the complex interactions which can happen within shared infrastructure.
And frighteningly enough, we are in the process of making the whole thing worse! The two darling storage technologies of the moment; thin provisioning and de-dupe scare me witless. Both of these technologies in the wrong hands have the capability to bring a server estate to its knees. By wrong hands, I mean just about anybody.
Both of them allow you to virtualise capacity and allocate storage which isn’t actually there and hope that you never need it.
Thin provisioning is predicated on the inefficient practises that we have come to know and love; we all know that when a user asks for storage, they nearly always ask for too much! Thin provisioning allows us to logically allocate the disk and only when it gets written to, does it actually get consumed.
The problem is, what happens in the event of a perfect storm and every application wants its capacity at the same time? How much do you over commit your physical capacity? Or maybe not a perfect storm, you just realise that you’re going to add physical capacity above and beyond that which is supported by the array simply to cater for rate the thinnly provisioned storage is being consumed. A rapid application migration ensues.
And then there is the scarey proposition of de-duped primary storage. You could be many times over-subscribed with de-duped storage; certainly in a virtualised server environment or a development environment where you have many copies of the same data. And then someone does something; a user decides to turn on encryption and what was many deduped copies of the same data actually becomes many full copies of the same data and you have run out of storage space in a spectacular fashion.
Also migrating deduped primary storage between arrays is going to be a lot of fun as well and is going to need a lot of planning. Deduping primary storage may well be one of the ultimate vendor lock-ins if we are not careful.
Both thin-provisioning and primary storage dedupe take a lot of control away from the storage team; this is not necessarily a bad thing but the storage team now need to understand a lot more about what their users are doing.
It will no longer be enough to just to think about supplying spinning rust which can deliver a certain amount of capacity and performance. We are going to have to understand what the users are doing day-to-day and we are going to have to communicate with each other.
And yes, we’ll need better tools which allow us to see what is going on in the environment but also to model the complex interactions and impacts of various events. We are going to need to know if a user is intending to do something like enabling encryption, a big data-refresh, operating system patching; events which in the past were not hugely significant could now have serious ramifications to our storage capacity.
I still think that thin-provisioning and de-dupe are good ideas but like all good ideas; they come with challenges and a certain amount of risk…
