The importance of data is something I discuss daily with many enterprises. I ask how and why they are collecting more? What are their concerns over management and security? And, how they are looking at ways to do more with it?
Doing more with data covers many bases, but ultimately, we want to use it to provide information to help make better decisions more quickly and more accurately. Long gone are the days where an enterprise lets its sales report run overnight to be acted on the following day. Today, we demand answers to questions in real-time so we can make rapid, accurate, data-driven decisions addressing the most pressing business questions.
Supporting Data at Scale
However, we cannot just turn on the fast data tap. We need systems and infrastructure that support our thirst for more data and more information more quickly than ever. None of this is impossible for the modern enterprise. The rapid adoption of SSDs (Flash Drives) has revolutionized the way organizations expect their systems to perform.
A few years ago, I worked alongside a smart energy company and introduced them to the power of SSDs. It started with a small cache card, which lead to a small POC of an all-flash array, to which they migrated a sales database. The performance increase was revolutionary to them. Reports that had taken hours were now completed in minutes. The business immediately ordered their first all-flash array.
They didn’t order it because of speed alone. Rapidly processing our data allows the modern business to do things that it previously couldn’t. It presents new opportunities, new markets, and new products, transforming what an enterprise can do.
Chasing the Bottleneck
Is it all good news then? Of course, it is not that straightforward. There is an IT truism that I learned in the early part of my career, “You are always chasing the bottleneck”. Whether it’s compute, memory, networking, or of course, storage, as we fight to increase the performance of one element of our IT platform, we invariably introduce a new one elsewhere. It’s like the cartoon character trying to stop a leak in a dam. First, it’s one hole, then holes everywhere, which they are filling with fingers and toes until they can’t do any more and it bursts.
While SSD has revolutionized the performance we expect from our enterprise systems, it is introducing new bottlenecks. The broad adoption of SSDs is now starting to put a huge strain on the CPU. Even the performance of our newest and fastest processors is getting outstripped by the rapid development of SSDs. This will only get worse with the development of NVMe and evermore performant flash.
The faster our storage, the more software and CPU is swamped by the performance demands it presents. Overcoming that bottleneck is driving an exponential increase in spending on both storage and compute for only linear performance growth.
The demand for more data processed more quickly is not going away and our budgets are not getting bigger. So how can we overcome these challenges?
Overcoming the Challenge with Pliops
The answer may not be far away as we see the storage industry starting to focus on finding innovative ways to accelerate the performance of flash while reducing the impact on the CPU. From public cloud providers to well-known storage vendors, they are all looking to solve this conundrum, but is this something that could be available and affordable everywhere?
In this brief series of posts, I look at a potential approach to solving it and how new vendors like Pliops are aiming to not only solve it, but to make it accessible for all!
If you want to unleash your SSD, then read on.
