Welcome to the State of the Industry from Gestalt IT. This is a weekly series where our staff gets together, talks about what seems to be percolating within the enterprise IT space, and puts together a look at a few relevant stories. We consult with experts in the field to get their opinions, and bring it all you you in a concise post. Thanks for joining us!
State of the Industry: Container Storage
If hype was something that had mass, containers would affect the tides. It seems like everyone is talking about them. They’ve been pitched as a solution for just about every problem a data center could have, and probably have been blamed for a fair share of problems as well.
Lost in a sea of containers ][ Image Credit: David Merrett
The Problem?
It’s a pretty basic thing about containers that they’re inherently stateless. Or at least that data stored in a container is tied to the ephemeral nature of the container. Now, there are some ways to provide for storage, for containers, but what is provided natively is pretty basic. Specifically for Docker, Joep Piscaer broke it down for me. The issue comes down to the Docker image format, which leaves all layers but the very top as read-only, and that layer is volatile. Docker’s solution is their Docker Volumes, with accompanying drivers. These integrate with (generally Linux-based) storage solutions. These integrations, while they play nice with a ton of file systems are pretty limited.
This brings up the essential problem with container storage, it lacks intelligence and features. Everyone I spoke to on this story brought it up. These’s a huge feature disparity with what you can do with container storage and some of the Hyperconverged solutions. What some container storage solutions look (or claim) to provide is a way to intelligently connect persistent storage to constantly changing containers, dynamically and systematically.
The other thing they can provide are enterprise class data services. Deduplication and compression are not inherently provided in containers, but are vital components in the enterprise. At Tech Field Day last month, there wasn’t one storage product that didn’t either present a dedupe feature, or wasn’t questioned about it. The same thing goes for snapshotting and replication. Right now there isn’t something provided natively, so there’s a market opportunity to “fix” the problem. It’s a tough nut to crack.
Interestingly, while some of the problems of container storage are apparent, so far there hasn’t been a huge cry from the enterprise to fix them. This is even as companies flock to provide solutions. I spoke to James Green about this and he thinks a lot of these startups might run out of funding before their products really come into need. And all this while Docker works on it’s own solution to the problem, post-Infinit acquisition. Interestingly Joep thinks this may actually spur innovation in the space. My initial impression was that Docker buying a “winning” solution would edge out other competitors. Joep thinks this may actually spur innovation. Docker buying the space might validate it as a more legitimate category for investment and development. Even in Docker can tightly integrate Infinit’s distributed filesystem as a default, there may still be enough specific use cases for alternatives to keep a healthy ecosystem around it. Still, I don’t think this contradicts James’ point. For some of the companies focusing on Docker storage earlier, they may still run out of funding before either coming to market or running into a surfeit of competitors.
The Disparate Shapes of Pegs and Holes
When I began looking into this, I thought that some of the issues that we see cropping up with containers may have more to do with the people implementing them than the technology. I was discussing this with Tom Hollingsworth, and he mentioned seeing everything from firewalls to SQL being run in containers. Maybe some of the storage conundrums are simply square pegs being forced into round holes.
James Green thinks this misses the point a little bit. The way he sees it, it’s not so much that SQL or a firewall are “bad” for containers, or any other app. It comes down to whether these apps were developed with containers in mind. Generally this means you have to think of developing it with a distributed nature in mind. Admittedly, if they aren’t, it (should) be obvious fairly quickly. I think a lot of this comes down to the simple initial nature of getting containers up and running, especially with something like Docker. Even if it’s a bad idea, there isn’t all that much investment of time or resources to get something up and running. Indeed, even knowing something will break in a container, it might be interesting simply to see how something breaks when put under the peculiar instance of a container.
Containers get a lot of buzz because there are still things to figure out. While virtual machines still rule the roost, it’s a much more established solution. Not that VMs are “easy” compared to containers, but it’s a much less emergent space. But with the excitement about containers, there are limitations and frustrations. Storage and networking are the ones I hear the most about. Notably, Docker has now acquired companies to at least nominally address both. I think networking is both further along than storage, but also has more things overall to figure out. In some way, being inherently stateless gives storage an advantage. Sometimes it’s better to work with a relatively blank slate. There’s still nothing easy about true distributed storage. But the rising tide of containers in the enterprise should spur the market to meet that need.
