I always thought increasing hard drive capacity was an unequivocally good thing. Maybe it’s just my consumer focused naïveté, but I never thought there could be much of a downside of constantly increasing drive size. More storage is just better, right?
Well not when you’re worried about failure. Enrico Signoretti lays out the issue with new 14TB drives coming out. It sounds really great to have 13PB in a single rack, but just image when something goes wrong. When a small cluster goes down, image the huge amount of time it’ll take to rebuilt data and metadata with that kind of scale. And the problem only gets worse the larger disks get. Luckily, it seems there are a lot of companies working on a solution.
Enrico highlights OpenIO’s solution for this problem. They’re putting an ARM-based hardware interposer on each drive, essentially making each drive into its own little server. They’re not the only company with this idea, Enrico alludes to Seagate’s Kinetic drives as an example. I recently saw a something very similar from Igneous Systems. Both Igneous and OpenIO bill the ability to reduce the fault domain down to a single drive, which while it would still be a heft 14TB, is a whole lot less than even a small node cluster.
If this idea can continue to gain traction, the real differentiator will be the software powering these boxes. Igneous differentiates with the concept of overall fleet management across their entire product range. OpenIO seems to be able to offer a little more flexibility with layout. Igneous goes in 212TB nodes, while OpenIO is able to provide storage in considerably smaller increments.
Overall, Enrico does a good job of highlighting the perils of increasingly dense storage. Luckily, there seem to be an emerging class of companies eager to take on this challenge.
Juku.it comments:
I came across two interesting news items this week – OpenIO introducing a 96-HDD appliance for its object storage platform and Western Digital launching 12 and 14TB disks! At first glance if you sum the two, it’s like crazy: just think about a single 96-slot appliance full of 14TB disks, which means 1.3PB in a […]
Read more at: About scale-out storage ARM-ification… and $/GB
