I’ve always been a hardware guy. There’s nothing I like more than digging into a new product or piece of silicon. There’s something about the physicality that I enjoy. Maybe it’s knowing the engineering required to produce it. Maybe it’s just my lizard brain responding to tactile response. Regardless, I know what I like.
By that introduction, you might think I wouldn’t be excited by Excelero. The company recently came out of stealth, and wants to provide the software layer to allow businesses to build a high performance scale out storage infrastructure. It’s a software only play, running on commodity hardware. But I’m here to tell you, Excelero is exciting!
Software defined storage isn’t a new idea. But it seems to be increasing gaining ground on custom hardware. You don’t have to look any farther than Dell EMC’s decision to kill DSSD to see that. Excelero instead focuses on commodify hardware, and not even particularly high end stuff, at least on the CPU side. But what can they offer? How about 4.5 M IO/Sec and 2.5 M IO/Sec random 4k reads and writes, respectively?
Seeing what Excelero is doing, I have to keep asking myself where’s the catch. Maybe they’re getting crazy IOPS, but that’s mitigated by high latency. Well, in the demo I saw, they were averaging <250 microseconds on reads, which is to say not much latency at all. Like I said, the hardware is also pretty standard off-the-shelf stuff, 24 NVMe drives and a SuperMicro Mobo with an Intel E5 Xeon. That will maybe set you back $13-16k, but the dollar per IOP is insanely low.
Ok, so the performance is great, but how are they doing it? Excelero has iSCSI talk to a special derivative of the RDMA protocol, what they’re calling RDDA, Remote Direct Drive Access. This allows them to read and write data directly to NVMe, bypassing the CPU. This has two benefits. One, it eliminates the classic noisy neighbor problem, which should mean it won’t impact performance of other applications. The other is that RDDA effectively does not impact the target CPU performance.
When I first heard Excelero talk about this, they said the effect was “virtually zero”. Now if you’ve heard enough marketing, you know that “virtually zero” generally means “hey, it’s not zero, it’s fairly low, but it sounds way better if we say it this way”. But with RDDA, it’s as virtual a zero as you’ll get. I was seeing about 0.3% CPU usage during writes, which meant that the application they were using to monitor CPU usage was stressing the processor more than their storage solution.
If there is a “gotcha” with Excelero, it’s data services. If I put it charitably, it’s that Excelero gives organizations the freedom to find the best data services that fit their particular needs, and don’t impose their own as part of their high performance scale out storage infrastructure. Or if I was critical, I would say that they essentially have none at launch. They’re definitely on the roadmap, but coming out of stealth, they wanted to focus their narrative around exceptional performance on commodity hardware. I think they made the right choice, but for now, it does limit potential customers.
Clearly I’m not the only one intrigued by Excelero. Watching their presentation from Storage Field Day, you can see the delegates appreciating the implications. Check out Ray Lucchesi’s and Chan Ekanayake’s articles
[…] Excelero’s NVMesh Magic […]