I’ve been keen on rack-scale composable infrastructure for years. Decades even. But it’s only recently that we’ve had the technology to make it happen. You can now create a system or a rack that can flexibly allocate storage and compute using a shared I/O channel. But what if you could add more elements and “decompose” the server further? That’s what Liqid is promising with their latest announcements.
Liqid creates flexible compute systems that can access pooled resources like storage and compute but also other elements, including graphics and now storage-class memory!
PCIe Eats the World
I spent 2014 theorizing about rack-scale architecture with decomposed rack-scale storage. Since then, we’ve seen real innovation in terms of I/O channels that make that old vision seem attainable. PCIe has taken over storage (NVMe, M.2/U.2) and is the internal interconnect for most server components. Heck, it’s even a cabled interconnect thanks to Thunderbolt and OCuLink. And there have been PCIe switches for a few years. It’s really eating the world.
These days, the memory bus is the only important proprietary channel left in many servers, and that’s being challenged by “storage-class memory”. Although many SCM products use the memory bus, Intel has been pushing Optane Memory as an alternative. And guess where that “lives”: That’s right, PCIe!
At the same time, we’re seeing new important components emerge inside servers. It used to be that CPU, memory, storage, and networking were the biggies, with everything else relegated to second-class citizenship. But today’s servers are using GPUs for compute offload, and Intel is really pushing FPGAs as well. Some of these are integrated on the motherboard, but add-in cards (yeah, PCIe again) are more common.
Physical Virtualization
So if today’s servers are a mix of components interconnecting over a PCIe bus, wheels start turning. What if you could use PCIe switches or fabrics to “decompose” a server into a set of components and then “recompose” these resources into a flexible set of servers? Now you’ve got composable systems! Build a pool of CPUs, NVMe storage, storage-class memory, GPUs, and network cards, mix in some virtualization and management software, and draw on it as needed.
It’s an awful lot like server virtualization or containerization, but using physical resources instead of make-believe software. And it brings many of the benefits of virtualization, including flexibility, right-sizing, and efficiency. And this concept really meshes well with the current trend towards “software-defined” systems that are treated as cattle rather than pets. Imagine being able to call up any size and configuration on demand but instead of it being some cloud service it’s your server in your datacenter.
Storage and networking were pretty straightforward – Dell did it a while back with the VRTX and PowerEdge FX. And it’s not a huge leap to put some GPU cards on the other side of the PCIe switch too. When Liqid announced an FPGA solution at Supercomputing 2018, I was impressed. But when they showed up at Gartner Datacenter Conference with Intel Optane on board, I really perked up. Although it’s not DRAM, Optane gets awfully close in terms of latency and enables a whole new class of applications.
Stephen’s Stance
PCIe is the standard interconnect for system components, and companies like Liqid are making hay with it. Although we’re only at system scale right now, I can see solutions like this deployed in racks soon enough. Combine composable hardware with cloud management solutions and you’ve got a whole new system architecture. I can’t wait to see what comes next!
