Modern software-defined storages are increasingly borrowing from the cloud model to design their architectures. This is not an effort to just mimic the scaling flexibility, to make the scale-out model a cloud-scale or web-scale one. But unmistakable resemblances are also being seen in the technical implementation.
The Emergence of a New Generation of Scale-out Software-Defined Storage
Scale-out software-defined storage solutions aren’t new. To take advantage of NVMe flash, it’s required that the software is designed for high parallelism and ultra-low latency, from the initiator to the target. Of course, a good hardware architecture, and a high-performance network set up with efficient protocols like RDMA, are important supporting ingredients to this.
As workloads and use cases are evolving, for demanding technologies like AI, the need for a new generation of storage solutions is strongly felt.
On November 29th, 2023, Quantum announced the general availability of a new solution, Myriad. Quantum boasts a comprehensive end-to-end portfolio of software solutions known to unlock the true value of data. A company with a pedigree in storage, Quantum offers solutions ranging from archiving to file and object storage, each targeted at a particular set of uses cases. In all its solutions, data is the chief strategy and center of focus.
What Makes Quantum Myriad a Distinguished Solution?
Myriad is a new all-flash storage for files and objects aimed at high-performance, AI workloads and and entertainment use cases. The solution boasts a powerful and flexible scale-out cluster architecture. It relies on a 100G Ethernet fabric that interconnects different type of nodes namely, NVMe storage nodes and system load balancer nodes. A management network connects the storage nodes with the control nodes. The solution is based on standard x86 server for the storage node, and whitebox switches for the others.
The use of whitebox switches is quite an innovative approach, which some cloud providers have set the trend for. An example is Microsoft Azure’s wide use of SONiC. This allows operators to add some logic in the infrastructure part, and optimize the data-path or the manageability in a highly energy-efficient manner.
There is innovation on the software side as well, starting with a new POSIX filesystem designed to store files as objects in a key-value store. Everything is backed by NVMe disks that are divided by zones – as opposed to the legacy block approach – and grouped by zone sets, a concept typical to many cloud architectures.
The software uses familiar and proven cloud technologies, like microservices and Kubernetes to provide cluster orchestration, a thing that is becoming de facto in new storage solutions.
There is a focus on manageability and simplicity, not only in providing clear and easy user interfaces, but also enabling near zero-touch storage node configuration for new nodes.
A Powerful Data Platform
Filesystem snapshots and clones are intrinsically supported by the write-on-write nature of Myriad’s key-value store. The architecture also enables inline data services such as deduplication and compression, and metadata tagging to accelerate AI/ML data processing which will be available in a future release. As of today, snapshot immutability is not offered, but Myriad’s architecture has room for it to be added at a later date.
For data protection, there isn’t yet an integrated backup solution or a third-party native integration, but Quantum is working toward using other portfolio products to provide both data backup and replication.
Only NFS is available for front-end protocols,, but others are on the roadmap, likely starting with S3 and a CSI integration with Kubernetes.
It will be interesting see where this architecture fits in – computation storage, AI workload or data processing?
To learn more about Myriad’s architecture and why you should use it, explore Quantum’s product page. Also check out this episode of Gestalt IT Rundown for a first impression of the solution.
How does myriad compare to hammerspace?