Quantum recently Showcased Myriad, a software-defined, all-flash, scale-out file and object storage solution that released late last year. Myriad is based on a modern cloud-native architecture that is designed with modern workloads in mind.
This tech note delves into the types and characteristics of next-generation workloads, and their specificities of storage, with an overview of Quantum Myriad in the context.
Characteristics of Next-Gen Workloads
Data growth remains one of the key drivers of investment in data infrastructure systems, including storage. This trend is largely driven by cutting-edge workloads such as artificial intelligence and high-performance computing (HPC) applications in the fields of medical research, genomics, engineering, and more. These workloads typically ingest and process large amounts of data. High throughput and low latency are absolutely critical to avoid any bottlenecks, especially when GPU processing is involved, making general purpose storage systems redundant.
The use cases rely on data workflows where data is progressively transformed through a series of steps. Specific applications are used at each step, some requiring data to be served via file protocols (NFS or SMB), while others requiring it to be placed in object storage buckets, driving the need for multiprotocol-capable storage.
Modern Storage for Next-Gen Workloads
Because of its distributed nature, scale-out storage requires an efficient and modern codebase that can seamlessly withstand massive scalability while delivering billions of IOPS consistently.
Implementations where data services are containerized and distributed, offer the best scalability. Core data services like compression and deduplication are essential as they significantly improve storage density and efficiency, delivering more capacity, and reducing $ per GB ratio. Caching mechanisms must ensure sustained throughput, and data writes must be organized for maximum efficiency.
Another key aspect of scale-out storage is its upgradeability and overall lifecycle. Storage scales exponentially through addition of nodes over time. More nodes are added to increase capacity and performance, while older ones eventually retire. A software-defined, hardware-agnostic approach allows organizations to elastically scale their environment while managing costs and lifecycle aspects.
Storage solutions designed to support modern high-performance workloads must have plans to include support for NVIDIA GPUDirect to ensure seamless connectivity with NVIDIA GPUs.
Quantum Myriad: Future-Proof, High-Performance Scale-Out Storage
Quantum’s illustrious experience 40-year tenure in storage, and products like StorNext (scale-out file storage) and ActiveScale (object storage) under its belt make it a notable name in the industry. Targeted towards specific use cases, the solutions are great on their own, but are not meant for the highest-performance file workloads. This steered Quantum’ towards the development of Myriad.
Myriad combines the capabilities of its predecessors and delivers a file and object scale-out solution built on modern cloud-native architecture. It is software-defined, massively scalable, and architected around NVMe flash media, and fast, lossless network interconnects. It guarantees high throughput and low latency while remaining hardware agnostic.
From an architectural standpoint, a Myriad deployment consists of load balancer nodes, NVMe storage nodes, and one or more deployment nodes – all interconnected over an NVMe fabric. Customers can start with three storage nodes, and scale to hundreds in a single distributed, scale-out cluster, seamlessly achieving multi-petabyte deployments.
Myriad supports multiple protocols as seen in Figure 2 below, enabling data processing workflows that require multiprotocol support. In addition to snapshots and clones, it includes inline deduplication and compression. More feature-sets will be added in the future. It is worth noting that snapshots are currently not immutable.
The file system implementation is supremely interesting and deserves an article of its own, but here is a short version. It is based on a transactional key/value store designed to be lock-free, while also avoiding write collisions. Although this would be a challenge for collaboration solutions, the nature of workloads running on Myriad combined with its inherently low latency would eliminate the probability of a write collision.
Another interesting quirk is Myriad’s use of unusual networking concepts such as BGP protocol for load-balancing. But it doesn’t stop there. The solution offers a lot more to explore.
On its website, Quantum provides storage node specifications to help customers design their environments according to their requirements.
Conclusion
Myriad is the logical evolution of Quantum’s storage portfolio. The solution bridges the gap that existed between StorNext and ActiveScale by introducing unified file and object storage capabilities in a scale-out architecture.
As its predecessors focus on traditional storage use cases, Myriad marks a clear inflection point for the company. Based on a radically modern architecture, it is designed for the most demanding workloads of its time, such as AI.
Launching with a base set of services, the solution’s initial feature-set is sufficient for the most pressing needs around capacity, performance, and scalability, with aspects reminiscent of the approach adopted by hyperscalers.
Be sure to find out more about Myriad on your website. Also check out other Myriad-related tech notes here on Gestalt IT, that cover other aspects of the solution.
