AI is the next big stop in IT that every organization is pumped to get started with. But as organizations prepare for AI workloads, they’re coming to realize that the current technology landscape, especially storage systems are deficient to support its level of data-intensive applications. At the AI Field Day event in May, DDN presented A³I, their own brand of EXAScaler FS for AI that besides surpassing the performance capacity of traditional storage has all the optimizations required for high-performance workloads.
Conventional storage Is Unfit for AI
When getting started with AI, organizations often forget to scrutinize their existing IT infrastructures, mostly because they get immersed in preparing the compute side of things. The storage is often the most overlooked part in that. AI workloads for whom data is so fundamental, and massive amounts at that, at the very basic need robust storage systems that can keep up with big storage needs.
The AI pipeline encompassing the four stages of ingestion, preparation, training and inferencing has varying storage needs which peaks at ingestion and then again in inferencing. At the ingestion, enormous amounts of unstructured data are consumed and without an underlying storage system to support it, AI projects are doomed to fail. Additionally, with AI workloads, both read and write must happen at high speed, and traditional storage systems typically offer one or the other. The AI lifecycle is iterative, and running it the storage needs to stay active all through. Ergo, it is best to run AI projects on top of storage infrastructures that are a lot nimbler than traditional storage, and are high on all three counts of performance, scalability and flexibility.
DDN Presents a Storage System That Was Built to Power Data Intensive Workflows
DDN has a purpose-built storage system designed to meet all of these AI needs. In order to seamlessly support the varying storage requirements through each stage of the AI pipeline, DDN built A³I with an EXAScaler file system. At it’s root, the EXAScaler FS is a high-performance, scalable parallel file system that is extremely well-suited for AI workloads. To make sure that AI workloads do not experience bottlenecks in production that are fairly common with traditional storage, DDN chose this already powerful file system to be at the foundation of its storage solution.
DDN augments EXAScaler with select capabilities that all go into ensuring that workloads no matter how demanding enjoy consistently fast processing and high throughput. Where other seemingly powerful storage solutions fail to unleash full performance right out of the box, DDN A³I can give applications all of its power enabling them to work many times faster.
A³I – A Storage Optimized for AI
At the recent AI Field Day event in Silicon Valley, DDN showcased A³I, the AI storage solution. The presentation began by Senior VP of Products, James Coomer giving a high-level view of the challenges specific to storage surrounding adoption of AI at scale. For the rest of the presentation, he talked about A³I, drilling down into its architecture and explaining how DDN optimized the underlying EXAScaler for AI.
Inside the A³I architecture, DDN has not one, two file systems that fuel its power. The EXAScaler FS version 6 sits atop a second file system, Lustre, which DDN acquired from Intel three years back. Also a parallel file system, Lustre already has the reputation of being a product of distinction in supporting high-ingest performance and parallelism at reduced cost. Leveraging the two file systems, the A³I is a new class of Tier1 storage that delivers 100% granular parallelism combined with less operational cost and power consumption.
Optimized for AI, the A³I platform can be scaled limitlessly and cost-effectively. Users can configure their platforms choosing from the EXAScaler appliances which include NVMe all-flash and SAS slots. Easy to deploy and manage, it is available on all big cloud platforms including AWS, Azure and Google Cloud. Multi-Rail makes deployments simpler and networking easy with automatic detection and simple management of several network interfaces, all on one network.
Using the EXAScaler Intelligent Client, A³I provides intelligent monitoring whereby users have full visibility of data paths and workloads and timely insights on the native workload analyzer. A³I’s security framework enables easy and safe sharing of files between professionals and teams. Thanks to its wide set of features, access and authorization can be granularly managed and controlled to prevent unauthorized access.
In a case study called NVIDIA Selene, NVIDIA employed DDN’s A³I to power several AI supercomputers and the results established DDN’s performance claims for A³I. The deployment resulted in a dramatic uptick in performance enabling faster iteration. A point of differentiation here is that the EXAScaler 6 can eliminate hot spots from the storage architecture by balancing the I/O power to every server. More feature sets and simplification are underway.
Final Verdict
One of the greatest benefits of DDN A³I is that for all the compute and capacity it offers, it is a surprisingly simple storage system that wholly sidesteps the backend complexities that commonly plague operators. With end-to-end parallelism, simple and unlimited scalability, and high levels of performance, the A³I is a data solution of distinction that has uses across advanced industries. Safe to say, it has the potential to be enterprises’ secret to AI excellence.
For more on A³I, head over to the Tech Field Day website where you will find other presentations by DDN from the recent AI Field Day.
