One of the advantages of Kubernetes development and operations is the modular construction of the infrastructure itself. This advantage is extremely prevalent when it comes to integrating storage: Kubernetes has the right level of abstraction so it can keep a loosely-coupled connection to the back-end storage subsystem. Although the Kubernetes Container Storage Interface (CSI) is a helpful abstraction, one long-time challenge is that it limits the number of platform-specific features that can be utilized by containers. This makes Dell’s new Container Storage Modules (CSM) approach a huge benefit, since it unlocks all the features at the hardware and software storage subsystem to bring those capabilities closer to the applications running on Kubernetes.
Abstractions With Advantages
Container Storage Modules (CSM) are built on top of the Container Storage Interface (CSI). This allows abstraction to remain while also getting more intelligence from the container platform layer, bringing the benefits of scalable enterprise storage platforms closer to the cloud-native nature of Kubernetes. CSM removes the need to program key features into Kubernetes itself, reducing complexity in the core Kubernetes codebase.
The CSM concept creates appropriately-opinionated modules that are loosely coupled. This is the ideal level of interoperability, because any underlying storage system with enhanced capabilities can expose those features using CSM as a common method.
Who Will Use CSM Features?
The primary consumer and administrator of storage in traditional enterprise environments is a storage administrator (or perhaps the operations team). But companies are now creating a new practice group, dubbed platform operations. Even though they are more developer-centric, the platform ops teams are still on the “Ops” side of DevOps.
CSM now allows developers to pick and choose how to leverage features since it allows programmatic access to them. The current modules that are available and fully supported by Dell today include authorization, replication, resiliency, observability, and volume snapshotting. There are more in tech preview today (app mobility and secure), and we expect that these will reach production-level support soon.
It’s easier to think of the value if we look at some active use-cases that Kubernetes operators and cloud-native application developers will know very well.
Use-Case #1 – Scaling Container Storage
The original intent of containers was ephemeral computing, but this has shifted. Today, long-running workloads and shared container storage have become common as DevOps teams seek to get the most out of Kubernetes as a hosting platform without having to refactor their applications to be 100% stateless.
The issue with stateful, long-running workloads is that they can have operational patterns that aren’t “typical” for Kubernetes. One of the challenges is how to scale storage without re-spawning containers to a new location. Application developers and operations teams have been holding back app migrations to Kubernetes because of the limits to storage scalability.
Using CSM to abstract the underlying storage lets developers present and manage the storage endpoint programmatically. They will not have to worry about requesting access through a ticketing system to make changes because they can either be given access to manage it themselves or the operations teams can use simple, programmatic methods to operate and expand, contract, or set properties like tiers and capabilities for the storage system.
Use-Case #2 – Authorization For Container Storage
Access to storage is traditionally managed at the cluster layer, and there has not been effective granular access based in the core RBAC for Kubernetes. This is very risky and leads to issues like allowing unnecessary access or, even worse, leaking authentication code and secrets.
The authorization module in CSM remedies this concern through the programmatic selection of a storage target for the container based on some criteria (e.g. location, encryption state, type of storage). It also allows for programmatic management of authorization.
CSM authorization allows application developers to include storage access in their process with much less risk exposure. It also makes quotas and other properties of storage management much easier for the operations teams without having to manually allocate at the storage subsystem for every request. Authorization is now exposed to the container with the native CSI and existing properties that are already part of storage management in Kubernetes.
Use-Case #3 – Cross-Cluster Data Replication and Recovery
Another common challenge facing Kubernetes DevOps teams is cross-cluster data replication. Application failover and recovery to alternate clusters is a challenge when complex data and applications are involved. CSM allows for snapshots to be created and replicated to a secondary cluster. When the new containerized application is spawned in that cluster, it will have the latest revision of the data. This array-based replication for cloud-native environments allows DevOps teams to realize higher performance, simplified management, and an efficient use of resources. In particular to performance, the replication available through Dell’s CSM helps enterprises avoid the overhead of data transfer being done in software above the storage hardware, instead completing the replication at the storage array level.
The unique advantage is that CSM methods can combine all three of these use-cases. By having scalable underlying storage that does not require container and node restarts to recognize configuration chances is game-changing. The ability to authorize storage access extends quota management and storage type access management across the whole environment instead of just inside each cluster.
This is a fantastic solution for business continuity and disaster recovery and gets rid of the need for the replication to be handled inside the application itself. Considering how developers manage storage today, they will be very excited to hear about using CSM for Replication.
Conclusion
Kubernetes and cloud-native application design is a big change for many organizations. The lack of specific performance-oriented support of powerful underlying storage systems has long held back application modernization. CSM brings features like replication, resiliency, extended observability, and volume-level snapshotting to ensure programmatic control and the right abstraction to keep Kubernetes core as simple as possible.
The work being put out by the Dell platform team to expose and fully leverage storage features using Container Storage Modules is a huge win for enterprises adopting Kubernetes as a containerized application hosting platform. The team has produced lots of helpful CSM resources, from the core guide to specific live GitHub content for each of the associated modules. CSM is the beginning of a new era in storage integration that will hopefully drive more innovation by the entire Kubernetes community and supporting vendor ecosystem.
