Organizations starting out in the cloud spare no cost in setting up the largest storage for their workloads. But no sooner do they move their applications, than they max out. Cloud is many things – it is resilient, scalable and easy – but one thing it is not is free. Cloud storage costs a good deal, and if businesses are to reduce their cloud bills, the only way forward is to manage their storage requirements efficiently. The rule of thumb for that is to gain clear and total visibility.
At the recent Storage Field Day event, AWS presented Amazon S3 Storage Lens, an observability solution that is designed to provide organization-wide visibility from the broadest down to the lowest level, so that users can keep a close watch on their usage and stay on top of the cost.
The Cost Conundrum
Looking at the cost problem in cloud as a whole, it is easy to conclude that cloud is expensive. But quite contrarily, vendors position cloud to be cost-saving. Why are the ground realities different from what’s promised? Because harnessing the cost savings in cloud is the real challenge, not the cost itself.
When we take a step closer, a myriad of factors meet the eye that are contributory to the cost escalation. Those are the problems that organizations need to address first to encounter the flaring cost situation, the first of which is capacity procurement pitfalls.
Overbuying is perpetual in the cloud for two reasons. First, forecasting the storage requirements of applications is tricky business. You can easily go over or under, and rarely ever hit the mark. So, to be safe, companies overprovision. Especially, with unlimited capacity and easy scalability, it is tempting to scale up than to downsize capacity.
20/20 Visibility
A research conducted by Nutanix found that 43% organizations find managing costs in multi-cloud challenging. One way to overcome this challenge would be to separate capacity organizations need from capacity they waste.
AWS is guiding its customers towards better cost management in the cloud with Amazon S3 Storage Lens. Its mission is to help customers “understand storage from an aggregated broad view to a granular narrow view”.
“Amazon S3 Storage Lens is what we recommend as your first stop in getting visibility into your storage because it has the broadest view of your entire organization and account and a wide array of different metrics to investigate,” says Lee Kear, Principal Solution Architect, Amazon S3.
Kear laid out the USP of S3 Storage Lens – “The essence of why customers use Storage Lens comes down to visibility.”
While Amazon S3 Storage Lens provides a holistic view of the organization, Amazon lends users the means to drill down into the deeper and lower levels of their accounts using other observability features, namely S3 Inventory, S3 Storage Class Analysis, S3 CloudWatch Metrics and S3 Server Access Logs. Available at different price points, these features enable users to inspect elements inside each bucket forensically for close monitoring.
Amazon S3 Storage Lens
At the recent Storage Field Day event, Lee Kear gave a presentation on the Amazon S3 Storage Lens. She kicked off the session with a rundown of the top features of S3 Storage Lens. Kear followed it up with a live demonstration of the solution to make its capabilities visual.
S3 Storage Lens provides data tailored to 29 metrics. It comes in two flavors- Free and Advanced. The Free version is auto-enabled in all AWS accounts, whereas the Advanced tier is a paid version that requires to be upgrade to. The Free version affords full visibility, but customers want to upgrade to the paid version to avail the additional capabilities of optimizing and automating observability.
The Free version has 15 metrics, and the Advanced, 14, broken down into categories. The metrics available in the Free version include Summary, Cost Optimization and Data Protection. Under Advanced, there’s Activity metric which includes GETs and PUTs, download/upload bytes, retrieval rate and more.
“These are some of the most valuable S3 Storage Lens metrics that you can use to analyze bucket level access patterns such as identifying your hottest buckets or to allocate abandoned workloads,” said Kear.
Starting at the higher levels of organization and account, users can move down the lower levels of Region, Storage Class, Bucket and Prefix for a closer inspection. Kear says “S3 Storage lens is the first tool to provide this with visibility at the prefix level”.
There are three ways users can access S3 Storage Lens. It comes with “an Integrated Dashboard built directly into the S3 console”. Customers enjoy the “Integrated Dashboard experience” because it offers a simplified and organized view of the S3 Storage Lens and other storage-related workflows. The dashboard is pre-configured and needs no installation.
The second access method is Export Metrics. For customers who just want the raw data, the Export Metrics sends data directly to the S3 buckets in their accounts. Users can consume it through directly without viewing it on the UI.
The third option is Amazon CloudWatch, a feature that is now a part of the Advanced tier. Designed with DevOps people in mind that like to tally up the metrics with monitoring metrics coming in from other AWS services, enabling the CloudWatch Publishing option will send the data to CloudWatch where users can be access it via API or CloudWatch native features.
The Advanced version of S3 Storage Lens costs 20 cents per million objects monitored per month. Although it comes at an extra cost, upgrading to the advanced version has its benefits that the Free version does not cover. Kear highlighted granular visibility into the lowest prefix level and the longer 15 months historical range compared to the much shorter 14-day range in the Free version. The CloudWatch publishing feature is the cherry on top.
Wrapping Up
It is tempting to go for a XXL size storage in cloud because why not, after all its unlimited capacity. But when that reflects in the invoice, it rarely inspires anything but alarm. That’s why Amazon S3 Storage Lens is a vital tool to use when struggling with cost escalation from growing cloud footprint. S3 Storage Lens aggregates data from across all accounts, filters it to an easily consumable format, and presents it with analytics through an interactive dashboard making it infinitely easier for IT personnel to keep a tab on storage usage, activities and most importantly, get recommendations on best practices.
To know more about Amazon S3 Storage Lens, be sure to check out the presentation above. Also check out other presentations by AWS from the recent Storage Field Day event to know what’s in store.
