I’ve got a new video podcast up and running: Raising the Floor is a series of discussions about the future of enterprise IT. I kicked the series off talking about one of my favorite topics: Cloud storage.

I invited two excellent guests to join me for this conversation:

1. Josh Goldstein, VP of Marketing, Cirtas
2. Andres Rodriguez, Founder and CEO, Nasuni

It was a pretty broad discussion, all packed into less than half an hour. I urge you to check out the podcast (and subscribe in iTunes, rss, or email) but I wanted to share a few excerpts. You can also read the entire transcript over at Foskett Services: I’ll be posting it as a series of articles over the next week!

Andres Rodriguez: I think one of the things that hurt StorageNetworks at the time was the fact that they couldn’t deploy the equipment in an efficient, multi-tenant way. And so, if you look at the new cloud architectures, (places like Nirvanix, places like Amazon) those systems are designed from the get-go to be shared among many, many users, and make very efficient use of the equipment and the software running it across that user base.

…

Josh Goldstein: Today, the cloud is built on top of process that are very difficult for most companies to replicate on their on. So, the price you’re paying to your cloud provider includes not just storing your data but also keeping multiple replicas of that data spread across different geographical sites.

You’re highly protected against not only a disk drive failure, but also an entire array failure or even an entire site failure where your information’s still is survived those kinds of events and is remaining accessible to your when you need it.

That’s something that for most organizations to engineer that level of reliability is extremely expensive and difficult for them. The cloud providers have been able to do that at scale and still deliver the capacity to you with that type of protection at a price point that’s really pretty amazing.

…

Andres Rodriguez: I’ll start there and I said anything that traditional storage world, we have file based systems. We have block based systems and that is exactly where Cirtas and Nasuni stand. We are the equivalent of say a company named EMC and NetApp for the cloud world. The approaches are complimentary. And they are both trying to solve the same problem. I’ll start on the file side but Josh can take on the block. But on the file level is really, look, if you want to have something that behaves very much like a file server, say like a NetApp box. It means you are going to have a file system, you want to have a protocol to export it locally on so something like CIFS, you are now going to have access to directory integration so that you can have access control. This is what makes a file server useful in the datacenter.

…

Andres Rodriguez: One of the really interesting things about that is that Josh is essentially describing thin provisioning in the classic sense. But unlike doing thin provisioning and this is the cloud is automatically provisioned. So he is essentially promising the volume. But then the volume is growing gradually, smoothly without any administrative interference. Without you having to worry about it; the volume is filling in its data as it goes. And that is one of the things we talked about in the beginning.

The cloud really allows you to build a different type of storage system, because automatic provisioning is really thin provisioning should be but isn’t. When you’re talking about physical linked arrays that are limited by actual hard drives, you know, running, spindles running in your datacenter.

Again, you can watch the video of the whole conversation right now over at Foskett Services, or you can read the transcripts that will be posted there over the next few days.

Disclaimer: I worked at StorageNetworks, and Nasuni is a client of Foskett Services.

© Stephen Foskett for Gestalt IT, 2011. | Talking Cloud Storage Gateways With Nasuni and Cirtas
Read more posts categorized as All, Storage, Tech Field Day Roundtable Podcast

Most thin provisioning solutions are colossal hack jobs. How about real automatic provisioning instead?

I’ve never been a fan of thin provisioning as a storage management tool. Don’t get me wrong, I love having thin provisioning in my toolkit to overcome the limitations of conventional filesystems. Thin provisioning just gets under my skin when folks try to use it to solve business problems like long deployment time and slow purchasing cycles. If you attended any of the thin provisioning sessions I’ve presented at Storage Decisions, Interop, E-Storm, or elsewhere then you’ve heard my wistful dreaming of real automatic provisioning without the hackery of thin provisioning systems. But perhaps I didn’t mention that actual automatic provisioning actually exists today! It’s one of the many things I love about API-driven cloud storage!

Why Shouldn’t People Use Thin Provisioning?

I’ve gone into this many times, but I probably said it most-succinctly in my post, Use Process Solutions For Process Problems, Technical Solutions For Technical Ones:

I have a long-standing love/hate relationship with thin provisioning, one of the many proposed technical solutions to the utilization problem. Thin provisioning eliminates many technical challenges: It simplifies adding capacity to the Drobo that serves as my home office storage center; the ability to automatically grow VMware images makes virtualization practical in the tight confines of a laptop; and it contribute to the usefulness of advanced solid-state storage systems like the new Nimbus S-Class. But I have serious reservations about using thin provisioning to over-subscribe enterprise storage systems due to failures of capacity planning and IT-to-business communication. Thin provisioning will only make process issues worse.

This isn’t always the case, of course. Some people (including me) use thin provisioning to solve technical issues relating to inflexible filesystems. As Mr. Backup notes, you shouldn’t go around saying that everyone using thin provisioning is stupid. It’s just that some people (and vendors) over-rely on thin provisioning and use it to cover up a more-serious business problem.

So What’s Really Wrong With Thin Provisioning?

The biggest problem with most thin provisioning implementations is that they’re not really all that thin most of the time. I don’t blame the storage vendors: It’s really, really hard to “do” thin provisioning with conventional filesystems and block storage.

I spend lots of time talking about this in my “thin session” presentation, but I’ll sum it up here:

1. There’s a lack of information exchange between the application, file system, volume manager, and array controller so no one “knows” what to thin and when
2. De-allocating on delete is a pain because most filesystems don’t really delete data
3. There are two thin provisioning options, neither of which are simple or bulletproof:
   1. You can add smarts to the server, reporting back to the array when data is deleted
   2. Or you can add smarts to the array, snooping on the filesystem or reclaiming zeroes
4. Thin provisioning granularity has an impact on effectiveness, but not as much as you might think
5. Most arrays use lazy and ineffective thin reclamation; it takes real engineering to have in-line reclamation that doesn’t kill performance
6. Filesystems just weren’t designed to be thinned – they fragment, have alignment issues, etc…

It all boils down to a simple fact: Conventional systems expect to be stored “fat” on local disks, not thinned and virtualized and mangled. It’s possible to make it work, but takes so much engineering and processing power that you start wondering if it’s all really worth the bother.

We Need Automatic Provisioning Instead

I have a dream for automatic provisioning rather than reverse-engineering filesystem layouts, adding shims and semaphores, and hunting for zeroes. I want a “storage platform” that has a concept of data stored and deleted and allows applications to communicate needs beyond basic provisioning. In short, I want cloud storage.

Most people get so bound up thinking about the “cloud” part of cloud storage (service providers, REST, public/private, etc) that they overlook the obvious “storage” benefits! Cloud storage protocols enable applications to do amazing things with storage, decoupling them from the old assumptions about “my disk” and “my filesystem”. Yes, the Internet has a speed limit (both for throughput and latency). Yes, cloud storage is more expensive on a per-used-bit basis than on-site hardware. But these limits evaporate when one looks at total cost of ownership or deploys local equipment.

Real automatic provisioning is enabled by cloud storage access methods, made real by cloud gateways, and goes way beyond what any conventional thin provisioning system is capable of:

1. Imagine actually paying for exactly and only the storage capacity you are using! I use cloud storage services from Amazon and Nirvanix to host all of the images on this blog as well as the Field Day Roundtable Podcast video files. There is zero waste here: I’m paying only for used capacity and data transfer, and not a dime for empty space.
2. Imagine having unlimited scalability with no migrations! I don’t care where my service providers store my data and what equipment they use. They have an SLA to meet, and they’ve always met it. Internal or private clouds could do the same, liberated from the vendor support matrix lock-in game.
3. Imagine having your “bill” immediately reduced as soon as you delete data! I can temporarily use all the capacity I want, delete what I no longer need, and even switch providers with zero capacity “inertia” and clean-up. Public and private providers can immediately repurpose that capacity without any kind of reclamation process.

One of the dings against cloud storage is that it’s hard to start using. That is, since it uses a “proprietary protocol” it can’t be used by real-world applications. Well, more and more applications are supporting cloud providers directly. And cloud gateway products from companies like Nasuni, Cirtas, StorSimple, and others abound, allowing regular applications to go there, too. As Nasuni’s Rob Mason says, “there’s always more space available.”

Stephen’s Stance

We need to “get past” thin provisioning and the rest of the technical cruft that comes from using outdated concepts like block storage and filesystems. We need revolutionary storage that lets applications communicate in more-effective ways about capacity usage, retention requirements, replication, and similar needs. In short, we need cloud storage for a whole lot of reasons other than outsourcing of management. And it’s getting easier and easier to use cloud storage thanks to integration, cloud gateways, and the fact that the APIs are really simple and easy to use!

Watch my presentation, Extreme Tiered Storage: Flash, Disk, and Cloud for more detail on this topic!

Disclaimer: Although this is not a paid, sponsored, or for-hire article, Nasuni is a client of my consulting company, Foskett Services. I worked for Nirvanix and remain their customer. I am also a customer of Amazon and former customer or Rackspace. Symantec, maker of the Veritas storage products including the Thin API referenced here, is also a client and sponsor of Gestalt IT activities. I have lots of storage array companies as occasional clients and sponsors, including many like 3PAR, HP, Data Robotics, and EMC who make thin provisioning arrays.

Image Credit: “Push Pad to Open Automatic door. Right…” by @davestone

© Stephen Foskett for Gestalt IT, 2010. | Overcoming The Limits Of Thin Provisioning With Automatic Provisioning!
Read more posts categorized as All, Storage

IT is crossing a chasm, and we may not be prepared for the new direction we are taking

I’m an IT revolutionary. I talk all the time about the quaint backwards “state of the art” in enterprise IT, what with its (many) decades old protocols, paradigms, and practices. What we call modern is really just a charade of faked-out old-fashioned open systems infrastructure: Pretend servers talking to fake disks over frankenstein networking technology.

What is Flexible IT?

I’ll be doing a “Twitterview” about Flexible IT today with NetApp. While I’m sure most companies would like to define flexible IT as “what we have in our product catalog,” I’m not going to play that game. IT has to change its ways or The Techie/Business Schismwill get us all.

I see the roadmap ahead as one journey over two paths:

1. Tactically, IT infrastructure groups must immediately begin implementing technologies that are responsive to user needs and focused on improving the performance of the business applications they support. This is what NetApp is calling Flexible IT.
2. Strategically, IT infrastructure must change entirely, implementing revolutionary platforms to support the next-generation apps that are being widely developed. Call this “cloud” if you must, but just about everything we take for granted today will lose significance in this new era.

Flexible IT (as it stands today) is tactical. IT infrastructure must be entirely virtualized to enable “right now” provisioning and reconfiguration. This means embracing enabling technologies like server-side hypervisors, blade computing, converged networking and I/O, and advanced storage technologies.

What Does it Mean to be a Service Provider?

But real IT flexibility will mean higher-level abstraction. The next applications won’t want to run on “a server” – even a very flexible one. These apps will require a programming platform that abstracts away the entire concept of “server.” Although there will always be servers and disks and such, they will fade into the background, hidden under a layer of application services.

You know how some people like to think of themselves as “service providers”? They’re on the right track, though I’m not sure that they (yet) grasp the full significance of that concept.

A service provider has to provide the services required by consumers or it will fail. This means IT infrastructure has to prepare the environment demanded by IT application developers, rather than just shuffling around the chairs and tables and hoping customers will stream in.

What happens when the IT applications group asks the IT infrastructure manager to implement Microsoft’s Azure Services Platform? What if they want to run on App Engine or Force.com? The head-scratching in the VMware community about SpringSource is a sign that IT infrastructure just doesn’t comprehend what’s happening (yet).

Stephen’s Stance

This is a strategic shift, and we have to be ready. Being ready means changing not just what we do but also how we do it. The systems we employ in a decade might look familiar inside, but the process of delivering services on them will be entirely different. Making this shift requires real flexibility from IT, not just another round virtualization fake-outs.

Image credits: Highway construction Guizhou S0404 by sweart, VW engine by Grant Foskett

© Stephen Foskett for Gestalt IT, 2010. | Flexible IT and the Path to the Services Future
Read more posts categorized as All, Cloud Computing, Storage

You may not know Caringo, but you have probably heard of cloud storage, EMC Centera, and Dell DX. Read on to learn the link!

Now that the hype of “cloud everything” is subsiding, organizations are getting down to work deploying cloud storage to do actual useful tasks. The march from CAS to cloud to object storage has seen high-profile high-end flare-ups (think EMC Centera and Atmos) but the bulk of work is done by more pedestrian (think lower-cost) hardware and software. Through it all, Paul Carpentier has been at the forefront. Now his company, Caringo, is back in the news, delivering much-needed storage service features like multi-tenancy, named objects, and dynamic caching.

For essential background, check out my article, From CAS to Cloud: Revolutionary Storage

The Back-Story of Caringo

The “Caringo” company name refers to its three founders, CTO Paul Carpentier, President Jonathan Ring, and CEO Mark Goros. Carpentier is the man behind CAS pioneer FilePool, which EMC acquired and markets as Centera. The three formed Caringo and launched the CAStor product in 2006 as a software alternative to Centera.

Although you may not have heard of Caringo, you may have encountered their product in the form of the Dell DX object storage system. Many were puzzled when Dell, known for its EMC-powered storage offerings, embraced Caringo for object storage, but those familiar with the products weren’t surprised. Caringo’s approach is much more in line with Dell’s image of affordability, simplicity, and commodity products, and their relationship with EMC is increasingly shaky due to their recent acquisition strategy.

Caringo’s CAStor is a software product that transforms commodity servers into a scale-out object repository. It is all-inclusive, with compliance, tiering, spin-down, and replication part of the total package. Like most CAS and cloud storage solutions, CAStor uses a simple HTTP interface for client access, with “gateways” available for NAS along with some native support from applications.

What’s New in CAStor 5?

Caringo has set a course for the service provider market, adding essential features like multi-tenancy and flexible permissions to version 5 of CAStor. Although still pitched as an object store, CAStor 5 is close enough to be thought of as a cloud storage platform.

The ability to support and segregate multiple “tenants” is a holy grail for service provider storage systems and a key ingredient of cloud storage solutions. CAStor 5 can be segmented into multiple domains, each with its own security and authentication and each subdivided into “buckets” for different applications. This would be useful both for a public service provider and an internal-only solution, since segmenting applications is relevant in the enterprise as well.

CAStor 5 no longer clings to system-assigned names for objects, allowing users to assign their own names for public consumption. This is a huge advancement for CAS, and was one of the key differentiators of cloud solutions which often directly serve content to web clients. Another “ripped from the cloud” feature is dynamic caching, allowing high performance access to popular content, again useful for direct client access.

Stephen’s Stance

Caringo seems reluctant to wear the “cloud storage” mantle, but their product has been steadily moving in that direction. CAStor 5, with its multi-tenancy, segmented security and authentication, named objects, and caching, looks an awful lot like Amazon S3 and the rest. But the hype around “cloud storage” is dying away. Businesses looking for functionality rather than marketing labels will find a lot to like in CAStor.

© Stephen Foskett for Gestalt IT, 2010. | Caringo Bulks Up CAStor For Cloud Services
Read more posts categorized as All, Cloud Computing, Storage

We need to move beyond fake disks and deploy application-centric storage

Change is not a word normally associated with storage, and revolution is practically unheard of. Today’s modern enterprise storage systems and networks employ massive resources to do one simple thing: Emulate the basic hard disk drives used over three decades ago. But cracks are appearing in our mausoleum of fake disks: Application developers are discovering the value of object storage, and storage systems are appearing to support this need.

I also wrote about this two years ago, proclaiming that We Need a Storage Revolution and forecasting The Techie/Business Schism

The CAS Revolution

Caringo founder and CTO, Paul Carpentier, rose to prominence around 2000 at FilePool, one of the prime movers in the content-addressable storage (CAS) space. I recall a light going off in my head as Paul introduced me to FilePool’s CAS technology back then, imagining the possibilities of the concept. Files would be stored based on “what they were” rather than “where they were” and could be organized according to application needs rather than the conventional “extent of blocks” or tree heirarchy.

CAS discarded decades of filesystem and block storage baggage, introducing a new method for storing and retrieving data that better-matched the burgeoning web and enterprise applications of today. I had seen the failure of the first wave of storage service providers from inside StorageNetworks, and it was this desire for a real storage revolution that led me to dive into cloud storage at Nirvanix almost a decade later. Although I am now on my own, I remain convinced that the future belongs to storage systems that look nothing like today’s SAN and NAS.

Shortly after that 2001 meeting, EMC acquired FilePool and launched it as the Centera product line. But CAS systems quickly ran into a serious roadblock: Conventional applications cannot read and write to unconventional storage systems like Centera. EMC pushed key software vendors (especially in the archiving space) to create special Centera interfaces, and the industry bogged down developing the XAM standard. Other companies, like Seven Ten Storage Software, jumped in to help with the translation from proprietary CAS interfaces, but the transition from legacy files and blocks to object storage has been long and slow.

Cloud Storage: Another Dimension

Meanwhile, in an alternate dimension, web developers realized they had a serious problem. They were developing applications that scaled massively, spanning servers and exhausting conventional filesystems. Conventional systems just wouldn’t cut the mustard.

Since they were soaking in web applications, these developers applied the lessons of web services to storage: Why not just make an HTTP connection and ask for an object by a unique ID rather than walk a filesystem tree? Why not encapsulate the “state” of this request in the request itself rather than make a lasting connection and association between the client and server?

Thus was born cloud storage, and it was bookseller Amazon who opened the floodgates with their 2006 introduction of a “Simple Storage Service” or S3. They allowed anyone to store and retrieve objects from their massive web services infrastructure. S3 and similar services from Rackspace, Nirvanix, and others, are special-purpose web servers, and their simple interfaces are wonderfully attractive to web developers. For example, this WordPress-based blog uses cloud storage to serve images to your browser!

Similarities in CAS and Cloud

Although developed from vastly-differing starting points, CAS and cloud storage are essentially similar: Both reject conventional blocks and files in favor of object storage; both organize data with metadata databases; both multiply and scale out. There is one other major similarity between CAS and cloud storage: Both are attractive to service providers.

Imagine you operate a business that stores data for customers. You would want a flexible infrastructure that would scale with demand and segment each “tenant” from others for security and performance. As we learned at StorageNetworks, conventional SAN and NAS systems just weren’t meant to work in this kind of environment. Whether operating an internal service or a public cloud, service providers require something entirely different.

Cloud storage was designed from the start with service providers in mind, embedding per-object and per-”bucket” security, scalability, and abstraction between hardware and clients. Although quite complex to design, cloud storage is amazingly simple to use, provided an application can interface with it.

CAS wasn’t designed like this. Systems like EMC’s Centera were created for the needs of applications like enterprise archiving, but secure storage of content and extreme scalability are critical here as well. But early CAS systems didn’t need simple web-style interfaces or extreme hardware abstraction. These were enterprise systems, after all.

The CAS/Cloud Colission

CAS wasn’t exactly successful. Although object storage found a niche in enterprise archiving, the enterprise storage world has mostly continued with blocks and files. The major storage vendors all have some kind of object storage, but most are repurposed NAS rather than dedicated CAS like the Centera.

Although much skepticism has been raised about cloud storage in the enterprise, its impact on application development cannot be denied. Indeed, the majority of developers are now focused on programming platforms that abstract both compute and storage from conventional operating systems. The next generation of applications will run in “platform as a service” environments first, and cloud storage is a key component.

Storage vendors are rapidly moving to rework their conventional systems for cloud use. Although block and file systems from 3PAR, NetApp, Isilon, Symantec, HDS, HP, and others are useful in cloud environments, unconventional CAS becomes more valuable here. This is where EMC, Mezeo, and Caringo (with Dell) shine, and why HDS bought Parascale, NetApp bought Bycast, and what Overland could do with MaxiScale. In the mean time, “gateway” products from Nasuni, Cirtas, StorSimple, Twin Strata, and Asigra are awfully interesting.

Stephen’s Stance

The storage revolution is coming, whether we in the industry are ready or not. Developers are voting with their feet, targeting cloud storage and application platforms rather than conventional filesystems. Although the market for cloud storage products is slow to develop, the cloud storage concept will eventually dominate the landscape.

It seems most likely that this revolution will decimate the storage industry as we know it today. Unable to push high-margin storage arrays into the ballooning cloud space, product vendors will see their market share eroded by service providers with no use for these expensive systems. Monolithic file and block will soldier on in the new legacy applications, but the action will inevitably slip away.

The likely winners will be those who can leverage commodity hardware for scale-out cloud storage use. The proliferation of cloud platforms will settle down, with a few gaining traction and the rest discarded. Then we will see companies like HP, Dell, and Oracle rise to lead the storage sales charts with massive volume shipments to service providers.

Disclosure: I used to work for StorageNetworks (which is now defunct) and Nirvanix.

Image credit: Barcelona Graffiti by Aeioux

© Stephen Foskett for Gestalt IT, 2010. | From CAS to Cloud: Revolutionary Storage
Read more posts categorized as All, Cloud Computing, Storage

CommVault introduces an updated version of their all-encompassing Simpana suite

CommVault is one of those enterprise IT companies that likes to go their own way. A spin-out of AT&T’s famed Bell Labs, CommVault’s Simpana software integrates many aspects of data management, from backup to e-discovery, under one umbrella. Last year, the company impressed me by adding cloud storage as a backup target equal in status to disk and traditional tape. Now the company is doing the same for storage-based snapshots, accelerating data protection for virtual machines.

Simpana 9: A Broad Update

Nearly every aspect of CommVault’s Simpana 9 is refreshed, though development and unveiling has been an ongoing process throughout the year. Following Simpana 8 by two years, the new software integrates work done integrating cloud storage and virtual machine backup over the last year or so. The “data management” portion of the product (including backup, archiving, and replication) was detailed this week, and one expects more information about the other half (“information management”) to be forthcoming.

Simpana 9 is available as of October 5 from OEMs, resellers, also direct from CommVault. Pricing is simplified this time around with CommVault using a capacity licensing model. Cost is based on the largest backup or archiving job, though the old licensing model still still available. A “circuit breaker reset” allows the system to go over the licensed level temporarily in a pinch.

Pick Your Target: Tape, Disk, Cloud, or Snapshot

There has been a long-standing back-and-forth discussion in the enterprise backup community regarding snapshots as backups. My stance has been that, while snapshots are certainly a more-effective mechanism for copying data than dump-to-tape, it takes more than data to have a useful backup. Any system worth using must schedule, manage, organize, and recover data. Therefore, although snapshot-based backup is a good idea, it wasn’t ready for prime time until it was integrated with a solid backup management system.

Elevating snapshots as a first-class backup target is exactly what CommVault is doing. Just as their earlier releases added cloud as a backup target, Simpana 9 makes snapshots just another option. Commvault recognizes that needs change over the lifecycle of a backup, and the location of data should change, too. A snapshot makes an excellent “operational recovery” target, allowing frequent copying and quick restore, but is sub-optimal for long-term archiving. Conversely, tape is a poor frequent target due to its linear nature but has proven itself for long-term offline archive.

Simpana 8 supported HDS and Dell for snapshots, now Simpana 9 adds volume leaders EMC and NetApp, along with HP, IBM, LSI, and Sun. Data is indexed and cataloged, and these are stored with the data on disk, tape, or cloud for later access. Speaking of cloud, Simpana 9 supports Amazon, Azure, Nirvanix, Rackspace, Iron Mountain, EMC Atmos, Dell DX, and HDS HCP.

You might also want to read CommVault Gives Cloud Storage A Seat At The Adult Table

VM Backup Acceleration

CommVault was demonstrating their abilities to back up machines using VMware VADP at VMworld. There, they took just 17 minutes to back up 500 machines using snapshots on an IBM XIV storage array. I imagine most of the supported platforms could do just as well.

Simpana 9 automatically discovers running guests using VMware vCenter. This allows for daily movement of machines since they are located each time and the backups are coordinated for universal, consistent coverage. No machines will be missed, either. In fact, given that many snapshots work on a block level, it would be impossible to not backup up clients sharing the same LUN.

On the back side, Simpana uses an ESXi proxy to grab snap content and catalog the virtual machines contained there. It then moves the data off to disk, tape, or cloud just like any other backup content.

CommVault is also working on Hyper-V support, using VSS rather than VADP.

Universal Dedupe

The third element of Simpana 9 is integrated source and target deduplication. Simpana’s client software deduplicates local data on its own before sending it to the server. These deduped blocks are then checked against the universal set on the server before being stored, potentially bringing a “best of both worlds” solution to the source-versus-target argument.

Migration to Simpana 9 From NetBackup and TSM

One of the long-standing criticisms of Simpana is that it was just too far-reaching for easy testing, migration, and uptake. It has been seen as an all-or-nothing proposition, requiring a commitment that is hard to win and harder to back away from in the event of trouble.

Simpana 9 addresses part of this concern with a so-called “Fast Pass” migration system that allows configuration to be pulled in from more-traditional backup products for a seamless transition. The Simpana data connector talks to NBU 6.0, 6.5, 7.0 and TSM 6.1 master server, discovers backup policies, and imports these (clients, schedule, history, etc) into the new configuration. Then automatic install software pushes CommVault agents into place and takes over operations, leaving the old system running for historical recoveries only.

Stephen’s Stance

I love the idea of promoting data from operational snapshots to tape or cloud for archiving. And I love the idea that end-users can choose the combination that fits their needs within a unified package. I’ve long been a fan of CommVault’s products, and this upgrade makes them that much more compelling. I’ll leave it to wiser minds to debate their implementation and effectiveness, but it sure looks good for me. The simplified licensing and purchasing model is welcome, too – backup is a thorny combination of pricing models.

The idea of migrating seamlessly from old to new is interesting, but I wonder how well it will work in practice. My time as a backup consultant taught me that these environments tend to be complicated, finicky, and outdated. It’s nice that CommVault can read in a TSM 6.1 configuration, but what about older versions? The same goes for NetBackup. And will it correctly handle the weird rule sets found there? But a greater issue is whether it should import the old settings at all: Perhaps migrating to a new backup system is a good time to revisit the assumptions behind your configuration!

All in all, Simpana 9 looks like a compelling update for CommVault lovers or those disaffected by their current platform. Organizations having difficulty meeting backup windows should also give it a look, since the snapshot technology should be a big help there. And anyone looking at cloud storage as a backup and archiving target should start with CommVault. Others will certainly be intrigued, but I am not sure that they will be compelled to upgrade by these features alone.

© Stephen Foskett for Gestalt IT, 2010. | CommVault Simpana 9 Takes Backup To A New Level
Read more posts categorized as Server Virtualization, Storage

Like many I wonder what this does to the whole Unified Storage message because NetApp now have two disparate storage product sets which are not integrated; I’m sure that they are briefing the integration message under NDA and if not, I’d ask why? But I’d interested to see what form the integration takes, will be it be at the tools level or will be it more fundamental integration more akin to OnTap 8.

As NetApp have announced it under the Storage Management Software product set, it appears to be the former, certainly for the short to medium term and I suspect that NetApp are going to be very wary about going after a full blown integration or at least a public statement on it after the torturous integration of Spinnaker.

The data-sheet shows a software gateway layer sitting above the OnTap filers, well I think that’s what it shows. It says that the front-end app server supports NFS/CIFS/HTTP(Restful) protocols communicating with the back-end storage via NFSv3; so theoretically, the back-end storage could be anything supporting NFSv3? But at present the data sheet actually shows a very restricted storage environment supported, namely FAS31x0 and FAS20xx and only SATA drives, so there seems to be no way of utilising your legacy storage in your StorageGrid. This is a little disappointing but no huge surprise, if EMC decide to ‘support’ third party storage with Atmos, it should be no biggie for NetApp to follow suit with StorageGrid; or perhaps vice-versa.

And as ByCast StorageGrid was resold by a number of other vendors, what is the ongoing roadmap for those customers who are running StorageGrid with different vendors storage behind it? Are these customer’s going to be expected to move to NetApp storage?

Also from the diagram in the data-sheet;

‘NetApp StorageGRID object-based storage solution brings the best of NAS and RESTful HTTP client access together’

Now I am willing accept that NetApp claim that the Filer product set are the best of NAS but to provide this ‘best of breed functionality’ with the StorageGrid product would imply a deeper level of integration than I can currently see or are they claiming that the Bycast product was actually the best NAS product out there?

Is the Filer behind the Gateway being treated as pretty a dumb-share-only Filer and not leveraging any of the OnTap features at all? Even if this is the case, it is a cute move politically as the sales-team will not see any potential Filer sales being cannibalised by this new product. A problem that I believe that EMC might have had to deal with the Atmos product set.

One of the keys will be how NetApp present the integration; will they add StorageGrid to Ops Manager? It seems to make sense that you add it at that level because Ops Manager is the preferred way of managing multiple Filers and to get the most out of StorageGrid, there will be many Filers.It also keeps it in the realms of the familiar.

If it is seen as very much a different product it makes the Unified Storage pitch a little harder as it becomes mostly-Unified-Storage product which is a bit like being a little bit unique.

So this announcement asks many more questions than it answers!

And one final comment, what is the difference between an Storage Grid and a Storage Cloud? Is it an Object Cloud or an Object Grid? Does the Object Cloud live in the Storage Grid??

© Martin for Gestalt IT, 2010. | NetApp StorageGrid – More Questions than Answers?
Read more posts categorized as Cloud Computing, Storage

Before Google could even take to the stage to announce their new “Google Storage for Developers” cloud storage offering in their I/O conference keynote, Amazon hit back with a new low-cost “Reduced Redundancy Storage” option for S3. The titans are at war, and cloud storage is the new battle ground. But what was really announced? And should you care?

Defining Durability

Amazon begins their RRS rollout with an apt discussion of durability. To quote:

Let’s define durability (with respect to an object stored in S3) as the probability that the object will remain intact and accessible after a period of one year. 100% durability would mean that there’s no possible way for the object to be lost, 90% durability would mean that there’s a 1-in-10 chance, and so forth.

I like this turn of phrase, and encourage other storage vendors to consider it as well. It’s especially appropriate for a discussion of public cloud storage, since the traditional SLA (“availability”) relies on many systems between data and application. There are many failure scenarios that can result in a loss of availability without any risk of data loss, and Amazon is wise to make the distinction here.

Also see Amazon CTO Werner Vogels’ discussion of S3 durability

Amazon claims that “the durability of an object stored in Amazon S3 is 99.999999999%. If you store 10,000 objects with us, on average we may lose one of them every 10 million years or so. This storage is designed in such a way that we can sustain the concurrent loss of data in two separate storage facilities.” That’s eleven nines – a Very Large Number. RRS changes this: “Objects stored using RRS have a durability of 99.99%, or four 9’s. If you store 10,000 objects with us, on average we may lose one of them every year. RRS is designed to sustain the loss of data in a single facility.” Eleven nines remains the default for data written to S3, but an API call can set the REDUCED_REDUNDANCY flag on demand.

The idea is that some data can be regenerated or simply requires less “durability”. The value for the customer is reduced cost: RRS is 1/3 cheaper than regular S3 storage at $0.10 per GB per month to start, a substantial discount designed to counter Google’s forthcoming offering.

Stephen’s Stance

Amazon S3 RRS is a risky move, but the company handled the announcement exactly right. Rather than focusing on the reduction in cost, they turned the spotlight to the built-in reliability of their existing offering. Yet it’s a fact that not all data is equal, and much could be stored with less durability and at a lower price, and Amazon’s new option enables this. The company also made the right move by making reduced-reliability an opt-in option. RRS is a good move for Amazon, and I expect competitors (Google, Rackspace, etc) to follow.

© Stephen Foskett for Gestalt IT, 2010. | Amazon Redefines Durability with Reduced Reliability Storage (RRS)
Read more posts categorized as Cloud Computing, Storage

This product is targeted directly at the SME / SOHO / Branch Office market. It offers a lot of storage for cheap money and can be dynamically expanded by swapping drives using their BeyondRAID technology. Since there is no integration with Microsoft’s Active Directory (it supports workgroup style authentication for CIFS shares), it won’t appeal to big companies except for niche applications.

No iSCSI

Note that the DroboFS doesn’t support iSCSI (and that the DroboShare does). I suspect that iSCSI connectivity will be released in a future version, they just want to put the unit into the market.

And while the unit probably can support VMware volumes, there are no plans to have it certified which is apparently important to some virtualisation people ( Rant:why pay the protection money for something that really should work ? Standards are meant to be standards.)

Caption Text.(Click for a full size image)

Best Features

The Drobo disk system makes having a lot of storage easy to manage. If you don’t have the time or the knowledge on managing a RAID array, and you have some extra money, then this unit has some great features. The best feature is the ability to use HDD’s that are of different size. The Drobo BeyondRAID software will automatically adapt itself to provide redundancy of drives and build a single drive.

I have a Drobo (( disclosure: I received a small discount to purchase the unit as part of Tech Field Day )) that look like a single 16TB drive. However, I only have three 1.5TB drives installed. I can insert or swap drives as need to get to 16TB in the future. Since I often have spare drives around the place, I am often removing small drives and adding larger drives. When I do this, the system automatically adapts itself. I don’t have to have three / four / five identical drives. I don’t have to move the data somewhere else while I am upgrading the array.

In my lab, this means I can grow my storage from the current 2.5TB up to 16TB without much hassle. That works for me, I don’t have a lot of time to muck around with the Open Source NAS software and hand built computers.

Jumbo Frames

The system supports 9000 byte ‘jumbo’ frames. It’s not vital for a networking lab, but it’s a good to have feature for higher performance when cloning drives in VMware.

Cloudy – it’s always cloudy

Drobo have partnered with someone called “Oxygen Cloud” (no, I’ve never heard of them either) to deliver internet storage backup of the DroboFS. This means that your DroboFS can automatically replicate it’s data over the Internet to an external storage company.

Caption Text.(Click for a full size image)

This has been provided using an API. Drobo will be releasing details and support for developers who can create their own plugins using this API for the Drobo. For corporates, this offers some intriguing possibilities. You could easily create a content distribution network for remote sites. Imagine a DroboFS at your remote sites, with say, four 1TB SATA drives acting as a CIFS share for the site. All data is replicated according to a schedule via the Internet or via the internal WAN . The drive capacity can be easily increased to 16TB by buying cheap SATA drives without requiring site visits. This type of private pseudo-cloud technology is seeing a lot of deployment activity as corporates adapt available technologies to suit their actual needs.

The EtherealMind Position

This version of the Drobo has the possibility of being a good home storage system for lab environments for VMware ESX / vSphere (which is important for many networking people these days). It does support CIFS and NFS, but without iSCSI I can’t go the extra mile to buy it for my lab compared to my existing Drobo unit.

If you haven’t got one yet, then wait for the iSCSI to be released would be my advice.

© Etherealmind for Gestalt IT, 2010. | DroboFS: Gigabit Ethernet, Serverless and Cloudy
Read more posts categorized as Featured, Storage

My presence at Storage Expo NL may have been cursed, but my presentation went off without a hitch thanks to the creativity and flexibility of the VNU staff and Expo volunteers like Ilja Coolen. In my session, I talked about the shortcomings of traditional tiered storage as a way to advance the Noble Goals of Storage Management and proposed a solution: Although merely using different disk types will never deliver the goods, adding flash and cloud to an integrated, automated solution will be truly revolutionary. I look forward to the day when all of today’s buzz-worthy technologies (flash, cloud, thin provisioning, automated tiering, post-RAID) are mixed together to form a really revolutionary storage system.

I went ahead and recorded the entire presentation and posted it on Vimeo for anyone to see. But I just realized I never posted it here to my blog. So without further ado, I give you “Extreme Tiered Storage: Flash, Disk, and Cloud!”

Stephen Foskett on Extreme Tiered Storage from Stephen Foskett on Vimeo.

By the way, in case you were wondering, I used BoinxTV to create this video.

© Stephen Foskett for Gestalt IT, 2010. | Extreme Tiered Storage: Flash, Disk, and Cloud
Read more posts categorized as Cloud Computing, Storage