In 2020, KPMG, the accounting giant, accidentally lost chat records of 145,000 users. The reason, a human error. KPMG explained the nature of this error in its memo, saying that a mistake while changing the retention policy of one Microsoft Teams user accidentally erased the histories of all other users.
That the incident impacted users down the chain all around the world is something of an understatement. But it revealed a far more disturbing fact – the truth behind data recoverability of SaaS solutions. The reason the chats vanished in a whiff is because Teams’ chat data is not recoverable, making the loss permanent.
This may be a punch to that gut for those it is news, but software-as-a-service vendors do not keep backup of users’ data. This is true for at least all the top online services in use in business today – Office 365, Salesforce, G Suite, Jira, Dropbox, and so on.
A Carefully Preserved Misbelief
Users operate off the assumption that data protection is built into SaaS applications, and it is not all their delusion. This myth has been standardized through years of subliminal messaging that implies that user data is backed up somewhere safe.
“But so many companies that are creating data that’s valuable to other companies are not being backed up in any way that I would recognize as a backup,” said author, backup guru, and long-time Field Day delegate, W. Curtis Preston, a.k.a. Mr. Backup, during a Gestalt IT Tech Talk at the recent Storage Field Day event in California.
Backing up data involves duplicating and storing it in a different location than the one in which the application is running. ERP and CRM systems, emails, productivity tools, all follow a sloppy policy instead. Most of them rely on in-application rollback. If private data gets deleted, a record is saved in the app’s history that lets you recall it.
Sounds like backup, right? Except it’s not. “It’s really a nice convenience copy, and it will probably recover you from many of the things that you do that cause damage,” says Preston.
This copy is saved in the same database as the application. Therefore, it is no more protected than the original copy. “You can’t store everything all in one place and call that back up,” argues Preston.
Finding the Facts in the Fine Prints
One way to find out if or what backup capabilities a provider has, is to read the service agreement. Preston says to skim the fine prints for phrases like “backup and recovery” or something similar. The odds of you finding anything close is however, incredibly low, he says.
But let’s assume for a moment that a vendor does have “backup and recovery” mentioned in the contract. Is your personal data safe with them?
Unfortunately, a lot of the times, features that are passed off as backup are really not backup at all. “These fancy features that feel like backup, its really important to understand that they’re not,” Preston cautions.
Other times, backup capabilities within SaaS application are so limited that they do not meet the necessary security standards.
Threat Meets Vulnerability
So where does this leave companies that wish to prioritize data protection for their clients but are shackled by the low capabilities, or the lack thereof?
Data protection is at a precarious point. It’s already happening, reminds Preston. “Companies are losing data but I just see that ballooning.”
The recent rise of ransomware attacks is a relevant conversation this. “If a ransomware attacker gets in and deletes your account or just obliterates it in such a way that you just can’t understand it,” it makes recovery all the harder.
In those situations, the in-application rollback or recycle bin feature is not likely to come in handy. For one, the duplicate version of lost data, if found in the recycle bin, is not going to be in a format that’s recognizable, making one-click restores a distant dream.
Many vendors have retention policies that offer users the ability to download or export data from the applications, but putting back millions of records is another worry altogether owing to format incompatibility. The time needed to recover jumps to weeks and months.
“It’s an e-discovery tool not a backup and recovery tool. If you set up a retention policy that says that your all emails get kept for nine months, it means they won’t be deleted. They’ll be stored during that retention period, but again you will get them back in a mess.”
The KPMG incident serves as a jolting reminder of how imperfect retention policies really are.
Hackers know better than to leave the data in the recycle bin. In modern attacks, they go into backup systems of all manner, wiping them out before exiting, making sure that there is no easy way for the victim to recover the information.
“They’re specifically going in and disabling, for example, setting the number of versions to one and then encrypting everything.” This leaves victims no good copies to recover.
A Way Around
Vendors’ argue that not all data requires backing up. “SaaS products create a lot of data that is not mission critical. There’s a lot of it that is transient,” agrees Preston.
Using backup tools for the important data when available takes users beyond SaaS vendors’ de facto backup policies. “Why would this one part of IT be any different than the rest of it,” Preston reminds.
To encourage investing in backup, vendors need to put in work to make backup less of a hassle and more of a point-and-click affair, so that with a push of a button, versions of data get sent to a user-controlled storage in another location.
To take it a step further, a uniform data protection approach for both on-prem and cloud must be accessible to organizations using either or hybrid infrastructures to centrally backup their data, no matter where it is.
The additional costs of backup may keep companies wanting to walk the plank budget-wise with the available SaaS recovery features, but Preston urges them to think that if all that data went away one day in an incident like KPMG’s, how would that impact the company, and how much money it would cost them. That number, he concludes, will justify the additional backup and recovery spend.
For more, be sure to check out the Gestalt IT Tech Talk from the recent Storage Field Day event. For questions related to backup, head over to Preston’s blog, The Backup Wrap-Up.
Connect with W. Curtis Preston
W, Curtis Preston is a backup and recovery systems expert and the host of the Backup Wrap-up Podcast. You can connect with Curtis on LinkedIn, listen to his podcast, and read more about backup on his website Backup Central.
