Your vSphere VMs may be running from snapshots even though you didn’t create them. That is, if a scheduled job which auto creates snapshots runs into a datastore with insufficient free space. In this scenario a special Consolidate Helper snapshot will be created.
The following screenshot shows the mysterious snapshot as I found it on my lab domain controller
In my case, the scheduled job that needs to create and then normally commit the snapshot is a Veeam Backup and Replication job. At some point in the life of my lab I did run out of space, and although it’s not an issue anymore, the snapshot still exists even when the jobs aren’t running! I was a bit surprised that backup jobs could even complete with this snapshot in place.
VMware KB article 1003302 explains that the Consolidate Helper snapshot is actually created when trying to commit (delete all) a snapshot.
“If you try to initiate a Delete All snapshot for a virtual machine using Snapshot Manager, and if that virtual machine is on a datastore that does not have sufficient space for the snapshot, the following message displays in VMware Infrastructure (VI) Client:
msg.hbacommon.outofspace: there is no more space for the redo log of <VMname>-0000xx.vmdk.
You are given the option to abort or retry.
- If you choose Abort, the virtual machine is powered off, the snapshot is aborted, and a Consolidate Helper snapshot is created. The Snapshot Manager UI displays that Consolidate Helper snapshot. You can delete the Consolidate Helper snapshot after you have made space available.
- If you click Retry, the Snapshot Manager returns to Consolidate Helper snapshot mode unless you have made more disk space available.”
Personally, I never saw the VI/vSphere Client message.
The KB Article suggests deleting the snapshot when your datastore has enough free space and even offers instructions for extending your datastore.
Luckily, my snapshot was only consuming 29.14 KB of space so it went under the radar for about 8 days. I’ve blogged before about how unnoticed ESX snapshots can be dangerous.
Clicking the Delete All button from the Snapshot Manager Window (previous screenshot) after making sure there were no active backup jobs removed my snapshot painlessly this time.
This apparently is one of a few new Changes to the ESX Snapshot mechanism.
