VMware VMSafe – Are there any actual products yet?

I was doing some work out of hours the other night on my employers Virtual Infrastructure when bang on time the little red triangles started popping up against certain ESX hosts in vCenter.  Why you ask? well it’s AV scanning time on our VM’s of course, or the Sophos summit as we affectionately call it due to its uncanny resemblance to a mountain range when you look at the CPU performance stats in vCenter.

It got me thinking, has any one vendor actually got a product out there utilising the VMSafe API that could help me rid our virtual infrastructure of this problem?

My first stop was of course the main VMSafe page where I did find a large list of official partners who are working on developing products to utilise the VMSafe API. The pleasing thing to see was that there are plenty of mainstream security vendors taking part.  However I’ve still to see any of them releasing a product to market that actually utilises VMSafe.

Earlier this year in Glasgow I heard Mcafee talk about VMSafe as part of the VMware vSphere launch road show.  They talked about building a vApp that could sit in your Virtual Infrastructure and take care of AV scanning with the aim of reducing the CPU overhead that AV scanning introduces. I did a little trawl of the web and couldn’t find anything official, I did however find the following forum post (quoted below) which is definitely the unofficial line.

Virus Scan for Offline images is available, which uses VMSafe APIs to scan offline disks accessed via ESX

Nothing is currently road mapped for on-access scanning – no AV vendor has this technology available (or even road mapped as far as I’m aware) yet.

I did a bit more digging on this “scan offline disks” comment and found a recent article by VMware’s Richard Garsthagen.  This article reveals that a piece of software called the VMware Virtual Disk Development Kit (VDDK) can be used to conduct an offline scan of disks attached to powered on or off virtual machines (quoted below).

VMware VDDK (also being seen as part of the VMsafe initiative, but has been available for longer). The VDDK is an disk API, that allows other programs to access a virtual machine’s hard disk like the VMware Consolidated Backup solution does. It does not matter is the VM is powered on of off, but a disk can just be ‘extra’ mounted to another virtual machine that for instance runs a virus scanner. The clear downside of VDDK is that nothing is real time.

Surely this would rid me of my daily scheduled Sophos summit, wouldn’t it? Think of a hypothetical scenario where you have a VDI setup with 1000 windows XP VM’s,  imagine the strain put on your ESX clusters by 1000 machines kicking off a scheduled daily AV Scan. Would an appliance that could offline scan disks reduce the strain? Well thinking about it, possibly not.  It would still have to conduct a scan of 1000+ virtual disks, only this time it wouldn’t have nearly as many CPU cycles available to churn through the work. All it would have is the resources assigned to the vApp which is likely to be completely inadequate for such a large task. With this in mind it’s likely that it would probably take a large amount of time to complete.  It could even take longer than a day which wouldn’t be much use for a daily AV scan. I’m sure some companies would rather suffer the ESX CPU resource pain point as opposed to sacrificing security through ineffective or untimely AV Scanning.

Richard’s article along with the solutions tab on the VMSafe webpage did however reveal that a couple of products that use VMSafe have made it to market.  One is called vTrust from Reflex Systems which appears to be a multi faceted application, which according to their site provides dynamic policy enforcement and management, virtual segmentation, virtual quarantine and virtual networking policies.  The other application is a hypervisor based firewall appliance from Altor that supports virtual segmentation and claims to provide better throughput by using the Fast Path element of the VMSafe API.

So it would appear on the surface that progress has been slow.  To only find two VMware certified appliances in the market place was, I have to admit, quite a surprise!  It looks like it’s going to be a while before we see VMsafe being fully utilised by vendors, even then we will  have those wary individuals who will never quite be convinced.

Neil Macdonald of Gartner makes a good point about the potential for VMSafe appliances to introduce possible security vulnerabilities at a lower level in the infrastructure.

If I’m responsible for VM security, I’ll consider it after the APIs ship, after the vendors finally ship their VMSafe-enabled solutions, after I’ve got a level of comfort that these VMSafe-enabled security solutions don’t in of themselves introduce new security vulnerabilities

Edward L Haletky who is very much focused on virtualisation security also makes a good point about low level vulnerabilities and the interaction of multiple VMSafe appliances.

I fully expect VMware to not only ensure the VMSafe fastpath drivers do nothing harmful to the virtual environment, but also address interaction issues between multiple VMSafe fastpath drivers. In addition, I would like such reports made available to satisfy auditing requirements.

So was VMSafe simply something to bolster the vSphere marketing launch,  an announcement made before it should have been?  Usually VMware are quite good at keeping these kind of things under wraps and releasing them when they are a little more mature and ready for use in real world scenarios.  Now I don’t know what work was done with partners in advance but I would have liked to have seen a couple of the major security vendors releasing appliances at the same time as VMSafe was announced.  For me that certainly would have installed a little more confidence in VMSafe than writing this article has.

If anyone out there is writing appliances utilising the VMSafe API and wants to comment, please do.  I would love to hear some news from the front line as to what is being developed, where it will be applied and when we can expect to see it.

  • MB-NS

    Hello,

    I think Trend Microo already does this :
    http://us.trendmicro.com/us/solutions/enterpris

    From what I gathered, today VMsage API allows detection and scanning (and reporting) from outside the VM, but an agent inside the VM is still required to actually quarantine or delete the file. The scanning, however, is centralized on the VMSafe Appliance.

    If one look at the PDF on the Trend page, the “scanning agent” likely the VMSafe appliance, and the “real-time agent” should be the software which acts upon the scan result.

  • http://www.virtualpro.co.uk Virtualpro

    Hey, Thanks for the feedback. This is an interesting topic and one I'd really like to see VMware and vendors succeed with.

    I had read that Trend Micro were pretty close to having a product ready for release. I had a quick read of the link you posted and as far as I can see the only mention of a “real-time agent” is a red dot on a diagram. The rest of the document talks about the scanning agent, offloading the resource utilisation of a scan and adding another layer of immunity by running the scan from a seperate VM. This and it's ability to include dormant machines seems to suggest the VDDK mounting of virtual disks is in use for this part of the operation. Would the “real-time agent” simply be a traditional AV client dealing with on-access scanning and doing the bidding of the VMsafe vApp, i.e. the final part of the operation, the actual quarantine / removal? That might work, the only bit I would still need to clarify is how the Trend Micro vApp communicates with the on machine agent.

    Suppose one other question that doesn't get answered is how does the trend micro vApp deal with a virus on a dormant machine if there is no real time agent.

    Sometimes delving deeper keeps turning up more questions :o )

  • Jay

    Why on earth would you want to scan desktop virtual machines (or any desktop / laptop systems) on a daily basis? The value of this is pretty much zero in most situations (there are some very specific exceptions to this rule, such as a part of a cleanup after an outbreak); the resource requirement is huge, regardless of whether its physical or virtual.

  • jay

    Take a closer look, this is fud. They have on-demand scanning using vmsafe but absolutely NO on-access scanning using vmware.

    What they do have is an ips/ids appliance (from third brigade aquisition) using vmsafe and a way of writing that is (IMHO) deliberately misleading.

    Note the “real-time agent” is the standard AV scanner you install on all types of systems, e,.g. sep11, virusscan enterprise etc.

  • Lance Reddick

    The problem is VMSAFE-Net has “restrictions” when dealing with agentless scanning. The problem is what to do with a file in a virtual environment after a piece of Malware has been detected. VMSAFE-Net currently allows you to “alert” but if you try and make changes to the file (cure, delete, quarantine) there is a high probability of corruption in the virtual machine.

    Stay tuned for some announcements soon from Trend Micro in regards to agentless scanning. The approach will be quite unique in the market.

  • Lance Reddick

    The problem is VMSAFE-Net has “restrictions” when dealing with agentless scanning. The problem is what to do with a file in a virtual environment after a piece of Malware has been detected. VMSAFE-Net currently allows you to “alert” but if you try and make changes to the file (cure, delete, quarantine) there is a high probability of corruption in the virtual machine.

    Stay tuned for some announcements soon from Trend Micro in regards to agentless scanning. The approach will be quite unique in the market.