Offline VMware disk shrink?

After playing around with different VMware Appliances and VMware Server, I noticed that the disk space consumed by the virtual machines were rapidly increasing. It's a well known fact that when a vmdk is defined you set a max disk size, and you can chose not to allocate all the disk space at once, and it'll expand the virtual disk as needed when you run the VM. All thats fine and dandy, and it helps keeping the filesize of distributed VMs low. But, what happens when you fill a virtual disk and then delete files from it again? Does the virtual disk shrink in size? No, it doesn't.

There is a perfectly logical explanation to that, as explained by rtfm-ed.co.uk

Well, when Windows deletes a file, it doesn’t actually delete the data in the disk; it just deletes the references/pointers in the file allocation table. So when ESX is exporting a VMDK and is looking at the raw disk, it’s seeing values that aren’t empty (non-zero), and exports them as such. The result is more disk space is used and takes longer to export the disk.

VMware Workstation as well as Server and ESX server all include a tool to "shrink" a disk, but VMware Player doesn't. This poses a problem for users that use VMware Appliances to test software and play around with it. In many cases you have to re-download the appliance, delete the old one, and start a new instance of it to reclaim disk space used inside the VM while testing.

I've been looking for an "offline" VMware disk shrink utility, but I'm unable to find one. The closest I came was the VMware Command Line Tools Package, but the vmshrink command included there is supposed to be run inside the VM and not applied directly to the .vmdk files when the VM itself isn't running. This means that you either need to get the file, for that particular OS, into the VM, or it has to be prepackaged by the virtual machine developers.

Another option was to use a modified sdelete utility, as outlined by "Shrinking Virtual Disks ... with the Sdelete Tool". That seems like a viable option, but it does mean you do need to have a compiler handy and do the modifications yourself. In most cases, thats not possible.

The third option I found, was the commercially available Invirtus VM Manager which does seem to fit the bill. At $39.00 for the personal edition, it's price isn't that high but still probably higher than what people who use VMware Appliances for testing would want to spend.

So, why isn't there a simple command line utility available to do offline shrinking of VMware virtual disks?
Does anyone know of such a tool hidden somewhere that I'm just not aware of?

August 16, 2006 at 2:14pm | 26 Comments
Tagged: and

26 Comments so far

  1. Jason Stevenson, on August 22, 2006 at 9:03pm, said:

    I totally agree. My department uses VMWare dev machines. We have 2003 and 2005 Visual Studio VMs. They grow and grow. But we didn't create the VMs so we run them with VMPlayer.

    If you ever find a good inexpensive solution please let me know.

    Jason

    Edit Comment

  2. h0bbel, on August 22, 2006 at 9:38pm, said:

    Well, an inexpensive solution would be to install VMware Server somewhere, and use that to do "inline" shrinking of the VMs. But, I would love a command line tool, that way it could even be scheduled to run at night when the VMs aren't in use.

    Edit Comment

  3. Christopher Ottley, on September 10, 2006 at 3:03pm, said:

    I believe that the VM Server (on Windows anyway) has an executable file vmware-vdiskmanager.exe which can be used to shrink virtual disks. It looks like you have to prepare the disk for shrinking before you do it anyway. For a windows guest mount, prepare, unmount and then shrink using the vmware-vdiskmanager. For a linux guest (without X) you have to fill the partition with a file that's all zero inside the guest (mounting doesn't work in Windows - ext3, so that's how it's prepared), delete the file and then in the host run the shrink using the vmware-vdiskmanager.

    Edit Comment

  4. lazyllama, on September 20, 2006 at 6:01pm, said:

    You could try 'vmkfstool -i vmdkFile -d thin vmdkFile2'.
    That should clone the disk as a "thin" disk so without allocating all the space. You can then get rid of the original vmdkFile and use vmdkFile2 in its place.

    Messy but it ought to work, I think

    Edit Comment

  5. h0bbel, on September 20, 2006 at 6:08pm, said:

    Not really, as I'm looking for a tool end users can use after spending time using the Gallery Appliance.

    Edit Comment

  6. Talon, on October 7, 2006 at 5:30pm, said:

    You could try VMware DiskMount GUI at http://petruska.stardock.net/software/VMware.html#DiskMan

    AFAIK it's just a GUI to VMware's diskmanager.

    Edit Comment

  7. Talon, on October 7, 2006 at 5:32pm, said:

    You could try VMware DiskManager GUI at http://petruska.stardock.net/software/VMware.html#DiskMan.

    Edit Comment

  8. Ralf Mimoun, on October 22, 2006 at 11:26am, said:

    Hi,

    it seems that the most important part of the Invirtus tool is that little program that fills the disk with a huge file, containing only 0. That's not a new technology, it's been done since decades to help disk imaging software.

    Anybody who is interested in such a tool can download ZapDisk from http://stuff.rad-on.de/ZapDisk.zip. It is only 32 kb big, and it's very simple to use. "ZapDisk C:" will create a huge file named (~zapdisk~.~fill~" in the root directory of the C: drive and delete it. No rocket science involved. Just run it on all your drives in the virtual machine and start the shrink process.

    I wrote it in about 15 minutes in Delphi, it's free, and it works. Ok, at least for WinNT, Win2k and WinXP virtual machines with NTFS formatted drives.

    Edit Comment

  9. h0bbel, on October 22, 2006 at 2:54pm, said:

    That util still doesn't adress the issue I put forward in my original post, nor will it help for all the community virtual appliances out there, as none of then use Windows (nor can they due to licensing issues).

    Edit Comment

  10. Ralf Mimoun, on October 22, 2006 at 3:19pm, said:

    I thought that I stated clearly what that little tool can do - and that it can't do much more :-) All it can do (and everybody has to verify that on his own) is help to get smaller WinNT based VM images. Starting with VMs running VS2003 and VS2005, that seems to fit at least partially.

    That's all a little bit strange, the company Jason Stevenson is working for is probably in the software development business, if as a business or for internal needs. They probably have Visual Studio licenses (there is not _that_ much that you can do as a professional with the free version). But they don't buy VMWare Workstation. YGWYPF.

    Edit Comment

  11. h0bbel, on November 24, 2006 at 8:59am, said:

    That is indeed very interesting, and yes that would be the answer to my initial question. In this particular case, the Gallery Appliance, we could create a script that zerospaces the disk and then provide documentation on how to run vmware-vdiskmanager to shrink it.

    Thanks a lot, I'll look into that for the next version of the appliance.

    Edit Comment

  12. Shrinking VMDK files - h0bbel, on January 2, 2007 at 1:18am, said:

    [...] distribute between the different team members. So, I decided to try out the technique outlined by a random comment by “Guest” on a previous [...]

    Edit Comment

  13. Jon, on February 8, 2007 at 6:15pm, said:

    The suggestion that on *nix based guest of using cat /dev/zero etc.. Worked like a charm :)

    I managed to shave about 200MB off my VM image.

    Thanks

    Edit Comment

  14. h0bbel, on February 8, 2007 at 7:40pm, said:

    Yep, works great as mentioned in http://h0bbel.p0ggel.org/2007/01/02/shrinking-vmdk-files/

    Edit Comment

  15. coComment - Site comments by h0bbel, on March 16, 2007 at 10:32am, said:

    View this article on its blog

    Edit Comment

  16. me, on May 30, 2007 at 12:40pm, said:

    Only works on Windows it seems:

    # /usr/bin/vmware-vdiskmanager
    VMware Virtual Disk Manager - build 44356.
    Usage: vmware-vdiskmanager OPTIONS diskName
    Offline disk manipulation utility
    Options:
    -c : create disk; need to specify other create options
    -d : defragment the specified virtual disk
    -n : rename the specified virtual disk; need to
    specify destination disk-name
    -q : do not log messages
    -r : convert the specified disk; need to specify
    destination disk-type
    -x : expand the disk to the specified capacity

    Additional options for create and convert:
    -a : (for use with -c only) adapter type (ide, buslogic or lsilogic)
    -s : capacity of the virtual disk
    -t : disk type id

    Disk types:
    0 : single growable virtual disk
    1 : growable virtual disk split in 2Gb files
    2 : preallocated virtual disk
    3 : preallocated virtual disk split in 2Gb files

    The capacity can be specified in sectors, Kb, Mb or Gb.
    The acceptable ranges:
    ide adapter : [100.0Mb, 950.0Gb]
    scsi adapter: [100.0Mb, 950.0Gb]
    ex 1: vmware-vdiskmanager -c -s 850Mb -a ide -t 0 myIdeDisk.vmdk
    ex 2: vmware-vdiskmanager -d myDisk.vmdk
    ex 3: vmware-vdiskmanager -r sourceDisk.vmdk -t 0 destinationDisk.vmdk
    ex 4: vmware-vdiskmanager -x 36Gb myDisk.vmdk
    ex 5: vmware-vdiskmanager -n sourceName.vmdk destinationName.vmdk

    Edit Comment

  17. zargon, on May 31, 2007 at 5:44pm, said:

    I have been looking to do the same thing since I went too large with the virtual disk of a set size.

    The host machines are running linux, which complicates matters since vmware-mount doesn't work and vmware-vdiskmanager doesn't have the shrink option (-k).

    After polking around the web for a bit, which is how a stumbled across this site. I found the best solution for my situation. I am using vmware-vdiskmanager to create a new virtual disk from the old one and forcing the size smaller.

    vmware-vdiskmanager -r virtual_disk.vmdk -t 2 -s 4Gb virtual_disk_temp.vmdk

    I then rename the old one, rename the new one, boot the system to test and then delete the old one. Done and pretty simple, I think it is less mess than the other method, even on a Windows based host.

    Edit Comment

  18. Guggi, on June 20, 2007 at 4:51pm, said:

    Faced the same problem and found this site, http://chitchat.at.infoseek.co.jp/vmware/vmtools.html!
    Very useful tool to automate online shrink in guest.

    Edit Comment

  19. jziemian, on June 22, 2007 at 7:04pm, said:

    Anybody find a Windows (XP/Vista) zero fill utility?

    Edit Comment

  20. VirtuaMag.net, on July 9, 2007 at 9:51pm, said:

    [...] Offline VMware disk shrink ? [...]

    Edit Comment

  21. VMware, on July 27, 2007 at 1:36am, said:

    From the SDelete site as hosted by Microsoft (http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx):

    Usage: sdelete [-p passes] [-s] [-q]
    sdelete [-p passes] [-z|-c] [drive letter]
    -c
    Zero free space (good for virtual disk optimization)
    ...

    Just set the -c flag and you can do it w/ the latest sdelete. No need to find other obscure utils.

    Edit Comment

  22. visitingalien, on August 2, 2007 at 3:27pm, said:

    I came here to try and find out something useful about VMWare but couldn't because of your REALLY STUPID colour scheme. I won't be back and don't bother emailing me.

    Edit Comment

  23. shrink virtual disk wmware - Eniro, on August 10, 2007 at 1:39pm, said:

    [...] Offline VMware disk shrink? - h0bbel [...]

    Edit Comment

  24. synarchy, on April 27, 2008 at 4:21am, said:

    Thank you Zargon.That's just what I needed.

    Edit Comment

  25. Anonymous, on August 7, 2009 at 3:54pm, said:

    You guys really have to get a grip on what you are trying to do. There are 2 ways to shrink a disk.

    One is the -x flag. This literally shrinks the disk using vmware-vdiskmanager.exe, taking the disk from a 16G to a 10G disk or whatever.

    The next is the -k flag. This removes the the extra file space from the image and creates more space available on your drive without removing space from your VM image.

    Because remeber that even if later you enlarge the C drive on a machine, you can never expand it because MS won't let you do it. Even if it is dynamic.

    If you are interested in free then use VMServer, the command line util is there and is much better then player.

    Edit Comment

  26. h0bbel, on August 8, 2009 at 11:32am, said:

    @Anonymous: You can expand the C:\ drive, it just takes some work as outlined in Howto: Expand VM boot partition.

    Edit Comment

Leave a Comment?


« VMware Appliance Contest Winners Announced  —  Gallery 2.1.2 Security Fix Release »

Recent Comments