After playing around with different VMware Appliances and VMware Server, I noticed that the disk space consumed by the virtual machines were rapidly increasing. It's a well known fact that when a vmdk is defined you set a max disk size, and you can chose not to allocate all the disk space at once, and it'll expand the virtual disk as needed when you run the VM. All thats fine and dandy, and it helps keeping the filesize of distributed VMs low. But, what happens when you fill a virtual disk and then delete files from it again? Does the virtual disk shrink in size? No, it doesn't.
There is a perfectly logical explanation to that, as explained by rtfm-ed.co.uk
Well, when Windows deletes a file, it doesn’t actually delete the data in the disk; it just deletes the references/pointers in the file allocation table. So when ESX is exporting a VMDK and is looking at the raw disk, it’s seeing values that aren’t empty (non-zero), and exports them as such. The result is more disk space is used and takes longer to export the disk.
VMware Workstation
I've been looking for an "offline" VMware disk shrink utility, but I'm unable to find one. The closest I came was the VMware Command Line Tools Package, but the vmshrink command included there is supposed to be run inside the VM and not applied directly to the .vmdk files when the VM itself isn't running. This means that you either need to get the file, for that particular OS, into the VM, or it has to be prepackaged by the virtual machine developers.
Another option was to use a modified sdelete utility, as outlined by "Shrinking Virtual Disks ... with the Sdelete Tool". That seems like a viable option, but it does mean you do need to have a compiler handy and do the modifications yourself. In most cases, thats not possible.
The third option I found, was the commercially available Invirtus VM Manager which does seem to fit the bill. At $39.00 for the personal edition, it's price isn't that high but still probably higher than what people who use VMware Appliances for testing would want to spend.
So, why isn't there a simple command line utility available to do offline shrinking of VMware virtual disks?
Does anyone know of such a tool hidden somewhere that I'm just not aware of?
26 Comments so far
Jason Stevenson, on August 22, 2006 at 9:03pm, said:
I totally agree. My department uses VMWare dev machines. We have 2003 and 2005 Visual Studio VMs. They grow and grow. But we didn't create the VMs so we run them with VMPlayer.
If you ever find a good inexpensive solution please let me know.
Jason
Edit Comment
h0bbel, on August 22, 2006 at 9:38pm, said:
Well, an inexpensive solution would be to install VMware Server somewhere, and use that to do "inline" shrinking of the VMs. But, I would love a command line tool, that way it could even be scheduled to run at night when the VMs aren't in use.
Edit Comment
Christopher Ottley, on September 10, 2006 at 3:03pm, said:
I believe that the VM Server (on Windows anyway) has an executable file vmware-vdiskmanager.exe which can be used to shrink virtual disks. It looks like you have to prepare the disk for shrinking before you do it anyway. For a windows guest mount, prepare, unmount and then shrink using the vmware-vdiskmanager. For a linux guest (without X) you have to fill the partition with a file that's all zero inside the guest (mounting doesn't work in Windows - ext3, so that's how it's prepared), delete the file and then in the host run the shrink using the vmware-vdiskmanager.
Edit Comment
lazyllama, on September 20, 2006 at 6:01pm, said:
You could try 'vmkfstool -i vmdkFile -d thin vmdkFile2'.
That should clone the disk as a "thin" disk so without allocating all the space. You can then get rid of the original vmdkFile and use vmdkFile2 in its place.
Messy but it ought to work, I think
Edit Comment
h0bbel, on September 20, 2006 at 6:08pm, said:
Not really, as I'm looking for a tool end users can use after spending time using the Gallery Appliance.
Edit Comment
Talon, on October 7, 2006 at 5:30pm, said:
You could try VMware DiskMount GUI at http://petruska.stardock.net/software/VMware.html#DiskMan
AFAIK it's just a GUI to VMware's diskmanager.
Edit Comment
Talon, on October 7, 2006 at 5:32pm, said:
You could try VMware DiskManager GUI at http://petruska.stardock.net/software/VMware.html#DiskMan.
Edit Comment
Ralf Mimoun, on October 22, 2006 at 11:26am, said:
Hi,
it seems that the most important part of the Invirtus tool is that little program that fills the disk with a huge file, containing only 0. That's not a new technology, it's been done since decades to help disk imaging software.
Anybody who is interested in such a tool can download ZapDisk from http://stuff.rad-on.de/ZapDisk.zip. It is only 32 kb big, and it's very simple to use. "ZapDisk C:" will create a huge file named (~zapdisk~.~fill~" in the root directory of the C: drive and delete it. No rocket science involved. Just run it on all your drives in the virtual machine and start the shrink process.
I wrote it in about 15 minutes in Delphi, it's free, and it works. Ok, at least for WinNT, Win2k and WinXP virtual machines with NTFS formatted drives.
Edit Comment
h0bbel, on October 22, 2006 at 2:54pm, said:
That util still doesn't adress the issue I put forward in my original post, nor will it help for all the community virtual appliances out there, as none of then use Windows (nor can they due to licensing issues).
Edit Comment
Ralf Mimoun, on October 22, 2006 at 3:19pm, said:
I thought that I stated clearly what that little tool can do - and that it can't do much more :-) All it can do (and everybody has to verify that on his own) is help to get smaller WinNT based VM images. Starting with VMs running VS2003 and VS2005, that seems to fit at least partially.
That's all a little bit strange, the company Jason Stevenson is working for is probably in the software development business, if as a business or for internal needs. They probably have Visual Studio licenses (there is not _that_ much that you can do as a professional with the free version). But they don't buy VMWare Workstation. YGWYPF.
Edit Comment
h0bbel, on November 24, 2006 at 8:59am, said:
That is indeed very interesting, and yes that would be the answer to my initial question. In this particular case, the Gallery Appliance, we could create a script that zerospaces the disk and then provide documentation on how to run vmware-vdiskmanager to shrink it.
Thanks a lot, I'll look into that for the next version of the appliance.
Edit Comment
Shrinking VMDK files - h0bbel, on January 2, 2007 at 1:18am, said:
[...] distribute between the different team members. So, I decided to try out the technique outlined by a random comment by “Guest” on a previous [...]
Edit Comment
Jon, on February 8, 2007 at 6:15pm, said:
The suggestion that on *nix based guest of using cat /dev/zero etc.. Worked like a charm :)
I managed to shave about 200MB off my VM image.
Thanks
Edit Comment
h0bbel, on February 8, 2007 at 7:40pm, said:
Yep, works great as mentioned in http://h0bbel.p0ggel.org/2007/01/02/shrinking-vmdk-files/
Edit Comment
coComment - Site comments by h0bbel, on March 16, 2007 at 10:32am, said:
View this article on its blog
Edit Comment
me, on May 30, 2007 at 12:40pm, said:
Only works on Windows it seems:
# /usr/bin/vmware-vdiskmanager
VMware Virtual Disk Manager - build 44356.
Usage: vmware-vdiskmanager OPTIONS diskName
Offline disk manipulation utility
Options:
-c : create disk; need to specify other create options
-d : defragment the specified virtual disk
-n : rename the specified virtual disk; need to
specify destination disk-name
-q : do not log messages
-r : convert the specified disk; need to specify
destination disk-type
-x : expand the disk to the specified capacity
Additional options for create and convert:
-a : (for use with -c only) adapter type (ide, buslogic or lsilogic)
-s : capacity of the virtual disk
-t : disk type id
Disk types:
0 : single growable virtual disk
1 : growable virtual disk split in 2Gb files
2 : preallocated virtual disk
3 : preallocated virtual disk split in 2Gb files
The capacity can be specified in sectors, Kb, Mb or Gb.
The acceptable ranges:
ide adapter : [100.0Mb, 950.0Gb]
scsi adapter: [100.0Mb, 950.0Gb]
ex 1: vmware-vdiskmanager -c -s 850Mb -a ide -t 0 myIdeDisk.vmdk
ex 2: vmware-vdiskmanager -d myDisk.vmdk
ex 3: vmware-vdiskmanager -r sourceDisk.vmdk -t 0 destinationDisk.vmdk
ex 4: vmware-vdiskmanager -x 36Gb myDisk.vmdk
ex 5: vmware-vdiskmanager -n sourceName.vmdk destinationName.vmdk
Edit Comment
zargon, on May 31, 2007 at 5:44pm, said:
I have been looking to do the same thing since I went too large with the virtual disk of a set size.
The host machines are running linux, which complicates matters since vmware-mount doesn't work and vmware-vdiskmanager doesn't have the shrink option (-k).
After polking around the web for a bit, which is how a stumbled across this site. I found the best solution for my situation. I am using vmware-vdiskmanager to create a new virtual disk from the old one and forcing the size smaller.
vmware-vdiskmanager -r virtual_disk.vmdk -t 2 -s 4Gb virtual_disk_temp.vmdk
I then rename the old one, rename the new one, boot the system to test and then delete the old one. Done and pretty simple, I think it is less mess than the other method, even on a Windows based host.
Edit Comment
Guggi, on June 20, 2007 at 4:51pm, said:
Faced the same problem and found this site, http://chitchat.at.infoseek.co.jp/vmware/vmtools.html!
Very useful tool to automate online shrink in guest.
Edit Comment
jziemian, on June 22, 2007 at 7:04pm, said:
Anybody find a Windows (XP/Vista) zero fill utility?
Edit Comment
VirtuaMag.net, on July 9, 2007 at 9:51pm, said:
[...] Offline VMware disk shrink ? [...]
Edit Comment
VMware, on July 27, 2007 at 1:36am, said:
From the SDelete site as hosted by Microsoft (http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx):
Usage: sdelete [-p passes] [-s] [-q]
sdelete [-p passes] [-z|-c] [drive letter]
-c
Zero free space (good for virtual disk optimization)
...
Just set the -c flag and you can do it w/ the latest sdelete. No need to find other obscure utils.
Edit Comment
visitingalien, on August 2, 2007 at 3:27pm, said:
I came here to try and find out something useful about VMWare but couldn't because of your REALLY STUPID colour scheme. I won't be back and don't bother emailing me.
Edit Comment
shrink virtual disk wmware - Eniro, on August 10, 2007 at 1:39pm, said:
[...] Offline VMware disk shrink? - h0bbel [...]
Edit Comment
synarchy, on April 27, 2008 at 4:21am, said:
Thank you Zargon.That's just what I needed.
Edit Comment
Anonymous, on August 7, 2009 at 3:54pm, said:
You guys really have to get a grip on what you are trying to do. There are 2 ways to shrink a disk.
One is the -x flag. This literally shrinks the disk using vmware-vdiskmanager.exe, taking the disk from a 16G to a 10G disk or whatever.
The next is the -k flag. This removes the the extra file space from the image and creates more space available on your drive without removing space from your VM image.
Because remeber that even if later you enlarge the C drive on a machine, you can never expand it because MS won't let you do it. Even if it is dynamic.
If you are interested in free then use VMServer, the command line util is there and is much better then player.
Edit Comment
h0bbel, on August 8, 2009 at 11:32am, said:
@Anonymous: You can expand the C:\ drive, it just takes some work as outlined in Howto: Expand VM boot partition.
Edit Comment