Shrinking VMDK files
With the steadily approaching release of Gallery 2.2, currently available as Release Candidate 1 we discovered the need to set up some sort of test environment. to run the unit tests that tests the various bits and pieces of the Gallery 2 application. The test environment would have to be flexible enough to run MySQL, Postgres and DB2 to cover some of the RDBMS that it supports. Since we already offer the Gallery Appliance, setting up a VMware Server/Player appliance was a natural thought for us, and this time around we opted to go with a basic Debian install, based on the minimal images from www.thoughtpolice.co.uk Without tweaking Debian packages all that much, we ended up with a compressed VM of about 700MB (715 242 245 bytes), which is somewhat large to distribute between the different team members. So, I decided to try out the technique outlined by a random comment by "Guest" on a previous post. Basically what was suggested was to zero-fill the remaining disk space in the VMDK files, and them use vmware-vdiskmanager.exe to run an offline shrink operation on the virtual disks. The steps I did was:- 1. Run cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill inside the VM
- 2. Run vmware-vdiskmanager.exe -k g2-developer-vm.vmk
- 2 093 547 520 g2-developer-vm-s001.vmdk
- 2 084 110 336 g2-developer-vm-s002.vmdk
- 786 432 g2-developer-vm-s003.vmdk
- 1 888 944 128 g2-developer-vm-s001.vmdk
- 1 930 625 024 g2-developer-vm-s002.vmdk
- 786 432 g2-developer-vm-s003.vmdk
Post metadata
Published January 2, 2007 01:17
19 comments
Tagged with debian, Gallery, virtual appliance, virtualization and VMWare
19 Responses to Shrinking VMDK files:
11 Pingbacks to Shrinking VMDK files:
-
0000-00-00 00:00:00view blog [IMG]
-
0000-00-00 00:00:00for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks to Christian Mohn for the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
-
0000-00-00 00:00:00for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks to Christian Mohn for the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
-
0000-00-00 00:00:00./diskTool -k 0 /path/to/f7.vmdk Shrink : 100% (9060864/9060864) done. This took a while and I noticed no change in size. I read somewhere that the utility needs a huge zero-filled file to mark the unused space. Back in the guest OS, I created a new partition /dev/sda3 with all the remaining space, formatted it as ext3 and put the zero-filled file in there:
-
2007-03-27 05:03:17[...] this an acceptable way to shrink the disk if the defrag and shrink does nto appear to do anything? http://h0bbel.p0ggel.org/2007/01/02/shrinking-vmdk-files/i tried it but got a "cat: write error: No space left on [...]
-
2007-04-04 03:32:32[...] [...]
-
2007-05-29 21:13:58[...] with the Virtual PC Pre-Compactor that comes with Virtual PC when hosting Windows, and there are Linux options for shrinking VM hard drives as [...]
-
2007-06-24 09:20:27[...] Virtual PC Guy, h0bbel) Inwieweit das in Kombination mit Shrink und Defragmentieren nötig ist, beziehungsweise etwas [...]
-
2007-07-03 08:48:34[...] Shrink Splitting guide JavaScript::Squish - Compact JavaScript code to minim.. Shrinking VMDK files - h0bbel ProGuard Automated Food Product Packaging Machines - Shrink Fi.. Step by Step: Ripping DVD Movies [...]
-
2007-08-20 18:57:23for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks toChristian Mohnfor the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
-
2007-08-21 17:06:50[...] system. Is it installed in Linux guest? By the way, have you read this shrinking-vmdk-files He suggest to do a cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f [...]
0000-00-00 00:00:00
view blog [IMG]
0000-00-00 00:00:00
for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks to Christian Mohn for the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
0000-00-00 00:00:00
for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks to Christian Mohn for the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
0000-00-00 00:00:00
./diskTool -k 0 /path/to/f7.vmdk Shrink : 100% (9060864/9060864) done. This took a while and I noticed no change in size. I read somewhere that the utility needs a huge zero-filled file to mark the unused space. Back in the guest OS, I created a new partition /dev/sda3 with all the remaining space, formatted it as ext3 and put the zero-filled file in there:
2007-03-27 05:03:17
[...] this an acceptable way to shrink the disk if the defrag and shrink does nto appear to do anything? http://h0bbel.p0ggel.org/2007/01/02/shrinking-vmdk-files/i tried it but got a "cat: write error: No space left on [...]
2007-03-27 19:26:58
The error is normal. It fills up the remainder of the disk (it will dump zeros to the huge file until there's no space left), sync (dump whatever's left in the cache) and finally, delete the file.
2007-04-04 03:32:32
[...] [...]
2007-04-06 18:17:47
Thank you very much for the article - it came in handy this afternoon while trying to back up a virtual server.
However, we ran into a gotcha that everyone should be aware of: There cannot be any snapshots extant for the virtual server in question. When attempting to shrink the disk image, we got the error "The selected operation can only be executed on a disk with no snapshots." This was fixed by using the VMware server console to delete the snapshots. It should be noted that this can take a while to run.
Also, make sure that there are no lock files for a virtual machine. I found that shutting down the instance and disconnecting all consoles from it remedied this.
When performing the disk space consolidation procedure it is also advisable to drop the virtual server into single user mode, if applicable, when creating the disk file full of zeroes. This prevents the operating system from panicking when the disk space is used up.
I hope that this helps other people who encounter this.
2007-04-06 20:31:53
Nice pointers, Eric. All of this is absolutely true, in my case I did this on a single user machine that we were packaging as an appliance. Doing this for backup purposes for a live running VM is something else, of course and your steps highlight that.
2007-04-06 22:48:50
Hey h0bbel,
I worked with Eric on shrinking our Solaris 10-x86 VM and wanted to post some numbers:
Before Shrink:
6,527,320,064
After Shrink:
6,393,692,160
However, after compressing the .vmdk we were able to fit it (4,651,149,126) onto a 4.7 GB DVD, which was our goal.
BTW, step two in your procedure has the name of the file as .vmkd, FYI.
;)
2007-04-10 20:59:27
@-X-: Where? I see no .vmdk :-)
Thanks, I've updated the post.
2007-05-29 21:13:58
[...] with the Virtual PC Pre-Compactor that comes with Virtual PC when hosting Windows, and there are Linux options for shrinking VM hard drives as [...]
2007-06-24 09:20:27
[...] Virtual PC Guy, h0bbel) Inwieweit das in Kombination mit Shrink und Defragmentieren nötig ist, beziehungsweise etwas [...]
2007-07-03 08:48:34
[...] Shrink Splitting guide JavaScript::Squish - Compact JavaScript code to minim.. Shrinking VMDK files - h0bbel ProGuard Automated Food Product Packaging Machines - Shrink Fi.. Step by Step: Ripping DVD Movies [...]
2007-08-20 18:57:23
for that. If you've just got one big partition, then you can't use the file transfer option and still have exactly the same machine. Fortunately there's a relatively painless way of reducing the size of the disk to just what's needed (thanks toChristian Mohnfor the technique). The first step is to zero out all the free space on each partition of the drive you want to shrink. This, in effect, marks the free space. You can do that easily with this command:
2007-08-21 17:06:50
[...] system. Is it installed in Linux guest? By the way, have you read this shrinking-vmdk-files He suggest to do a cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f [...]
2008-03-12 05:48:54
Finally, I found a process that might work. Go h0bbel.p0ggel. :) I've got a VM that has grown from 12GB to 28+ due to the fact that MySql tran logging was on and generating 1GB logs.
I'm going to play with this on my dev boxes this week. Thanks!
2008-03-15 15:40:40
I've been doing something similar for years.
dd if=/dev/zero of=delme bs=102400 || rm -rf delme
You need to do it on each partition, but not hard to do.
Also, zero out your swap partition. Paraphrased:
swapoff -a
dd if=/dev/zero of=/dev/hdc1
mkswap /dev/hdc1
(then shutdown without a "swapon")
2008-06-01 10:19:05
Since it is a Debian system, don't forget to delete the contents of /var/cache/apt before zero-filling the disk! This can save quite a lot of space!