quick and dirty sandboxing using unionfs

On occasion i've wanted to perform some dev work using a base system and not care what happens to the system. Usually VM images are the easiest way to do this; keep a backup copy of the image and overwrite the new one when you want to go back. But what about debugging? And diffing changes? Using a union filesystem overlay you can keep a base system and copy its writes to a separate location without affecting the base system.

Herein lies a guide to setting up a union sandbox for development purposes using unionfs-fuse. This is the quickest, dirtiest way to perform operations in a sandbox which will not effect the base system. All writes will end up in a single directory which can be cleaned between uses. With debugging enabled one can see any writeable actions that take place in the sandbox, thus allowing for a more fine-grained look at the effects of an application on a system.

Note that unionfs-fuse is not as production-ready as a kernel mode unionfs (aufs is an alternative) but this method does not require kernel patching. Also note that this system may provide unexpected results on a "root" filesystem.

Also note that this guide is for a basic 'chroot' environment. The process table and devices are shared with the host system, so anything done by a process could kill the host system's processes or damage hardware. Always use caution when in a chroot environment. A safer method is replicating the sandbox in a LiveDVD with writes going to a tmpfs filesystem. The image could be booted from VMware to speed development.

Unfortunately it seems like the current unionfs-fuse does not handle files which need to be mmap()'d. A kernel solution may be a better long-term fix, but for the short term there is a workaround included below.

1. set up unionfs

    # Make sure kernel-* is not excluded from yum.conf
    yum -y install kernel-devel dkms dkms-fuse fuse fuse-devel
    /etc/init.d/fuse start
    yum -y install fuse-unionfs

   
   
2. cloning a build box

    mkdir sandbox
    cd sandbox
    rsync --progress -a /.autofsck /.autorelabel /.bash_history /bin /boot /dev /etc \
     /home /lib /lib64 /mnt /opt /sbin /selinux /srv /usr /var .
    mkdir proc sys tmp spln root
    chmod 1777 tmp
    cd ..

   
   
3. setting up the unionfs

    mkdir writes mount
    unionfs -o cow -o noinitgroups -o default_permissions -o allow_other -o use_ino \
     -o nonempty `pwd`/writes=RW:`pwd`/sandbox=RO `pwd`/mount

   
   
4. using the sandbox

    mount -o bind /proc `pwd`/mount/proc
    mount -o bind /sys `pwd`/mount/sys
    mount -o bind /dev `pwd`/mount/dev
    mount -t devpts none `pwd`/mount/dev/pts
    mount -t tmpfs none `pwd`/mount/dev/shm
    chroot `pwd`/mount /bin/bash --login

   
   
5. handling mmap()'d files

    mkdir mmap-writes
    cp -a --parents sandbox/var/lib/rpm mmap-writes/
    mount -o bind `pwd`/mmap-writes/sandbox/var/lib/rpm `pwd`/mount/var/lib/rpm