I ran into this annoying thing about rsync when I was writing this new backup script in BASH . I wanted support for excluding directories in config files for clients being backed up, since rsync support –exclude=’some/path’ I could have config files with stuff like this:
EXCLUDE="--exclude='proc/*' --exclude='dev/*'"
But while testing this I noticed rsync simply ignored the EXCLUDE variable, so I created this little test where I put in some arbitrary proc and dev directories with some arbitrary sub-directories, check out this example:
joar@uranus:~/tmp$ rsync -va --delete --exclude "dev" --exclude "proc/*" rsynctest/ rsynctest2/ | grep proc jdk1.7.0_07/jre/lib/amd64/libsaproc.so jdk1.7.0_07/proc/ proc/ joar@uranus:~/tmp$ EXCLUDE="--exclude 'dev' --exclude 'proc/*'" joar@uranus:~/tmp$ rsync -va --delete $EXCLUDE rsynctest/ rsynctest2/ | grep proc jdk1.7.0_07/jre/lib/amd64/libsaproc.so jdk1.7.0_07/proc/ jdk1.7.0_07/proc/1 jdk1.7.0_07/proc/2 jdk1.7.0_07/proc/3 jdk1.7.0_07/proc/4 jdk1.7.0_07/proc/5 jdk1.7.0_07/proc/6 proc/ proc/1/ proc/2/ proc/3/ proc/4/ proc/5/ proc/6/ proc/7/ proc/8/ joar@uranus:~/tmp$
Notice when I specify –exclude= in the first rsync, the proc dir is rsync’ed without sub-dirs, but when i put that stuff into the EXCLUDE variable the subdirs of proc is being rsynced (!) . I poked around this for a while trying double quotes VS. single quotes, even tried expansion stuff like $(echo $EXCLUDE) .
After a good night sleep I came up with this workaround:
joar@uranus:~/tmp$ EXCLUDE='proc/* dev/*' joar@uranus:~/tmp$ for f in $EXCLUDE; do echo $f >> tmpfile; done joar@uranus:~/tmp$ cat tmpfile proc/* dev/* joar@uranus:~/tmp$ rsync -va --delete --exclude-from=./tmpfile rsynctest/ rsynctest2/ | grep proc jdk1.7.0_07/jre/lib/amd64/libsaproc.so jdk1.7.0_07/proc/ proc/ joar@uranus:~/tmp$
So I got what I want 🙂
The reason I want to cat stuff from the EXCLUDE variable into a tmpfile is because I want 1 config file pr. backup client. Those who setup new backups shouldn’t have to remember to setup a 2nd config file with directory exceptions.