rsync exclude according to .gitignore & .hgignore & svn:ignore like --filter=:C

As mentioned by luksan, you can do this with the --filter switch to rsync. I achieved this with --filter=':- .gitignore' (there's a space before ".gitignore") which tells rsync to do a directory merge with .gitignore files and have them exclude per git's rules. You may also want to add your global ignore file, if you have one. To make it easier to use, I created an alias to rsync which included the filter.

You can use git ls-files to build the list of files excluded by the repository's .gitignore files. https://git-scm.com/docs/git-ls-files

Options:

--exclude-standard Consider all .gitignore files.

-o Don't ignore unstaged changes.

-i Only output ignored files.

--directory Only output the directory path if the entire directory is ignored.

The only thing I left to ignore was .git.

rsync -azP --exclude=.git --exclude=`git -C <SRC> ls-files --exclude-standard -oi --directory` <SRC> <DEST>

After the hours of research I have found exactly what I need: to sync the destination folder with the source folder (also deleting files in the destination if they were deleted in the source), and not to copy to the destination the files that are ignored by .gitignore, but also not to delete this files in the destination:

rsync -vhra /source/project/ /destination/project/ --include='**.gitignore' --exclude='/.git' --filter=':- .gitignore' --delete-after

In other words, this command completely ignores files from .gitignore, both in source and in the destination. You can omit --exclude='/.git' part if want to copy the .git folder too.

You MUST copy .gitignore files from the source. If you will use LordJavac's command, the .gitignore will not be copied. And if you create a file in the destination folder, that should be ignored by .gitignore, this file will be deleted despite .gitignore. This is because you don't have .gitignore-files in the destination. But if you will have these files, the files described in the .gitignore will not be deleted, they will be ignored, just expected.

2018 solution confirmed

rsync -ah --delete 
    --include .git --exclude-from="$(git -C SRC ls-files \
        --exclude-standard -oi --directory >.git/ignores.tmp && \
        echo .git/ignores.tmp')" \
    SRC DST 

Details: --exclude-from is mandatory instead of --exclude because likely case that exclude list would not be parsed as an argument. Exclude from requires a file and cannot work with pipes.

Current solution saves the exclude file inside the .git folder in order to assure it will not affect git status while keeping it self contained. If you want you are welcome to use /tmp.

how about rsync --exclude-from='path/.gitignore' --exclude-from='path/myignore.txt' source destination?
It worked for me.
I believe you can have more --exclude-from parameters too.

I had a number of very large .gitignore files and none of the "pure rsync" solutions worked for me. I wrote this rsync wrapper script, it fully respects .gitignore rules (include !-style exceptions and .gitignore files in subdirectories) and has worked like a charm for me.

For mercurial you might use

hg status -i | sed 's/^I //' > /tmp/tmpfile.txt

to collect the list of files which are NOT under mercurial control because of .hgignore restrictions and then run

rsync -avm --exclude-from=/tmp/tmpfile.txt --delete source_dir/ target_dir/

to rsync all files except the ignored ones. Notice -m flag in rsync that will exclude empty directories from syncing because hg status -i would only list excluded files, not dirs

Try this:

rsync -azP --delete --filter=":- .gitignore" <SRC> <DEST>

It can copy all files to remote directory excluding files in '.gitignore', and delete files not in your current directory.

Check out the MERGE-FILES FILTER RULES section in rsync(1).

It looks like it's possible to create a rsync --filter rule that will include .gitignore files as traverses the directory structure.

Per the rsync man page, in addition to the standard list of file patterns:

files listed in a $HOME/.cvsignore are added to the list and any files listed in the CVSIGNORE environment variable

So, my $HOME/.cvsignore file looks like this:

.git/
.sass-cache/

to exclude .git and the files generated by Sass.

Alternatives:

git ls-files -zi --exclude-standard |rsync -0 --exclude-from=- ...

git ls-files -zi --exclude-per-directory=".gitignore" |...

(rsync only partly understands .gitignore)

Instead of creating exclude filters, you can use git ls-files to select each file to rsync:

#!/usr/bin/env bash

if [[ ! $# -eq 2 ]] ; then
    echo "Usage: $(basename $0) <local source> <rsync destination>"
    exit 1
fi

cd $1
versioned=$(git ls-files --exclude-standard)
rsync --verbose --links --times --relative --protect-args ${versioned} $2

This works even though git ls-files returns newline separated paths. Probably won't work if you have versioned files with spaces in the filenames.

Short answer

rsync -r --info=progress2 --filter=':- .gitignore' SOURCE DEST/

Parameters meaning:

-r: recursive

--info=...: show progress

--filter=...: exclude by the rules listed on the .gitignore file