ChatGPT解决这个技术问题 Extra ChatGPT

How can I configure rsync to create target directory on remote server?

I would like to rsync from local computer to server. On a directory that does not exist, and I want rsync to create that directory on the server first.

How can I do that?


m
mivk

If you have more than the last leaf directory to be created, you can either run a separate ssh ... mkdir -p first, or use the --rsync-path trick as explained here :

rsync -a --rsync-path="mkdir -p /tmp/x/y/z/ && rsync" $source user@remote:/tmp/x/y/z/

Or use the --relative option as suggested by Tony. In that case, you only specify the root of the destination, which must exist, and not the directory structure of the source, which will be created:

rsync -a --relative /new/x/y/z/ user@remote:/pre_existing/dir/

This way, you will end up with /pre_existing/dir/new/x/y/z/

And if you want to have "y/z/" created, but not inside "new/x/", you can add ./ where you want --relativeto begin:

rsync -a --relative /new/x/./y/z/ user@remote:/pre_existing/dir/

would create /pre_existing/dir/y/z/.


The server i'm accessing doesn't have mkdir, so i'm using the hack: --rsync-path="echo f > /public/.keep && rsync /public/.keep /public/newfolder/ && rsync" \ it does create an extra file but you can remove later.
This doesn't seem to be working on newer rsync protocols, I get an error similar to following when using && or ; in --rsync-path invalid characters in scp command! here:&& /usr/bin/rsync
@ghitesh did you try to escape the && ?
@FelipeAlvarez Yeah I did, but did not work. I am not able to find the link now, but some documentation said this is fixed, as I believe it could (and was) be exploited.
Thanks, this works. In my case I was running a local bash script on a remote server and within that running the ssh mkdir command which was executing, but then killing the rest of the script before running the rsync command. This allows the rest of the script to run without issues.
A
Alfredo Yong

Assuming you are using ssh to connect rsync, what about to send a ssh command before:

ssh user@server mkdir -p existingdir/newdir

if it already exists, nothing happens


ufk's request is for syntax that will cause "rsync to create [a missing target] directory on the server [&& copy the files]." Your response is valid in that it will create the missing target directory on the server ... but not with rsync, therefore, no answer.
This is not an option if ssh is not supported and rsync connects to the rsync daemon on the remote end.
S
Sergey Vyacheslavovich Brunov

From the rsync manual page (man rsync):

--mkpath                 create the destination's path component

This is the only option that worked for me.
Note that this option was only added in version 3.2.3 (6 Aug 2020).
Better a decade late than never! This flag is a lifesaver, you don't have to do any weird hacking of the syntax using e.g. --relative with this flag, you can simply leave a trailing slash on both source and destination paths and all recursive files/folders will be automatically synced between them...
e
e-sushi

The -R, --relative option will do this.

For example: if you want to backup /var/named/chroot and create the same directory structure on the remote server then -R will do just that.


You can also specify only part of the path to be created by using /./ in the source path. See man page for details.
Why upvote? The answer is wrong, rsync won't create directories automatically if target folders don't exist. Sigh.
I strongly suggest readers to check the manpage for --relative. It might not do what you think.
--relative sticks the full path, rather than the file, at the endpoint you provide on the server. Normally /tmp/foo/bar/file.c to remote:/tmp would produce remote:/tmp/file.c but with -R it creates /tmp/foo/bar/file.c Don't use this option unless you know what you are doing, you will get a really messy directory structure.
@Marki555 's comment is key to success. --relative in conjunction with /./ is what most newbies will be looking for. rsync -avz -e ssh --progress --relative ~/./Desktop/ TARGET_IP:~/ will copy the contents of the desktop to the target desktop. Don't know how differences in usernames might affect this.
k
kdubs

this worked for me:

 rsync /dev/null node:existing-dir/new-dir/

I do get this message :

skipping non-regular file "null"

but I don't have to worry about having an empty directory hanging around.


Thanks for the answer kdubs. That worked great. So, if I want to keep the files I'm syncing inside of the folder I'm pointing to as the source, then I just need to specify it, and it gets created. ex: rsync -avph /mytest/ /otherfolder/mytest/ Then I get the output "created directory /otherfolder/mytest"
Don't forget the slash ('/') at the end of new-dir, otherwise it will create a non-directory file.
it only works with single level of new-dir, if u want /new-dir1/new-dir2/ it won't work
true, in that case run it twice, or thrice... 8- )
C
Chris Dennis

I don't think you can do it with one rsync command, but you can 'pre-create' the extra directory first like this:

rsync --recursive emptydir/ destination/newdir

where 'emptydir' is a local empty directory (which you might have to create as a temporary directory first).

It's a bit of a hack, but it works for me.

cheers

Chris


No need to use temporary folder. See rsync /dev/null node:existing-dir/new-dir/ suggestion above.
l
lucid_dreamer

This answer uses bits of other answers, but hopefully it'll be a bit clearer as to the circumstances. You never specified what you were rsyncing - a single directory entry or multiple files.

So let's assume you are moving a source directory entry across, and not just moving the files contained in it.

Let's say you have a directory locally called data/myappdata/ and you have a load of subdirectories underneath this. You have data/ on your target machine but no data/myappdata/ - this is easy enough:

rsync -rvv /path/to/data/myappdata/ user@host:/remote/path/to/data/myappdata

You can even use a different name for the remote directory:

rsync -rvv --recursive /path/to/data/myappdata user@host:/remote/path/to/data/newdirname

If you're just moving some files and not moving the directory entry that contains them then you would do:

rsync -rvv /path/to/data/myappdata/*.txt user@host:/remote/path/to/data/myappdata/

and it will create the myappdata directory for you on the remote machine to place your files in. Again, the data/ directory must exist on the remote machine.

Incidentally, my use of -rvv flag is to get doubly verbose output so it is clear about what it does, as well as the necessary recursive behaviour.

Just to show you what I get when using rsync (3.0.9 on Ubuntu 12.04)

$ rsync -rvv *.txt user@remote.machine:/tmp/newdir/
opening connection using: ssh -l user remote.machine rsync --server -vvre.iLsf . /tmp/newdir/
user@remote.machine's password:
sending incremental file list
created directory /tmp/newdir
delta-transmission enabled
bar.txt
foo.txt
total: matches=0  hash_hits=0  false_alarms=0 data=0

Hope this clears this up a little bit.


This does not exactly work as expected. I tried the first command rsync -rvv /path/to/data/myappdata user@host:/remote/path/to/data/myappdata but what it created on the target is /remote/path/to/data/myappdata/myappdata/
If I make a slight change to your first command and add a slash '/' after the source path, then it works...like so: rsync -rvv /path/to/data/myappdata/ user@host:/remote/path/to/data/myappdata
M
Montage

eg:

from: /xxx/a/b/c/d/e/1.html

to: user@remote:/pre_existing/dir/b/c/d/e/1.html

rsync:

cd /xxx/a/ && rsync -auvR b/c/d/e/ user@remote:/pre_existing/dir/


E
Eugene Chung
rsync source.pdf user1@192.168.56.100:~/not-created/target.pdf

If the target file is fully specified, the directory ~/not-created is not created.

rsync source.pdf user1@192.168.56.100:~/will-be-created/

But the target is specified with only a directory, the directory ~/will-be-created is created. / must be followed to let rsync know will-be-created is a directory.


E
Eric Aya

use rsync twice~

1: tranfer a temp file, make sure remote relative directories has been created.

tempfile=/Users/temp/Dir0/Dir1/Dir2/temp.txt
# Dir0/Dir1/Dir2/ is directory that wanted.
rsync -aq /Users/temp/ rsync://remote

2: then you can specify the remote directory for transfer files/directory

tempfile|dir=/Users/XX/data|/Users/XX/data/
rsync -avc /Users/XX/data rsync://remote/Dir0/Dir1/Dir2
# Tips: [SRC] with/without '/' is different

m
mgutt

This creates the dir tree /usr/local/bin in the destination and then syncs all containing files and folders recursively:

rsync --archive --include="/usr" --include="/usr/local" --include="/usr/local/bin" --include="/usr/local/bin/**" --exclude="*" user@remote:/ /home/user

Compared to mkdir -p, the dir tree even has the same perms as the source.


u
user8150417

If you are using a version or rsync that doesn't have 'mkpath', then --files-from can help. Suppose you need to create 'mysubdir' in the target directory

Create 'filelist.txt' to contain mysubdir/dummy

mkdir -p source_dir/mysubdir/
touch source_dir/mysubdir/dummy
rsync --files-from='filelist.txt' source_dir target_dir

rsync will copy mysubdir/dummy to target_dir, creating mysubdir in the process. Tested with rsync 3.1.3 on Raspberry Pi OS (debian).