如何rsync只有一个特定的文件列表？


How to rsync only a specific list of files?

我有大约50个文件在不同的子目录中，我想推到一个远程服务器。我想rsync可以使用--include from选项为我实现这一点。如果没有--exclude="*"选项，目录中的所有文件都将被同步，如果没有该选项，则没有文件。

1
rsync -avP -e ssh --include-from=deploy/rsync_include.txt --exclude=* ./ [email protected]:/var/www/ --dry-run

我刚开始运行时是干的，0.0.0.0显然被远程服务器的IP所取代。rsync include.txt的内容是一个新的行分隔列表，列出了我要上载的文件的相对路径。

有没有更好的方法可以在周一的早上逃走？

有一个标志--files-from可以完全满足您的需要。来自man rsync：

1
--files-from=FILE

Using this option allows you to specify the exact list of files to transfer (as read from the specified FILE or - for standard input). It also tweaks the default behavior of rsync to make transferring just the specified files and directories easier:

  • The --relative (-R) option is implied, which preserves the path information that is specified for each item in the file (use --no-relative or --no-R if you want to turn that off).

  • The --dirs (-d) option is implied, which will create directories specified in the list on the destination rather than noisily skipping them (use --no-dirs or --no-d if you want to turn that off).

  • The --archive (-a) option’s behavior does not imply --recursive (-r), so specify it explicitly, if you want it.

  • These side-effects change the default state of rsync, so the position of the --files-from option on the command-line has no bearing on how other options are parsed (e.g. -a works the same before or after --files-from, as does --no-R and all other options).

The filenames that are read from the FILE are all relative to the source dir -- any leading slashes are removed and no".." references are allowed to go higher than the source dir. For example, take this command:

1
rsync -a --files-from=/tmp/foo /usr remote:/backup

If /tmp/foo contains the string "bin" (or even"/bin"), the /usr/bin directory will be created as /backup/bin on the remote host. If it contains"bin/" (note the trailing slash), the immediate contents of the directory would also be sent (without needing to be explicitly mentioned in the file -- this began in version 2.6.4). In both
cases, if the -r option was enabled, that dir’s entire hierarchy would also be transferred (keep in mind that -r needs to be specified explicitly with --files-from, since it is not implied by -a). Also note that the effect of the (enabled by default) --relative option is to duplicate only the path info that is read from the file -- it
does not force the duplication of the source-spec path (/usr in this case).

In addition, the --files-from file can be read from the remote host instead of the local host if you specify a"host:" in front of the file (the host must match one end of the transfer). As a short-cut, you can specify just a prefix of":" to mean"use the remote end of the transfer". For example:

1
rsync -a --files-from=:/path/file-list src:/ /tmp/copy

This would copy all the files specified in the /path/file-list file that was located on the remote"src" host.

If the --iconv and --protect-args options are specified and the --files-from filenames are being sent from one host to another, the filenames will be translated from the sending host’s charset to the receiving host’s charset.

NOTE: sorting the list of files in the --files-from input helps rsync to be more efficient, as it will avoid re-visiting the path elements that are shared between adjacent entries. If the input is not sorted, some path elements (implied directories) may end up being scanned multiple times, and rsync will eventually unduplicate them after
they get turned into file-list elements.

相关讨论

如果要保持绝对路径不变，则--files-from=参数需要使用尾随斜杠。所以你的命令会变成如下：

1
rsync -av --files-from=/path/to/file / /tmp/

这可以像有大量文件一样完成，并且您希望将所有文件复制到X路径。因此，您将找到这些文件并将输出抛出到如下文件：

1
find /var/* -name *.log > file

据记录，上述答案除了一个没有任何帮助。综上所述，您可以使用--files-from=执行备份操作，方法如下：

1
 rsync -aSvuc `cat rsync-src-files` /mnt/d/rsync_test/

或

1
 rsync -aSvuc --recursive --files-from=rsync-src-files . /mnt/d/rsync_test/

前一个命令是不言自明的，除了文件rsync-src-files的内容，我将在下面详细说明。现在，如果您想使用后一个版本，您需要记住以下四点：

  • 注意一个需要同时指定--files-from和源目录
  • 需要明确说明--recursive。
  • 文件rsync-src-files是一个用户创建的文件，它被放置在用于此测试的SRC目录中。
  • rsyn-src-files包含要复制的文件和文件夹，并相对于源目录进行复制。重要事项：确保文件中没有尾随空格或空行。在下面的示例中，只有两行，而不是三行(通过偶然的方式计算出来)。rsynch-src-files的含量为：

    • FuldNAME1FuldNAME2

    如果要查找特定的文件列表，可以将它们直接放在命令行上，这样做可能会更容易：

    1
    # rsync -avP -e ssh `cat deploy/rsync_include.txt` [email protected]:/var/www/

    但是，假设您的列表不长，命令行长度会有问题，并且rsync_include.txt文件只包含真正的路径(即没有注释，也没有regexps)。

    相关讨论

    1
    2
    3
    4
    5
    $ date
      Wed 24 Apr 2019 09:54:53 AM PDT
    $ rsync --version
      rsync  version 3.1.3  protocol version 31
      ...

    语法：rsync 。

    文件夹名称(此处，带有培训/；例如Cancer - Evolution/)位于文件夹列表文件(例如：cm_folder_list_test)中：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # /mnt/Vancouver/projects/ie/claws/data/cm_folder_list_test
    # test file: 2019-04-24
    Cancer/
    Cancer - Evolution/
    Cancer - Genomic Variants/
    Cancer - Metastasis (EMT Transition ...)/
    Cancer Pathways, Networks/
    Catabolism - Autophagy; Phagosomes; Mitophagy/
    Catabolism - Lysosomes/

    如果不包括那些尾随的/，则会创建rsync的目标文件夹，但这些文件夹是空的。

    这些文件夹名附加到其路径的其余部分(/home/victoria/Mail/2_RESEARCH - NEWS中)，从而提供rsync的完整文件夹路径；例如：/home/victoria/Mail/2_RESEARCH - NEWS/Cancer - Evolution/。

    请注意，您还需要使用--files-from=…，而不是--include-from=……

    rsync -aqP --delete --files-from=/mnt/Vancouver/projects/ie/claws/data/cm_folder_list_test "/home/victoria/Mail/2_RESEARCH - NEWS" $IN/

    (在bash脚本中，我定义了变量$IN，如下所示。)

    1
    2
    BASEDIR="/mnt/Vancouver/projects/ie/claws"
    IN=$BASEDIR/data/test/input

    使用的rsync选项：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
     -a  :   archive: equals -rlptgoD (no -H,-A,-X)
        -r  :   recursive
        -l  :   copy symlinks as symlinks
        -p  :   preserve permissions
        -t  :   preserve modification times
        -g  :   preserve group
        -o  :   preserve owner (super-user only)
        -D  :   same as --devices --specials
      -q  :   quiet (https://serverfault.com/questions/547106/run-totally-silent-rsync)

      --delete
        This  tells  rsync to delete extraneous files from the RECEIVING SIDE (ones
        that AREN’T ON THE SENDING SIDE), but only for the directories that are
        being synchronized.  You must have asked rsync to send the whole directory
        (e.g. "dir" or"dir/") without using a wildcard for the directory’s contents
        (e.g."dir/*") since the wildcard is expanded by the shell and rsync thus
        gets a request to transfer individual files, not the files’ parent directory.
        Files  that  are  excluded  from  the transfer are also excluded from being
        deleted unless you use the --delete-excluded option or mark the rules as
        only matching on the sending side (see the include/exclude modifiers in the
        FILTER RULES section).  ...

    这个答案不是问题的直接答案。但它应该帮助您找出最适合您的问题的解决方案。

    分析问题时，应激活调试选项-vv。

    然后rsync将输出包含或排除哪些文件的模式：

    1
    2
    3
    building file list ...
    [sender] hiding file FILE1 because of pattern FILE1*
    [sender] showing file FILE2 because of pattern *