doc/rsync.sgml

   1 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
   2 <book id="rsync">
   3   <bookinfo>
   4     <title>rsync</title>
   5     <copyright>
   6       <year>1996 -- 2002</year>
   7       <holder>Martin Pool</holder>
   8       <holder>Andrew Tridgell</holder>
   9     </copyright>
  10     <author>
  11       <firstname>Martin</firstname>
  12       <surname>Pool</surname>
  13     </author>
  14   </bookinfo>
  15
  16   <chapter>
  17     <title>Introduction</title>
  18
  19     <para>rsync is a flexible program for efficiently copying files or
  20       directory trees.
  21
  22     <para>rsync has many options to select which files will be copied
  23       and how they are to be transferred.  It may be used as an
  24       alternative to ftp, http, scp or rcp.
  25
  26     <para>The rsync remote-update protocol allows rsync to transfer just
  27       the differences between two sets of files across the network link,
  28       using an efficient checksum-search algorithm described in the
  29       technical report that accompanies this package.</para>
  30
  31     <para>Some of the additional features of rsync are:</para>
  32
  33     <itemizedlist>
  34
  35       <listitem>
  36         <para>support for copying links, devices, owners, groups and
  37           permissions
  38         </para>
  39       </listitem>
  40
  41       <listitem>
  42         <para>
  43           exclude and exclude-from options similar to GNU tar
  44         </para>
  45       </listitem>
  46
  47       <listitem>
  48         <para>
  49           a CVS exclude mode for ignoring the same files that CVS would ignore
  50       </listitem>
  51
  52       <listitem>
  53         <para>
  54           can use any transparent remote shell, including rsh or ssh
  55       </listitem>
  56
  57       <listitem>
  58         <para>
  59           does not require root privileges
  60       </listitem>
  61
  62       <listitem>
  63         <para>
  64           pipelining of file transfers to minimize latency costs
  65       </listitem>
  66
  67       <listitem>
  68         <para>
  69           support for anonymous or authenticated rsync servers (ideal for
  70           mirroring)
  71         </para>
  72       </listitem>
  73     </itemizedlist>
  74   </chapter>
  75
  76
  77
  78   <chapter>
  79     <title>Using rsync</title>
  80     <section>
  81       <title>
  82         Introductory example
  83       </title>
  84
  85       <para>
  86         Probably the most common case of rsync usage is to copy files
  87         to or from a remote machine using
  88         <application>ssh</application> as a network transport.  In
  89         this situation rsync is a good alternative to
  90         <application>scp</application>.
  91       </para>
  92
  93       <para>
  94         The most commonly used arguments for rsync are
  95       </para>
  96
  97       <variablelist>
  98         <varlistentry>
  99           <term><option>-v</option></term>
 100           <listitem>
 101             <para>Be verbose.  Primarily, display the name of each file as it is copied.</para>
 102           </listitem>
 103         </varlistentry>
 104
 105
 106         <varlistentry>
 107           <term><option>-a</option></term>
 108           <listitem>
 109             <para>
 110               Reproduce the structure and attributes of the origin files as exactly
 111               as possible: this includes copying subdirectories, symlinks, special
 112               files, ownership and permissions.  (@xref{Attributes to
 113               copy}.)
 114             </para>
 115           </listitem>
 116         </varlistentry>
 117       </variablelist>
 118
 119
 120
 121       <para><option>-v </option>
 122
 123       <para><option>-z</option>
 124         Compress network traffic, using a modified version of the
 125         @command{zlib} library.</para>
 126
 127       <para><option>-P</option>
 128         Display a progress indicator while files are transferred.  This should
 129         normally be ommitted if rsync is not run on a terminal.
 130       </para>
 131     </section>
 132
 133
 134
 135
 136     <section>
 137       <title>Local and remote</title>
 138
 139       <para>There are six different ways of using rsync. They
 140       are:</para>
 141
 142
 143
 144       <!-- one of (CALLOUTLIST GLOSSLIST ITEMIZEDLIST ORDEREDLIST SEGMENTEDLIST SIMPLELIST VARIABLELIST CAUTION IMPORTANT NOTE TIP WARNING LITERALLAYOUT PROGRAMLISTING PROGRAMLISTINGCO SCREEN SCREENCO SCREENSHOT SYNOPSIS CMDSYNOPSIS FUNCSYNOPSIS CLASSSYNOPSIS FIELDSYNOPSIS CONSTRUCTORSYNOPSIS DESTRUCTORSYNOPSIS METHODSYNOPSIS FORMALPARA PARA SIMPARA ADDRESS BLOCKQUOTE GRAPHIC GRAPHICCO MEDIAOBJECT MEDIAOBJECTCO INFORMALEQUATION INFORMALEXAMPLE INFORMALFIGURE INFORMALTABLE EQUATION EXAMPLE FIGURE TABLE MSGSET PROCEDURE SIDEBAR QANDASET ANCHOR BRIDGEHEAD REMARK HIGHLIGHTS ABSTRACT AUTHORBLURB EPIGRAPH INDEXTERM REFENTRY SECTION) -->
 145       <orderedlist>
 146         <listitem>
 147           <para>
 148             for copying local files. This is invoked when neither
 149             source nor destination path contains a @code{:} separator
 150
 151         <listitem>
 152           <para>
 153             for copying from the local machine to a remote machine using
 154             a remote shell program as the transport (such as rsh or
 155             ssh). This is invoked when the destination path contains a
 156             single @code{:} separator.
 157
 158         <listitem>
 159           <para>
 160             for copying from a remote machine to the local machine
 161             using a remote shell program. This is invoked when the source
 162             contains a @code{:} separator.
 163
 164         <listitem>
 165           <para>
 166             for copying from a remote rsync server to the local
 167             machine. This is invoked when the source path contains a @code{::}
 168             separator or a @code{rsync://} URL.
 169
 170         <listitem>
 171           <para>
 172             for copying from the local machine to a remote rsync
 173             server. This is invoked when the destination path contains a @code{::}
 174             separator.
 175
 176         <listitem>
 177           <para>
 178             for listing files on a remote machine. This is done the
 179             same way as rsync transfers except that you leave off the
 180             local destination.
 181
 182         </listitem>
 183       </orderedlist>
 184           <para>
 185 Note that in all cases (other than listing) at least one of the source
 186 and destination paths must be local.
 187
 188           <para>
 189 Any one invocation of rsync makes a copy in a single direction.  rsync
 190 currently has no equivalent of @command{ftp}'s interactive mode.
 191
 192 @cindex @sc{nfs}
 193 @cindex network filesystems
 194 @cindex remote filesystems
 195
 196           <para>
 197 rsync's network protocol is generally faster at copying files than
 198 network filesystems such as @sc{nfs} or @sc{cifs}.  It is better to
 199 run rsync on the file server either as a daemon or over ssh than
 200 running rsync giving the network directory.
 201       </para>
 202     </section>
 203   </chapter>
 204
 205
 206
 207   <chapter>
 208     <title>Frequently asked questions</title>
 209
 210
 211     <!-- one of (CALLOUTLIST GLOSSLIST ITEMIZEDLIST ORDEREDLIST SEGMENTEDLIST SIMPLELIST VARIABLELIST CAUTION IMPORTANT NOTE TIP WARNING LITERALLAYOUT PROGRAMLISTING PROGRAMLISTINGCO SCREEN SCREENCO SCREENSHOT SYNOPSIS CMDSYNOPSIS FUNCSYNOPSIS CLASSSYNOPSIS FIELDSYNOPSIS CONSTRUCTORSYNOPSIS DESTRUCTORSYNOPSIS METHODSYNOPSIS FORMALPARA PARA SIMPARA ADDRESS BLOCKQUOTE GRAPHIC GRAPHICCO MEDIAOBJECT MEDIAOBJECTCO INFORMALEQUATION INFORMALEXAMPLE INFORMALFIGURE INFORMALTABLE EQUATION EXAMPLE FIGURE TABLE MSGSET PROCEDURE SIDEBAR QANDASET ANCHOR BRIDGEHEAD REMARK HIGHLIGHTS ABSTRACT AUTHORBLURB EPIGRAPH INDEXTERM SECTION SIMPLESECT REFENTRY SECT1) -->
 212     <qandaset>
 213       <!-- one of (QANDADIV QANDAENTRY) -->
 214
 215       <qandaentry>
 216         <question>
 217           <!-- one of (CALLOUTLIST GLOSSLIST ITEMIZEDLIST ORDEREDLIST
 218           SEGMENTEDLIST SIMPLELIST VARIABLELIST CAUTION IMPORTANT NOTE
 219           TIP WARNING LITERALLAYOUT PROGRAMLISTING PROGRAMLISTINGCO
 220           SCREEN SCREENCO SCREENSHOT SYNOPSIS CMDSYNOPSIS FUNCSYNOPSIS
 221           CLASSSYNOPSIS FIELDSYNOPSIS CONSTRUCTORSYNOPSIS
 222           DESTRUCTORSYNOPSIS METHODSYNOPSIS FORMALPARA PARA SIMPARA
 223           ADDRESS BLOCKQUOTE GRAPHIC GRAPHICCO MEDIAOBJECT
 224           MEDIAOBJECTCO INFORMALEQUATION INFORMALEXAMPLE
 225           INFORMALFIGURE INFORMALTABLE EQUATION EXAMPLE FIGURE TABLE
 226           PROCEDURE ANCHOR BRIDGEHEAD REMARK HIGHLIGHTS INDEXTERM) -->
 227           <para>Are there mailing lists for rsync?
 228         </question>
 229
 230         <answer>
 231           <para>Yes, and you can subscribe and unsubscribe through a
 232           web interface at
 233             <ulink
 234               url="http://lists.samba.org/">http://lists.samba.org/</ulink>
 235           </para>
 236
 237           <para>
 238             If you are having trouble with the mailing list, please
 239             send mail to the administrator
 240
 241             <email>rsync-admin@lists.samba.org</email>
 242
 243             not to the list itself.
 244           </para>
 245
 246           <para>
 247             The mailing list archives are searchable.  Use
 248             <ulink url="http://google.com/">Google</ulink> and prepend
 249             the search with <userinput>site:lists.samba.org
 250             rsync</userinput>, plus relevant keywords.
 251           </para>
 252         </answer>
 253       </qandaentry>
 254
 255
 256       <qandaentry>
 257         <question>
 258           <para>
 259             Why is rsync so much bigger when I build it with
 260             <command>gcc</command>?
 261           </para>
 262         </question>
 263         <answer>
 264           <para>
 265             On gcc, rsync builds by default with debug symbols
 266             included.  If you strip both executables, they should end
 267             up about the same size.  (Use <command>make
 268             install-strip</command>.)
 269           </para>
 270         </answer>
 271       </qandaentry>
 272
 273
 274       <qandaentry>
 275         <question>
 276           <para>Is rsync useful for a single large file like an ISO image?</para>
 277         </question>
 278         <answer>
 279           <para>
 280             Yes, but note the following:
 281
 282           <para>
 283    Background: A common use of rsync is to update a file (or set of files) in one location from a more
 284    correct or up-to-date copy in another location, taking advantage of portions of the files that are
 285    identical to speed up the process. (Note that rsync will transfer a file in its entirety if no copy
 286    exists at the destination.)
 287
 288           <para>
 289    (This discussion is written in terms of updating a local copy of a file from a correct file in a
 290    remote location, although rsync can work in either direction.)
 291
 292           <para>
 293    The file to be updated (the local file) must be in a destination directory that has enough space for
 294    two copies of the file. (In addition, keep an extra copy of the file to be updated in a different
 295    location for safety -- see the discussion (below) about rsync's behavior when the rsync process is
 296    interrupted before completion.)
 297
 298           <para>
 299    The local file must have the same name as the remote file being sync'd to (I think?). If you are
 300    trying to upgrade an iso from, for example, beta1 to beta2, rename the local file to the same name
 301    as the beta2 file. *(This is a useful thing to do -- only the changed portions will be
 302    transmitted.)*
 303
 304           <para>
 305    The extra copy of the local file kept in a different location is because of rsync's behavior if
 306    interrupted before completion:
 307
 308           <para>
 309    * If you specify the --partial option and rsync is interrupted, rsync will save the partially
 310    rsync'd file and throw away the original local copy. (The partially rsync'd file is correct but
 311    truncated.) If rsync is restarted, it will not have a local copy of the file to check for duplicate
 312    blocks beyond the section of the file that has already been rsync'd, thus the remainder of the rsync
 313    process will be a "pure transfer" of the file rather than taking advantage of the rsync algorithm.
 314
 315           <para>
 316    * If you don't specify the --partial option and rsync is interrupted, rsync will throw away the
 317    partially rsync'd file, and, when rsync is restarted starts the rsync process over from the
 318    beginning.
 319
 320           <para>
 321    Which of these is most desirable depends on the degree of commonality between the local and remote
 322    copies of the file *and how much progress was made before the interruption*.
 323
 324           <para>
 325    The ideal approach after an interruption would be to create a new file by taking the original file
 326    and deleting a portion equal in size to the portion already rsync'd and then appending *the
 327    remaining* portion to the portion of the file that has already been rsync'd. (There has been some
 328    discussion about creating an option to do this automatically.)
 329
 330    The --compare-dest option is useful when transferring multiple files, but is of no benefit in
 331    transferring a single file. (AFAIK)
 332
 333    *Other potentially useful information can be found at:
 334    -[3]http://twiki.org/cgi-bin/view/Wikilearn/RsyncingALargeFile
 335
 336    This answer, formatted with "real" bullets, can be found at:
 337    -[4]http://twiki.org/cgi-bin/view/Wikilearn/RsyncingALargeFileFAQ*
 338
 339           </para>
 340         </answer>
 341       </qandaentry>
 342     </qandaset>
 343   </chapter>
 344
 345
 346   <appendix>
 347     <title>Other Resources</title>
 348
 349     <para><ulink url="http://www.ccp14.ac.uk/ccp14admin/rsync/"></ulink></para>
 350   </appendix>
 351 </book>