Synchronizing files and directories between two systems with rsync

The rsync utility, which is available for Unix, Linux, OS X, and Microsoft Windows systems, can be used to synchronize files and directories on two systems. Rsync is widely used for mirroring one system to another, for backups, and for copying files and directories. If files to be transferred already exist at the destination system, but are older versions, the tool contains a delta-transfer algorithm that reduces the amount of data that needs to be sent over a network when using it to transfer files to another system over a network; the algorithm allows rsync to send only the differences between source and destination files rather than entire files. By default, rsync determines if files need to be transferred by using a "quick check" algorithm that looks for files that have changed in size or in last-modified time. The utility will copy links and devices and will preserve owner and group permissions on files and directories. Rsync also has an option to exclude specified files and directories from the synchronization operation. It can can use any transparent remote shell, including Secure Shell (SSH) or remote shell (rsh).

E.g., suppose a user, ann, on a Linux system wants to synchronize the contents of her directory mysite to another backup system at She could use a command like the one in the example below:

[ann@localhost ~$ rsync -avz public_html/mysite --exclude 'logs' --exclude mysite.tar.gz's password:
sending incremental file list
<text snipped>

sent 3515156970 bytes  received 145788 bytes  1147479.27 bytes/sec
total size is 3513501409  speedup is 1.00
[ann@localhost ~]$

In the above example, Ann wants to transfer all files and subdirectories beneath the public_html/mysite directory, which is beneath her home directory, but she wants to exclue the logs directory and the files within it and also wants to exclude a file mysite.tar.gz which she created to hold a local backup of her site on the source system. If there is no logs directory already on the remote system, none will exist after the transfer, either. However, if there are files or directories that exist on the remote system within the destination directory, but not on the source system, those will not be deleted from the destination system; they will remain.

The -avz parameters she uses have the following meanings:

-a, --archive               archive mode
-v, --verbose               increase verbosity
-z, --compress              compress file data during the transfer

After those parameters, she specifies the directory, in this case the public_html/mysite directory beneath the one from which she ran the command, that she wishes to synch with the remote system.

The file transfer will take place via an SSH connection since she is using to connect to the remote system where her userid is also ann. A remote-shell transport, such as SSH, is used whenever the source or destination path contains a single colon (:) separator after a host specification. If she were using a direct connection to an rsync daemon, instead, she would use a double colon (::). When the source or destination path contains a double colon separator after a host specification or when an rsync:// URL is specified that indicates that the connection is to an rsync daemon. By using :public_html she is also indicating that the files and directories she is transferring should be placed in a directory named public_html beneath her home directory on the remote system. The location following the colon is relative to her home directory unless it is preceeded by a forward slash (/). E.g., :/home/bill, which would indicate that the files should be placed in the /home/bill directory; of course, her account on the remote system would need write access to that directory in order to place files there.

In the case above where Ann is using the SSH protocol for the transfer, if I issued the command ps -A x | grep sshd | grep -v grep, when the transfer was proceeding, I would see something similar to the following:

# ps -A x | grep sshd | grep -v grep
 3346 ?        Ss     0:00 /usr/sbin/sshd -D
 3465 ?        Ss     0:00 sshd: root@pts/0
 3588 ?        Ss     0:00 sshd: ann [priv]
 3590 ?        S      0:10 sshd: ann@notty

In the above case, only root was logged in via an interactive session. The user ann was performing an rsync file and directory sync from the source system to the system on which I ran the above command, so the command is showing the connection via SSH for the rsync directory transfer.

At the end of the rsync process, the utility reports the bytes sent and the transfer speed. I.e., in this case:

sent 3515156970 bytes  received 145788 bytes  1147479.27 bytes/sec

So the transfer rate was about 8.75 Mbs (1147479.27 bytes * 8 bits/byte / 1024 bits/kilobits / 1024 kilobits/megabits).


TechRabbit ad 300x250

Justdeals Daily Electronics Deals1x1 px