Resume scp/rsync file transfer
November 19, 2008
This is very basic Linux knowledge, but the question has arised too often and I decided to document it.
Problem:
Transfer (or upload/download) a large tree of files and directories from one machine to another using a ssh connection, and being able to resume/continue if the operation is interrupted.
Solution:
Until now, I believe that the best solution is using rsync over ssh, since rsync has a feature to resume interrupted file transfer, even when an entire tree of directories is involved.
Command line:
rsync -vrPtz -e ssh host:/remote_path/* /local_path/
Explained:
-e ssh rsync will use ssh client instead of rsh
-z compress file transfer
-t preserve time (other attributes as owner or permissions are also possible)
-P resume incomplete file transfer
-r recursive into subdirectories
-v verbose
5 Comments Add your own
Leave a Comment
Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <pre> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
Trackback this post | Subscribe to the comments via RSS Feed
1. Lucas | November 21, 2008 at 5:33 pm
Good stuff Daniel! Even though it is basic knowledge, keeping up with several man pages fighting for space in your head is hard, so some tips once in a while are very helpful!
See ya buddy,
2. Phil LaPier | December 3, 2008 at 7:13 am
Thanks Daniel.
You saved me so much hassle with this helpful tip. Apparently filezilla has a bug in it so that when you are resumeing a file larger than 2GB or so, it messes up the filesize and checksum – leaving the file corrupt. Again, thank you.
3. ssh, you touch me in all the right ways. | Phil LaPier | December 3, 2008 at 11:06 pm
[...] thanks to Daniel Ferbers, I managed to transfer the file with success and without reading through lots of man [...]
4. Grant | August 20, 2009 at 5:26 pm
Good info. Thanks!
5. Dan | August 27, 2009 at 12:51 pm
Hi Daniel.
I get the following answer when executing the command.
sent 16 bytes received 62 bytes 7.43 bytes/sec
total size is 1350389760 speedup is 17312689.23
and it stops there. Any idea?
Thank you
Dan