I came across a real life-saver today…

All of my backup scripts are based on using RSYNC. This is because I am used to working between linux systems on which I always have root/sudo access and also have SSH. However I was recently tasked with a “fun” project that involved me needing to script the syncing of some directories between a linux server and a windows server running third-party SFTP software… that doesn’t allow SSH, SCP, etc.

Now, IMHO, the built-in SFTP program for Ubuntu has a syntax that is a royal PIA for any kind of complex or sophisticated file copy operations. Therefore, getting RSYNC like “mirroring” capability using plain SFTP was going to be a scripting nightmare (perhaps not, maybe I didn’t dig deep enough but that is certainly what it looked like to me…).

After much searching of the Google (thankfully Donald Trump hasn’t turned the internet off yet…) I came across a small program called LFTP referenced on several sites linked below in my “references” section.

First I installed LFTP which is in the ubuntu repositories (love Ubuntu…)

apt-get install lftp

Once installed, here is the scriptable syntax:

/usr/bin/lftp -c "open -p 9922 -u jdoe,passw0rd sftp://sftpsiteurl.com; mirror -c -e -R -L /source-folder /destination-folder/"

The above tells LFTP to execute two commands separated by a semi-colon. The first opens up a connection to the SFTP server using the username “jdoe” and password “passw0rd”. Notice you need to specify the protocol by pre-appending “sftp://” to the url. Also I am using a non default port with the -p flag of “9922”. If you are using key authentication just put some random text where the password should be.

The next command after the semi-colon says I want to “mirror” the two directories and the flags do the following:

-c, –continue – continue a mirror job if possible
-e, –delete delete files not present at remote site
-R, –reverse reverse mirror (put files, vs. a normal mirror used for pulling files I guess??)
-L, –dereference download symbolic links as files

One final note, I think RSYNC is still preferable as I believe (not sure but I think this is correct) it can transfer byte-level changes. So if you have large files and something small changes it will only transfer the changes and not the entire file again. I also don’t know how “intelligent” LFTP is… i.e. does it just check for file names or does it do more in-depth checks like making sure file-size matches or even MD5 sums? I don’t know. So I will continue to prefer RSYNC where I can use it but I will keep LFTP handy going forward when RSYNC isn’t an option.


1 of 1

2 comments on: LFTP – An Alternative When RSYNC Is Not An Option – Mirror Directories over SFTP

  1. Jon

    Thank you this continues to be helpful

  2. Catherine Lee

    I like rsync and was a regular user of it. But alternatives of it on windows are very less and if they are there they dont seem to be useful like this software. However, after searching a lot I found GS Richcopy 360 works really well and provides all the options that rsync provided plus its multi threaded file transfer saves a lot of time while transferring file. I love it, try it hope it helps you too!

Leave a Reply