lftp mirroring for distribution points replication

[Eigger]

Hi there jamfnationers! A few weeks ago, I discovered lftp, super awesome small utility. Pretty much like rsync in steroids. What I like the most about this utility is its ability to split files for faster transfer and continue interrupted file tranfers. I adopted some scripts from the internet to use it for my distributions points. I'm not really good at scripting, but I really like to learn more, because its fun!

lftp -u casper-rw -e "set sftp:connect-program 'ssh -a -x -T -o Compression=no'; mirror -v -c --delete --use-cache --loop --use-pget-n=5 -P 5 /Volumes/ServerHD/CasperAFPShare/Packages /Volumes/ServerHD/CasperAFPShare/Packages; quit" sftp://0.0.0.0:22.

lftp-u is the user name for the DP. connect using sftp, Compression no to save cpu resources, mirror -v (verbose) -c (continue interrupted) --delete (delete files thats not in the source) --use-cache (it uses local cache instead of scanning the remote server on the next session) --loop (keep mirroring until no more changes) --use-pget-n=5 (number of pieces to split the file) -P 5 (number of files to transfer in parallel) total of 25 TCP connections, Source then Target and finally the DP address and port.

Here's the original http://www.commandlinefu.com/commands/view/13759/fastest-segmented-parallel-sync-of-a-remote-directory-over-ssh

Here's from the man page
https://lftp.yar.ru/lftp-man.html

For now I can run this manually from the terminal, but my intention is to use the script above and to be able to put read in paramaters for User Name, Password, Distribution Point (IP or DNS name) Set Mirror for lftp to try to split up files in pieces for parallel downloading, Number of files to download simultaneously and finally output a Log File in JSS or the server itself after syncing or better yet send an email. I would like to push this as a policy for my enrolled DP's and use the frequency to sync from the Master DP on specified time of day everyday. For me, this is a lot better than rsync solution posted by jamf to replicate distribution points if we can make this work. Heck probably even reliable and faster than JDS.

[Eigger]

#!/bin/bash
# dpsync.sh
# Master Distribution Point Server and Credentials
master_dp="ip_address"
user=admin
pass='password'

# Master DP Casper Share Path
remote_dir="/path/of/casper_share/"

# Remote DP Casper Share Path
local_dir="/path/of/casper_share"

echo "Replicating Share from Master DP"
# call lftp, replicate casper share from master dp
lftp -u $user,$pass -e "set sftp:connect-program 'ssh -a -x -T'; mirror -v --log=/var/log/dpsync.log -c --use-cache --delete --only-newer --loop --use-pget-n=5 -P 5  $remote_dir  $local_dir; quit" sftp://$master_dp:22

Getting close, I just need to test it out running this using JSS Policy on my distribution servers. My idea is to make static group of my dp's, run the script using policy with recurring check in or custom event, or just use cron instead of the JSS. I will also use ssh generated keys so I dont need to put the password in the script. I hope someone can help me improve this or just let me know if there's a better solution besides JDS. Thanks in advance.