RSync back up size allowing for exclusions

We use a script to get home folder sizes individually once a day and then EAs report those sizes.

We use public/private key setup established in advance, (only 70 users), and a rsync that backups all most of home folder but excludes trash, downloads, spotify etc. over SSH to a ZFS network volume.

It's set to run daily on laptops if in to office, and then I have a smart group to warn me if it's been longer than 5 days without a backup.

We call it "Sup-Homes";-)

#!/bin/bash
# GET USER FOLDER SIZES AND WRITE TO PLIST
# Calculates storage (GB) used by a subfolder in logged-in user's homedir.
# This should be run periodically to update the folder size.

# Common JAMF Script Variables
DRIVE="$1"
COMPUTER_NAME="$2"
JAMF_USER="$3"

# Output plist and key for dictionary
RESULTS_FILE="$DRIVE/Library/Preferences/com.MYCOMPANYPREFERENCE.plist"
KEY="HomeFolderUsage"

# Folders to scan
SUBFOLDERS=(
    # Consider including entire home folder, which is not directly accessible
    '/'
    '.Trash'
    'Desktop'
    'Downloads'
    'Documents'
    'Movies'
    'Music'
    'Pictures'
)

# Attempt to auto-detect the last-logged-in user.
CONSOLE_USER=`defaults read "$DRIVE/Library/Preferences/com.apple.loginwindow.plist" lastUserName`

# Calculate size for each specified folder.
# If a meaningful end-user is not available, size is calculated from all user
# folders on the machine.
for SUBFOLDER in ${SUBFOLDERS[*]}; do
    if [ "$CONSOLE_USER" = '' ] || [ "$CONSOLE_USER" = 'CASPERUSERNAME' ]; then
        USAGE=`du -sg "$DRIVE"/Users/*/"$SUBFOLDER" | cut -f 1`
    else
        USAGE=`du -sg "$DRIVE/Users/$CONSOLE_USER/$SUBFOLDER" | cut -f 1`
    fi

    # Place findings in plist for future recovery
    defaults write "$RESULTS_FILE" "$KEY-$SUBFOLDER" "$USAGE"
done

# Display new contents for reference
# defaults read "$RESULTS_FILE"

#!/bin/bash
#SUP-HOMES BACKUP SCRIPT-V6
MOUNT_POINT="$1" # TODO Integrate into other variables
COMPUTER_NAME="$2"
# USER_NAME="$3" # only defined at login

# Convert global config from v5 to per-user config for v6
OLD_CONFIG=/Library/Preferences/com.MYCOMPANYPREFERENCE.plist
if [ -f $OLD_CONFIG ]; then
    for U in $( defaults read $OLD_CONFIG "Backup Users" ); do
        NEW_CONFIG="/Users/$U/Library/Preferences/com.MYCOMPANYPREFERENCE.plist"
        if [ -f "$NEW_CONFIG" ]; then
            echo User $U has already been converted.
            echo Consider removing the original config file from $NEW_CONFIG.
        else
            echo "Creating new entry for user $U at $NEW_CONFIG."
            defaults write "$NEW_CONFIG" "Enabled" -bool true
        fi
    done

    # TODO Once everyone is safely migrated, begin deleting the original file
    # rm "$OLD_CONFIG"
fi

# Run per-user sync
for LOCAL_HOME in /Users/*; do
    CONFIG_FILE="$LOCAL_HOME/Library/Preferences/com.MYCOMPANYPREFERENCE.plist"
    if [ -f "$CONFIG_FILE" ]; then
        START_TIME=$( date +'%Y-%m-%d %H:%M:%S' )
        defaults write "$CONFIG_FILE" "Last Run" -date "$START_TIME"

        # Extract User Name from home folder
        U=`basename $LOCAL_HOME`

        # Copy relevant files from user home to rsync folder in remote home.
        echo Starting backup for $U
        time rsync --archive --partial --numeric-ids --one-file-system --stats 
            --delete-after --delete-excluded 
            -e "ssh -i $LOCAL_HOME/.ssh/sup-homes" 
            --exclude=Downloads --exclude=.Trash --exclude=.FileSync 
            --exclude=Documents/Microsoft User Data 
            --exclude=Library/Application Support/MobileSync 
            --exclude=Library/Application Support/Adobe 
            --exclude=Library/Application Support/Google 
            --exclude=Library/Application Support/Firefox 
            --exclude=Library/Application Support/Skype 
            --exclude=Library/Application Support/iLifeAssetManagement 
            --exclude=Library/Mail 
            --exclude=Library/Logs 
            --exclude=Library/Caches 
            --exclude=Music/iTunes 
            --exclude=Library/Caches 
            --exclude=Library/Developer 
            --exclude=Dropbox 
            --exclude=Google Drive 
        "$LOCAL_HOME/" 
        $U@sup-homes:rsync/

        RSYNC_STATUS=$?

        END_TIME=$( date +'%Y-%m-%d %H:%M:%S' )
        if [[ $RSYNC_STATUS -eq 0 ]]; then
            defaults write "$CONFIG_FILE" "Last Success" -date "$END_TIME"
        fi
    fi
done

@hkabik @markkenny

rsync -an --stats suits what i need perfectly, i can run it before the copy to get the copy size, also found a nice snippet to convert it from bytes to a more palatable form.

#sure this ain't so pretty!
#bytes conversion from https://gist.github.com/mlegenhausen/9365461
syncsize=$( caffeinate rsync -an --stats --exclude-from=/var/tmp/temp_exclusions.txt /Users/bill.danforth/Desktop/READ/ /Users/bill.danforth/Desktop/Aqui | grep "Total transferred file size" | sed 's/[^0-9]*//g' | awk '{ sum=$1 ; hum[1024**3]="GB";hum[1024**2]="MB";hum[1024]="KB"; for (x=1024**3; x>=1024; x/=1024){ if (sum>=x) { printf "%.2f %s
",sum/x,hum[x];break } }}' )

echo "$syncsize"

Liking the Sup-Homes script, got me thinking about using keys instaed of prompting for the password each time.

Thanks for your help!