Disk Usage Report Self Service Policy

dan-snelson
Valued Contributor II

To aid end-users in determining where all their hard drive space has gone, we have a Self Service policy which leverages du and outputs text files of the top 75 directories of the root volume and the user's home folder. The utilitarian results are sorted by size, in gigabytes, and saved to the user's desktop.

6130ca4017b94f0e85d78a47502df686

Self Service Description

Disk Usage Report Outputs two text files to your Desktop, which includes a listing of the top 75 directories of the root volume and the top 75 directories of your user folder. The results are saved to your desktop as: Computer-DiskUsage.txt and {YourUserName}-DiskUsage.txt.

Disk Usage: Root Volume

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Disk Usage
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 8-Dec-2014, Dan K. Snelson
#
####################################################################################################
# Import logging functions
source /path/to/logging/script/logging.sh
####################################################################################################

loggedInUser=`/bin/ls -l /dev/console | /usr/bin/awk '{ print $3 }'`
loggedInUserHome=`dscl . -read /Users/$loggedInUser | grep NFSHomeDirectory: | cut -c 19- | head -n 1`
machineName=`scutil --get LocalHostName`

/bin/echo "`now` *** Calculate Disk Usage for / ***" >> $logFile

/usr/bin/du -axrg / | sort -nr | head -n 75 > $loggedInUserHome/Desktop/$machineName-ComputerDiskUsage.txt


exit 0      ## Success
exit 1      ## Failure

Disk Usage: Home Directory

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Disk Usage
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 8-Dec-2014, Dan K. Snelson
#
####################################################################################################
# Import logging functions
source /path/to/logging/script/logging.sh
####################################################################################################

loggedInUser=`/bin/ls -l /dev/console | /usr/bin/awk '{ print $3 }'`
loggedInUserHome=`dscl . -read /Users/$loggedInUser | grep NFSHomeDirectory: | cut -c 19- | head -n 1`

/bin/echo "`now` *** Calculate Disk Usage for $loggedInUserHome  ***" >> $logFile

/usr/bin/du -axrg "$loggedInUserHome" | sort -nr | head -n 75 > "$loggedInUserHome"/Desktop/"$loggedInUser"-DiskUsage.txt


exit 0      ## Success
exit 1      ## Failure

--
Dan
36 REPLIES 36

dferrara
Contributor II

This looks fantastic, I will definitely use this. Thanks for sharing it.

asegura
Contributor

@dan.snelson how did you get the end of the text bold in your self service item? I tried to use html code thinking that might do it but it didn't. Thanks for sharing this.

dan-snelson
Valued Contributor II

@asegura Take a look at: Using Markdown to Format Text


--
Dan

asegura
Contributor

@dan.snelson thanks that worked.

dferrara
Contributor II

@dan.snelson Dan, dumb question, can you explain the purpose of the source command at the start of the script? Where do you set the path?

mm2270
Legendary Contributor II

Silly question, but why did you decide to split these into 2 scripts? I assume they are both in the same policy given the Self Service description, so why not have both du commands run in one script, one after the other?

Also, for getting the loggedInUserHome, if you want, you can cut that command down to this:

dscl . -read /Users/$loggedInUser NFSHomeDirectory | awk '{print $NF}'

No need for grep, cut and head.

dan-snelson
Valued Contributor II

@dferrara The source command imports an external script (in this case, a logging script which we put on machines during imaging / enrollment).

@mm2270 Our Tier 1, 2 and 3 support personnel sometimes run this for end-users via Casper Remote and they like the flexibility of running one, the other, or both. (I like your loggedInUserHome better than the one I'm using; thanks.)


--
Dan

russeller
Contributor III

@dan.snelson Thanks for sharing. Can you elaborate on your logging script?

adamcodega
Valued Contributor

This is great, easier to do than deploying OmniDiskSweeper or the like. Thanks for the contribution!

Look
Valued Contributor III

Just created a variation on this as well, thanks for the idea.

Added this bit at the end to make it open automatically in the users browser (obviously you will need to work out the variables first).

if [ -f "/Users/$CURRENT_USER/$FILE_NAME" ]; then
su - $CURRENT_USER -c "open -a safari /Users/$CURRENT_USER/$FILE_NAME"
fi

dan-snelson
Valued Contributor II

@ssrussell Our logging script was inspired by Stack Overflow and is installed on each client during imaging or when enrollment is completed. (My personal preference is to not write anything to jamf.log so that JAMF support isn't slowed down by my nonsensical gibberish when debugging their code.)

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Standard logging functions which are imported into other scripts
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 22-Nov-2014, Dan K. Snelson
#
####################################################################################################
#
# LOGGING
#
# Logging variables
    logFile="/var/log/com.company.division.department.log"
    alias now="date '+%Y-%m-%d %H:%M:%S'"
#
# Check for / create logFile
    if [ ! -f "${logFile}" ]; then
        ### logFile not found; Create logFile
        /usr/bin/touch "${logFile}"
    fi
# Save standard output and standard error
    exec 3>&1 4>&2
# Redirect standard output to logFile
    exec 1>>"${logFile}"
# Redirect standard error to logFile
    exec 2>>"${logFile}"
#
# Example
#   /bin/echo "`now` Hello, world!" >> $logFile
#
# Enable all logging from this point forward
#   set -xv; exec 1>"${logFile}" 2>&1
#
####################################################################################################

--
Dan

dan-snelson
Valued Contributor II

@Look Nice. Simplifies the end-user finding the output.


--
Dan

rgranholm
Contributor

This is exactly what I'm looking for to help my company slim down their hard drives, we have too many users using 90% +

Is there a simple way you can think to get the results sent to someone else or back to the JAMF server?

mm2270
Legendary Contributor II

@rgranholm You could look at sending the resulting data in an email to yourself, or a dist from the command line using Unix mail.

echo "$some_data" | mail -s "My Subject" rgranholm@company.com

There are a few other more advanced options you can throw in there, but the above is the basic idea. For example, since we're talking about a very simple text (log) file, you can try attaching it using uuencode:

uuencode "${logFile}" com.company.division.department.log | mail -s "My Subject" rgranholm@company.com

Or include a message body along with the attachment:

(echo "This is my message body"; uuencode "${logFile}" com.company.division.department.log) | mail -s "My Subject" rgranholm@company.com

Edit: forgot to mention that, using the JSS API, you can also attach the log file as an attachment to the computer record in the JSS. That way anyone with access to the detailed computer records could download and view it.
And finally, there's including the report information in an EA. Meaning, pipe the results of the policy into the log file as per above, then have an EA script pick up the contents so its viewable directly within the record. Only issue with the latter approach is it may be a lot of info to have in an EA field, so you may want to try some of the other approaches.

rgranholm
Contributor

@mm2270 .. thanks! I'm a complete novice when it comes to scripting so I'm copying and pasting and testing my way there.

I'm testing adding the uuencode line to the end of the script, but it spit an error that said "uuencode: : No such file or directory....I'm unsure how to define the location of that log file.

I'm really interested in attaching the log file to the computer record, would it be too much trouble to show me how I'd need to edit the script to do this? ... the e-mail option works as well.

Thank you!

mm2270
Legendary Contributor II

@rgranholm See modified version of Dan's script from above. This is the one that does the top level disk usage, but the principle could be used for the other script as well. Keep in mind I copied his logging script into /private/var/scripts/ on my Mac, so its referenced in the source import line below. You'd need to customize that if you put it in a different location and name it something else.
The other items that need to be changed would be the emailRec variable, which would be either a single email address, or any number of email addresses separated by a comma (no spaces) And also any text in the echo that creates the body of the email.
The end result will be an email with the attachment with whatever you specify as the name, and a subject and body that reflect what machine its coming from.
See screenshot below (with some items blocked out). As you can see, it includes the computer name in both the file name as well as subject and body text. I also sent it both to my company email and my personal gmail account as seen below.

19a52418ed4f4ffea399f6613cc0af01

Here's the modified script:

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Disk Usage
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 8-Dec-2014, Dan K. Snelson
#
####################################################################################################
# Import logging functions
source /var/scripts/logging.sh
####################################################################################################

loggedInUser=$(stat -f%Su /dev/console)
loggedInUserHome=$(dscl . read /Users/$loggedInUser NFSHomeDirectory | awk '{print $NF}')
machineName=$(scutil --get LocalHostName)

/bin/echo "`now` *** Calculate Disk Usage for / ***" >> $logFile

/usr/bin/du -axrg / | sort -nr | head -n 75 > $loggedInUserHome/Desktop/$machineName-ComputerDiskUsage.txt

## var of path to log file
userLogFile="$loggedInUserHome/Desktop/$machineName-ComputerDiskUsage.txt"
logFileName="$machineName-ComputerDiskUsage.txt"

## Email recipient address
emailRec="somebody@mycompany.com"

(echo "Disk usage report attached for ${machineName}"; uuencode "$userLogFile" "$logFileName") | mail -s "Disk Usage Report - ${machineName}" "$emailRec"

Few other items: you can also add both cc's and bcc's to any email by doing something like:

-c somebody2@company.com -b somebody3@company.com

The -c is CC and -b is BCC

One last thing on the unix email option. YMMV on whether it will actually work. I've seen some environments that block sending email from the command line, and the email ends up stuck in the local machine's outbox. So you'll need to test this under as many conditions as you can to ensure it works.

As for attaching to the computer record in the JSS, give me a few and I'll post some code to do that as well. In the interim, feel free to search around on here because I know there are existing examples of using the API to attach a file. In fact, I think I may have already posted an example on another older thread somewhere on how to do that.

rgranholm
Contributor

Hmm immediately fails with uuencode: /Users/rgranholm/Desktop/RFG-MbProR15-ComputerDiskUsage.txt: No such file or directory

Seems as though it tries to execute uuencode first, I'll keep tinkering

rgranholm
Contributor

Ah nevermind, saw the comments out line for generating the .txt file, removed that and made progress.

mm2270
Legendary Contributor II

Oops, yeah, I did comment that out toward the end and forgot to remove it before posting. Sorry for the confusion. Post above updated.

Edit: To be clear though, you should only remove the comment hash sign, not the entire line, since that actually does the bulk of the work of generating the report file.

dan-snelson
Valued Contributor II

@rgranholm: Thanks for the idea …@mm2270: Thanks for pulling it off; I'll have to try it out.


--
Dan

rgranholm
Contributor

No problem, it executed, but no email was received.

I've been testing it by executing it as a shell .sh file in terminal, you mentioned some environments wouldn't play nice with e-mail so I'll look into that next.

The terminal gave me the below output, and several more denied folders...but otherwise produced the text file.

RFG-MbProR15:Desktop rgranholm$ sh diskusage.sh 
diskusage.sh: line 23: now: command not found
diskusage.sh: line 23: $logFile: ambiguous redirect
du: /.DocumentRevisions-V100: Permission denied
du: /.fseventsd: Permission denied
du: /.Spotlight-V100: Permission denied
du: /.Trashes: Permission denied

dan-snelson
Valued Contributor II

@rgranholm Since the JAMF binary executes scripts as root, you may want to elevate your permissions in your shell to simulate how the JSS will execute scripts.


--
Dan

rgranholm
Contributor

Thanks Dan, I'll give that a try now.

mm2270
Legendary Contributor II

@rgranholm As Dan mentioned, you need to run the script as root since du is going to need to scan a bunch of locations that may otherwise have permissions protections on them.
Plus, it needs to be able to both create and write to the log file, which I think by default is going into a normally root protected location. I was not able to run the script normally in my user context and have it work. Only when I put sudo in front of it did it work.

Also, that ambiguous redirect makes me think you didn't set up the logging script that @dan.snelson posted up earlier in the thread. Its going to try to reference that for logging purposes and if it can't locate it it won't really work.

Hope that helps.

rgranholm
Contributor

Hmm using sudo eliminated all the permission issues.

I put a blank txt file in the var/scripts location named logging.sh see here: http://take.ms/VSGjo

Howver I still do get the below

diskusage.sh: line 23: now: command not found
diskusage.sh: line 23: $logFile: ambiguous redirect

dan-snelson
Valued Contributor II

@rgranholm Comment out (#) my lame logging functions:

# source /var/scripts/logging.sh

… and …

# bin/echo "`now` *** Calculate Disk Usage for / ***" >> $logFile

--
Dan

mm2270
Legendary Contributor II

@rgranholm You have to copy the script located at this post above into a place like /var/scripts/ or whatever location makes sense for you. If you put a blank file there its not going to be able to load the script for logging.

Once that's there, just update my script above where it says:

source /var/scripts/logging.sh

to make sure the location and name of the script you saved from the step above match. Dan is using one script called something like "logging.sh" to do the log functions, which is loaded within the second script that does the disk usage information stuff by calling it with the source line. Does that make sense?

rgranholm
Contributor

It does, and it worked, the log was sparse online after running, but it produced a log file in the location.

2016-04-14 15:58:35 Calculate Disk Usage for /

Thanks

mm2270
Legendary Contributor II

Posting an updated version that includes uploading the final log as an attachment to the JSS Computer record. It determines the JSS ID of the machine its running on and uses that to upload the file.

You need to use an API account that has read and write privileges to computer objects, and perhaps a few other things. Not 100% sure of the required privs since I used my all purpose API account for testing, which has access to everything.

Obviously you can just remove the whole upload attachment section, or the send email section and use whichever one you want, or keep both. When I ran a test, I got an email with the attachment AND saw the new attachment in my computer record in our JSS that I could download and view.

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Disk Usage
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 8-Dec-2014, Dan K. Snelson
#
####################################################################################################
# Import logging functions
source /var/scripts/logging.sh
####################################################################################################

loggedInUser=$(stat -f%Su /dev/console)
loggedInUserHome=$(dscl . read /Users/$loggedInUser NFSHomeDirectory | awk '{print $NF}')
machineName=$(scutil --get LocalHostName)
dateString=$(date +"%Y-%m-%d")            ## Added date string for use with the local output filename

/bin/echo "`now` *** Calculate Disk Usage for / ***" >> $logFile

/usr/bin/du -axrg / | sort -nr | head -n 75 > $loggedInUserHome/Desktop/$machineName-ComputerDiskUsage-${dateString}.txt       ## Changed file name to use date string at end

############################# Section for emailing log as attachment ###############################

## var of path to log file
userLogFile="$loggedInUserHome/Desktop/$machineName-ComputerDiskUsage-${dateString}.txt "      ## Changed file reference to reflect the date in the file name being generated
logFileName=$(basename "$userLogFile")

## Email recipient address (customize this address!)
emailRec="somebody@mycompany.com"

(echo "Disk usage report attached for ${machineName} - $(date)"; uuencode "$userLogFile" "$logFileName") | mail -s "Disk Usage Report - ${machineName}" "$emailRec"


##################### Section for uploading log as attachment using the API ########################

## API Username & Password (account needs write privs to Computer objects)
## You can also pass the API Username and Password as script parameters in the policy ($4 and $5) instead of hardcoding them in (more secure)
apiuser="apiuser"
apipass="apipass"

## Get the JSS URL from the Mac the script is running on
jssURL=$(defaults read /Library/Preferences/com.jamfsoftware.jamf.plist jss_url | sed 's|/$||')

## Pull ioreg data on the Mac
ioregData=$(ioreg -rd1 -c IOPlatformExpertDevice)

## Get the Mac's Serial Number and UUID
SerialNum=$(awk -F'"' '/IOPlatformSerialNumber/{print $4}' <<< "$ioregData")
UUID=$(awk -F'"' '/IOPlatformUUID/{print $4}' <<< "$ioregData")

## Determine which one to use to pull the Mac's JSS ID
if [[ -z "$SerialNum" ]] && [[ ! -z "$UUID" ]]; then
    sourceID="$UUID"
    sourceString="uuid"
elif [[ ! -z "$SerialNum" ]]; then
    sourceID="$SerialNum"
    sourceString="serialnumber"
fi

## Get the Mac's JSS ID via API call
jssID=$(curl -H "Accept: text/xml" -sfku "${apiuser}:${apipass}" "${jssURL}/JSSResource/computers/${sourceString}/${sourceID}/subset/general" | xmllint --format - | awk -F'>|<' '/<id>/{print $3; exit}')

## Upload attachment via API, using the JSS Computer ID
curl -fku "${apiuser}:${apipass}" "${jssURL}/JSSResource/fileuploads/computers/id/${jssID}" -F name=@"${userLogFile}" -X POST

EDIT: Made a slight change to the local output file name to include a date string. Reason: If you run this multiple times per machine, before all uploading files would have an identical name, with no way of easily knowing which was the most recent. The date string being part of the file name now differentiates them.

rgranholm
Contributor

Awesome, time to test it out! Thanks so much for all this help, it's going to be incredibly useful for reporting users who are abusing the space on their computers for personal content.

I'm completely new to the API, if I'm a JSS admin, can I assume my username/password will work for the API username and password?

Where should I start to dig if the e-mail continues to fail?

Checking for policies triggered by "recurring check-in" for user "rgranholm"...
Executing Policy TEST RFG - Disk Usage Report
Running script disk_usage_report.sh...
Script exit code: 26
Script result: 
Error running script: return code was 26.
Submitting log to https://jss.insigniam.com:8443/

stevewood
Honored Contributor II

Also, if you are bound to AD and maintain the email address field in AD, you can use this to capture that address to send the report:

emailRec=$(dscl . read /Users/$loggedInUser EMailAddress | awk '{print $NF}')

rgranholm
Contributor

Thanks, for our purposes, sending it one static department address will work, but I have no idea how to get the e-mail to actually send, does anyone know what the error code 26 refers to?

mm2270
Legendary Contributor II

@rgranholm Wish I could help, but not really sure where the issue would be. I don't know what error 26 means. I'm not even sure its related to sending the mail, but I assume it is.

Have you double checked everything in the script you're working with to make sure its correct? If I use the one I posted above and simply change a few variables to make sure they are valid, it works perfectly for me.

I did have one other thought for you. Can you try just manually sending yourself an email from the command line? Something like this?

echo "This is the message" | mail -s "This is the subject" youremailaddress@company.com

See if that goes thru OK and you receive it. If it does, then it may be something around the attachment causing the error. If that errors too, then its just sending mail in general that would be the issue.

rgranholm
Contributor

Thanks, yeah no e-mail with that either, but no error. Thanks for the help this far, looks like I'll have to dig into some materials and work through it, part of the job eh! Thank you.

sean
Valued Contributor

FWIW, we use launchd to handle this and set a threshold value, e.g. 85%. We have a separate Users drive, but it could be adapted.

To get percentage used:

df /Users |  awk  '/dev/disk/  {print substr($5,1,length($5) - 1)}'

As a LaunchAgent you can just use $HOME, so you don't have to worry about who is logged in or how to obtain
their home path.

/usr/bin/du -axrg "$HOME" | sort -nr | head -n 75

With this method, you aren't waiting for the user to completely fill their disk and then work out why it is full. Instead, with the periodic nature of launchd, it can help prompt the user to make sure they don't let it get too full in the first place.

Regarding email, this may be your mail server. No error is just saying the command ran correctly, but it will have no idea if the email server received the mail or did anything with it. Did you check your junk folder. Mails like this can easily become junked.

dan-snelson
Valued Contributor II

Happy New Year!

I just now noticed the verbiage has changed in macOS 10.12 (i.e., "Volume Free Space:" vs. "Volume Available Space:").

Here's an updated version, sans some of the racing stripes from above.

#!/bin/sh
####################################################################################################
#
# ABOUT
#
#   Disk Usage: Home Directory
#
####################################################################################################
#
# HISTORY
#
#   Version 1.0, 8-Dec-2014, Dan K. Snelson
#   Version 1.1, 8-Jun-2015, Dan K. Snelson
#       See: https://jamfnation.jamfsoftware.com/discussion.html?id=14701
#   Version 1.2, 4-Jan-2017, Dan K. Snelson
#       Updated for macOS 10.12
#
####################################################################################################
# Import logging functions
source /path/to/client-side/logging/script/logging.sh
####################################################################################################


# Variables
loggedInUser=$(/usr/bin/stat -f%Su /dev/console)
loggedInUserHome=$(/usr/bin/dscl . -read /Users/$loggedInUser NFSHomeDirectory | /usr/bin/awk '{print $NF}') # mm2270
machineName=$(/usr/sbin/scutil --get LocalHostName)
volumeName=$(/usr/sbin/diskutil info / | grep "Volume Name:" | awk '{print $3,$4}')

osMinorVersion=$(/usr/bin/sw_vers -productVersion | /usr/bin/cut -d. -f2)

    if [[ "$osMinorVersion" -lt 12 ]]; then
        availableSpace=$(/usr/sbin/diskutil info / | grep "Volume Free Space:" | awk '{print $4}')
        totalSpace=$(/usr/sbin/diskutil info / | grep "Total Size:" | awk '{print $3}')
    else
        availableSpace=$(/usr/sbin/diskutil info / | grep "Volume Available Space:" | awk '{print $4}')
        totalSpace=$(/usr/sbin/diskutil info / | grep "Volume Total Space:" | awk '{print $4}')
    fi

percentageAvailable=$(/bin/echo "scale=1; ($availableSpace / $totalSpace) * 100" | bc)
outputFileName="$loggedInUserHome/Desktop/$loggedInUser-DiskUsage.txt"


# Output to log
ScriptLog "### Disk usage for "$loggedInUserHome" ###"
ScriptLog "* Available Space:  $availableSpace GB"
ScriptLog "* Total Space:      $totalSpace GB"
ScriptLog "* Percentage Free:  $percentageAvailable%"


# Output to user
/bin/echo "--------------------------------------------------" > $outputFileName
/bin/echo "`now` Disk usage for "$loggedInUserHome"" >> $outputFileName
/bin/echo "* Available Space:  $availableSpace GB" >> $outputFileName
/bin/echo "* Total Space:      $totalSpace GB" >> $outputFileName
/bin/echo "* Percentage Free:  $percentageAvailable%" >> $outputFileName
/bin/echo "--------------------------------------------------" >> $outputFileName
/bin/echo " " >> $outputFileName
/bin/echo "GBs Directory or File" >> $outputFileName
/bin/echo " " >> $outputFileName
#/usr/bin/du -axrg / | /usr/bin/sort -nr | /usr/bin/head -n 75 >> $outputFileName
/usr/bin/du -axrg $loggedInUserHome | /usr/bin/sort -nr | /usr/bin/head -n 75 >> $outputFileName


if [ -f $outputFileName ]; then
    /usr/bin/su - $loggedInUser -c "open -a safari $outputFileName" # thanks, Samuel Look
fi

exit 0      ## Success
exit 1      ## Failure

--
Dan