Skip to main content
Solved

Uploading audit files via API


AVmcclint
Forum|alt.badge.img+21

We have an annual PCI audit that we have to run on randomly selected Macs once a year. After the Macs have been selected, I scope those Macs to a policy that runs the script below. Everything works up to the point of uploading the resulting .zip file to the computer record on the Jamf server. I keep getting a "curl: (26) Failed to open/read local data from file/application" error. Unfortunately my curl skills are weak and I can't figure out what I'm doing wrong. I think I'm doing something wrong with the -F option or I'm not using the variables correctly in the curl command (line 91).

#!/bin/bash # this script s used to collect a full system profiler dump in txt format # as well as user accounts, groups, crowdstrike info, open ports # it will put all the files in /Library/company/PCI/<computername>/ # for later retrieval and submission for PCI audit # Set variables for the JAMF API jamf_url="https://company.jamfcloud.com" jamf_user="API-only-account" jamf_password="password_here" computer_id=$(jamf recon | grep '<computer_id>' | awk -F'[<>]' '{print $3}') echo $computer_id SERIAL=`ioreg -l | awk '/IOPlatformSerialNumber/ { print $4;}' | sed 's/"//g'` echo $SERIAL PCI_PATH="/Library/company/PCI/$SERIAL" timestamp=$(date "+%Y.%m.%d") year=$(date "+%Y") auditlog="$SERIAL-$timestamp.zip" echo $auditlog # if there is already a folder in the target location (probably from an earlier capture) this will delete it if [[ -d /Library/company/PCI/$SERIAL ]] ; then rm -Rf /Library/company/PCI/$SERIAL fi # Creates the ./PCI/<serial number> folder mkdir -p $PCI_PATH # Local user accounts # dscl . list /Users | grep -v '_' > /Library/company/PCI/$SERIAL/User.txt # All accounts (human and system) and Directory with Name # dscacheutil -q user > /Library/company/PCI/$SERIAL/UserDirectory.txt # Groups # dscacheutil -q group > /Library/company/PCI/$SERIAL/groups.txt # All user and system accounts that do not start with _ dscacheutil -q user | perl -00ne 'print if !/name: _/' > $PCI_PATH/Users.txt echo "user and system accounts saved " # All groups that do not start with _ dscacheutil -q group | perl -00ne 'print if !/name: _/' > $PCI_PATH/Groups.txt echo "all non invisible groups saved" # Generates a plain text list of applications and version info system_profiler SPApplicationsDataType > $PCI_PATH/Applications.txt echo "list of all applications saved" # Crowdstrike Status /Applications/Falcon.app/Contents/Resources/falconctl stats > $PCI_PATH/crowdstrike.txt echo "Crowdstrike status saved" # Lists Services in a similar manner to what is listed in the computer record in Jamf cat << 'EOT' > /Library/company/PCI/$SERIAL/SERVICES_NOTE.txt the list of services was generated by the following command: /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' EOT echo "" > $PCI_PATH/Services.txt /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' >> $PCI_PATH/Services.txt echo "list of services saved" # full system profiler dump system_profiler -xml -detailLevel full > $PCI_PATH/system_profiler_full.xml echo "The system_profiler_full.xml file can be renamed with .spx extension and can then be opened directly with the System Information.app found in macOS of all versions" > /Library/company/PCI/$SERIAL/SYSTEM_PROFILER_NOTE.TXT echo "System Profiler dump saved " # Open ports listening lsof -i -P | grep -i "listen" > $PCI_PATH/openports.txt echo "list of open ports saved" # evidence of time sync touch /var/db/ntp-kod chmod 666 /var/db/ntp-kod sntp -sS time.apple.com > $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "commands used to generate this data: touch /var/db/ntp-kod ; chmod 666 /var/db/ntp-kod ; sntp -sS time.apple.com" >>$PCI_PATH/timesync.txt echo "time sync info saved" # Zips up all the files generated in the /PCI/<serial number>/ folder zip -r $auditlog $PCI_PATH # Upload file to computer record using JAMF API curl -X POST -u "${jamf_user}:${jamf_password}" "${jamf_url}/JSSResource/fileuploads/computers/id/${computer_id}" -F name=@auditlog -F file=@$PCI_PATH-$timestamp.zip echo "Evidence gathered for the $year Audit. Done"

 Can someone a little more curl-savvy point out where I went wrong?

Best answer by AVmcclint

SOLVED!! It turns out there were a couple things working against me on this.

1. I had some issues with the name= part of the curl command. To simplify it I copied the zip file to a known location "/tmp/" and then used a single variable for the file path. It turned out to be easier than trying to layer multiple variables into the curl command. This solved the curl 26 error.

2. I thought I had issues with using basic authentication so I decided it was time I jump in with Bearer Tokens anyway. Turns out that basic auth wasn't the problem (because I could basic authenticate with my global admin account), but I needed to move forward anyway.

3. I had to do some deep digging to discover that the API-only account I am using needed CREATE rights on Computers in addition to all the other obvious rights. This solved the http error I was getting.

Here is the final working (and obfuscated) script that I verified in CodeRunner, Terminal, and Self Service.

 

#!/bin/bash # this script is used to collect a full system profiler dump in txt format # as well as user accounts, groups, crowdstrike info, open ports # it will put all the files in /Library/company/PCI/<computername>/ # for later retrieval and submission for PCI audit # Set variables for the JAMF API jamf_url="https://company.jamfcloud.com" jamf_user="APIuser" jamf_password="APIpassword" #Variable declarations bearerToken="" tokenExpirationEpoch="0" getBearerToken() { response=$(curl -s -u "$jamf_user":"$jamf_password" "$jamf_url"/api/v1/auth/token -X POST) bearerToken=$(echo "$response" | plutil -extract token raw -) tokenExpiration=$(echo "$response" | plutil -extract expires raw - | awk -F . '{print $1}') tokenExpirationEpoch=$(date -j -f "%Y-%m-%dT%T" "$tokenExpiration" +"%s") } checkTokenExpiration() { nowEpochUTC=$(date -j -f "%Y-%m-%dT%T" "$(date -u +"%Y-%m-%dT%T")" +"%s") if [[ tokenExpirationEpoch -gt nowEpochUTC ]] then echo "Token valid until the following epoch time: " "$tokenExpirationEpoch" else echo "No valid token available, getting new token" getBearerToken fi } checkTokenExpiration computer_id=$(jamf recon | grep '<computer_id>' | awk -F'[<>]' '{print $3}') echo "Jamf computer ID: $computer_id" # Path and timestamp variables SERIAL=`ioreg -l | awk '/IOPlatformSerialNumber/ { print $4;}' | sed 's/"//g'` echo "serial number: $SERIAL" PCI_PATH="/Library/company/PCI/$SERIAL" echo "Path to log file: $PCI_PATH" timestamp=$(date "+%Y.%m.%d") year=$(date "+%Y") auditlog="$SERIAL-$timestamp.zip" echo "log file name: $auditlog" logpath=/Library/company/PCI/$auditlog echo "the full path to the zip file is $logpath" # if there is already a folder in the target location (probably from an earlier capture) this will delete it if [[ -d /Library/company/PCI/$SERIAL ]] ; then rm -Rf /Library/company/PCI/$SERIAL fi # Creates the ./PCI/<serial number> folder mkdir -p $PCI_PATH #### START OF AUDIT COLLECTION #### # All user and system accounts that do not start with _ dscacheutil -q user | perl -00ne 'print if !/name: _/' > $PCI_PATH/Users.txt echo "user and system accounts saved " # All groups that do not start with _ dscacheutil -q group | perl -00ne 'print if !/name: _/' > $PCI_PATH/Groups.txt echo "all non invisible groups saved" # Generates a plain text list of applications and version info system_profiler SPApplicationsDataType > $PCI_PATH/Applications.txt echo "list of all applications saved" # Crowdstrike Status /Applications/Falcon.app/Contents/Resources/falconctl stats > $PCI_PATH/crowdstrike.txt echo "Crowdstrike status saved" # Lists Services in a similar manner to what is listed in the computer record in Jamf cat << 'EOT' > /Library/company/PCI/$SERIAL/SERVICES_NOTE.txt the list of services was generated by the following command: /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' EOT echo "" > $PCI_PATH/Services.txt /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' >> $PCI_PATH/Services.txt echo "list of services saved" # full system profiler dump system_profiler -xml -detailLevel full > $PCI_PATH/system_profiler_full.xml echo "The system_profiler_full.xml file can be renamed with .spx extension and can then be opened directly with the System Information.app found in macOS of all versions" > /Library/company/PCI/$SERIAL/SYSTEM_PROFILER_NOTE.TXT echo "System Profiler dump saved " # Open ports listening lsof -i -P | grep -i "listen" > $PCI_PATH/openports.txt echo "list of open ports saved" # evidence of time sync touch /var/db/ntp-kod chmod 666 /var/db/ntp-kod sntp -sS time.apple.com > $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "commands used to generate this data: touch /var/db/ntp-kod ; chmod 666 /var/db/ntp-kod ; sntp -sS time.apple.com" >>$PCI_PATH/timesync.txt echo "time sync info saved" # Zips up all the files generated in the /PCI/<serial number>/ folder cd /Library/company/PCI/ zip -rj $auditlog $PCI_PATH echo "Zip file created" # copy the zip file to an easier path to insert in the curl command cp $logpath /tmp/ echo "uploading zip file to jamf server" # Upload file to computer record using JAMF API curl -H "Authorization: Bearer ${bearerToken}" $jamf_url/JSSResource/fileuploads/computers/id/$computer_id -F name=@/tmp/$auditlog -X POST echo "Evidence gathered for the $year Audit. Done"

 

I am so glad JamfNation exists. Even if I don't get the exact solution I need many times the replies I get spark something in my brain to get me thinking about other possibilities. I am also glad I have CodeRunner. This app has made script writing so much easier as far as finding where things break. I am glad that bash -x script.sh is a thing to let me see every step of the script in action as it runs. And I am glad removing the -s option for curl lets it tell me what's happening (sometimes).

I am pretty sure there are better ways of accomplishing this task of gathering lots of system information in various text files, zipping them up, and uploading to JamfPro, but this finally works for me.

View original
Did this topic help you find an answer to your question?

19 replies

sdagley
Forum|alt.badge.img+25
  • Jamf Heroes
  • 3536 replies
  • March 8, 2023

@AVmcclint Have you tried running the command without the 

 -F name=@auditlog

parameter? The Jamf Pro API may only want the file parameter. (And here's an example of using the API to upload an attachment if you haven't already seen it: https://github.com/kc9wwh/logCollection/blob/master/logCollection.sh)

You should update the script to use Bearer Token auth since simple auth is deprecated.


Forum|alt.badge.img+12
  • Contributor
  • 37 replies
  • March 8, 2023

I have a similar script that only uses "name=", not "file=". Like this:

curl -su $JAMF_PRO_USERNAME:$JAMF_PRO_PASSWORD $JAMF_PRO_URL/JSSResource/fileuploads/computers/id/$COMPUTER_ID -X POST -F name=@/tmp/somefile.zip

This script is from a year or so ago, but should still work. You should probably update your scripts to use modern authentication, though, since basic auth will go away at some point.


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 8, 2023

I know I need to update the authentication, but right now my focus is just figuring out how to upload files.

As for trying it without the name= or file= I have tried both (alternately) with the same error. The name of the zip file that is created will be unique to every Mac the script is run on so that's why I'm trying to use variables. Otherwise I would specify an exact filename.

For experimental purposes I changed it to -F name=@/exact/path/to/file.zip and it still failed with the same error.

I'll have to take a look at that logColleciton.sh script when I have some time.


AJPinto
Forum|alt.badge.img+26
  • Legendary Contributor
  • 2716 replies
  • March 8, 2023

Commenting as I am interested to see how this sorts out :). 

 

I thought JAMF deprecated the ability to upload files (attachments) directly to JAMF like this? I am reaching but something was mentioned about this in my JAMF 300 a couple of years ago.


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 8, 2023
AJPinto wrote:

Commenting as I am interested to see how this sorts out :). 

 

I thought JAMF deprecated the ability to upload files (attachments) directly to JAMF like this? I am reaching but something was mentioned about this in my JAMF 300 a couple of years ago.


Oh I hope you're wrong. There's an Attachments section in the computer record that I presumed was for exactly this purpose.


mm2270
Forum|alt.badge.img+16
  • Legendary Contributor
  • 7880 replies
  • March 8, 2023
AVmcclint wrote:

Oh I hope you're wrong. There's an Attachments section in the computer record that I presumed was for exactly this purpose.


Try this variation.

  1. Use name=@ instead of file=@
  2. Double quoting the zip file variable name (Note the open quote comes after the @ symbol and closes at the end of the variable name)
  3. Put the -X POST at the end of the command. This may not make any difference, and it shouldn't, but it's how my own API file upload script is set up, and it seems to work for me.

 

curl -u "${jamf_user}:${jamf_password}" "${jamf_url}/JSSResource/fileuploads/computers/id/${computer_id}" -F name=@auditlog -F name=@"$PCI_PATH-$timestamp.zip" -X POST

 


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 8, 2023

@mm2270 I tried what you suggested and it still failed with the same error. I remembered that I can get help on api calls by going to company.jamfcloud.com/api so I did and looked up the fileuploads section. I removed all the quotes and curly braces and stuff and let the variables sit as $variable in the command. I then ran it as bash -x script.sh so I could watch all the output. It still failed, but the -x option confirmed that my variables are correct... it looks like this: 

curl -k -u API-only-account:password_here https://company.jamfcloud.com/JSSResource/fileuploads/computers/id/1334 -F name=@/Library/company/PCI/SERIALNUMBER/SERIALNUMBER-2023.03.08.zip -X POST

Which according to the sample command, is formatted exactly how it should be.  However it still gives the same "

curl: (26) Failed to open/read local data from file/application" error.  My searches for that curl error find a wide variety of issues, but I can't seem to find what specifically is the problem here.

 


mm2270
Forum|alt.badge.img+16
  • Legendary Contributor
  • 7880 replies
  • March 8, 2023
AVmcclint wrote:

@mm2270 I tried what you suggested and it still failed with the same error. I remembered that I can get help on api calls by going to company.jamfcloud.com/api so I did and looked up the fileuploads section. I removed all the quotes and curly braces and stuff and let the variables sit as $variable in the command. I then ran it as bash -x script.sh so I could watch all the output. It still failed, but the -x option confirmed that my variables are correct... it looks like this: 

curl -k -u API-only-account:password_here https://company.jamfcloud.com/JSSResource/fileuploads/computers/id/1334 -F name=@/Library/company/PCI/SERIALNUMBER/SERIALNUMBER-2023.03.08.zip -X POST

Which according to the sample command, is formatted exactly how it should be.  However it still gives the same "

curl: (26) Failed to open/read local data from file/application" error.  My searches for that curl error find a wide variety of issues, but I can't seem to find what specifically is the problem here.

 


@AVmcclint what version of Jamf Pro are you running? Just curious, as I may try this on a more recent version to check that nothing broke (or changed) with that API resource. I hope it still works to upload file attachments this way, or one of my workflows is going to be broken.


Forum|alt.badge.img+12
  • Contributor
  • 37 replies
  • March 8, 2023

I can confirm that the file upload has not been disabled in 10.44.1. I did, however, need to update my script to use modern auth. 

This is my curl call:

curl -X POST -H "Authorization: Bearer $TOKEN" $JAMF_PRO_URL/JSSResource/fileuploads/computers/id/$COMPUTER_ID -F name=@"$FULL_FILE_PATH"

$TOKEN can be pulled using basic auth like this:

TOKEN=$(curl -X POST -H "Content-Type: application/json" -u "$USERNAME:$PASSWORD" "$JAMF_PRO_URL/api/v1/auth/token" | jq -r '.token')

 

Is it possible your API username or password contains bash-unfriendly characters such as "!"? This has led to confusing error message for me in the past. zsh generally handles string substitution better than bash, but my personal preference is to avoid wonky characters in the first place.

Also worth noting is that I keep my file paths free of spaces and other special characters as well. I'm not entirely sure if curl handles that properly. I assume it would with quotes but I have not tested.


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 9, 2023

We are running JamfPro 10.44.1.

After bashing my head against this for a while, I have come to the conclusion that I may not even be getting through the front door despite the error referring to "local data". I have never done the token authentication before, so I decided to tackle that. I tried your code, @mjhersh but the jq command is not recognized. I see that is not included with the macOS by default and would need to be a 3rd party install. I need this to work out of the box. I'm not using any special characters and have been very careful to not have spaces in any paths.

I can say that I'm getting closer. I found this article https://developer.jamf.com/jamf-pro/recipes/bearer-token-authorization that tells me how to generate a token.  I can say I'm no longer getting curl errors.  However, when I check the Attachments section of the computer record, there's nothing there.

Here is the final (obfuscated) curl command that is being sent:

curl -s -H 'Authorization: Bearer longencodedgibberishtokentextlongencodedgibberishtokentextlongencodedgibberishtokentext' https://company/JSSResource/fileuploads/computers/id/1334 -F name=@/Library/company/PCI/SERIALNUMBER/SERIALNUMBER-2023.03.09.zip -X POST

There are no errors after that command is sent and there is no sign of success either. 


mm2270
Forum|alt.badge.img+16
  • Legendary Contributor
  • 7880 replies
  • March 9, 2023
AVmcclint wrote:

We are running JamfPro 10.44.1.

After bashing my head against this for a while, I have come to the conclusion that I may not even be getting through the front door despite the error referring to "local data". I have never done the token authentication before, so I decided to tackle that. I tried your code, @mjhersh but the jq command is not recognized. I see that is not included with the macOS by default and would need to be a 3rd party install. I need this to work out of the box. I'm not using any special characters and have been very careful to not have spaces in any paths.

I can say that I'm getting closer. I found this article https://developer.jamf.com/jamf-pro/recipes/bearer-token-authorization that tells me how to generate a token.  I can say I'm no longer getting curl errors.  However, when I check the Attachments section of the computer record, there's nothing there.

Here is the final (obfuscated) curl command that is being sent:

curl -s -H 'Authorization: Bearer longencodedgibberishtokentextlongencodedgibberishtokentextlongencodedgibberishtokentext' https://company/JSSResource/fileuploads/computers/id/1334 -F name=@/Library/company/PCI/SERIALNUMBER/SERIALNUMBER-2023.03.09.zip -X POST

There are no errors after that command is sent and there is no sign of success either. 


Can you, or have you run the command without the -s flag for curl, to see if it's spitting back any specific error that may help diagnose the problem?

I also throw in the silent flag when using curl, because I normally don't want to see all that traffic chatter, but in this case, it may be worth seeing that information. A successful curl command to the API usually returns some information to indicate it worked.


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 9, 2023

When I remove the -s option, the error comes back. error 26. I guess I'm no closer than I was before. I've run the script in CodeRunner, Terminal, and even threw it into Jamf to run in Self Service - all with the same result.  The permission on the local directory is 755 with root as the owner. The zip file that is created has 644 permissions also with root as the owner. The script is being run as root in all cases.


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • March 9, 2023

I tried something else to eliminate the possibility of my path variables being the problem. I changed the 

name=@"$logpath" part to specify an exact file on my desktop: name=@/users/me/desktop/file.txt 

This time I got something!

<html> <head> <title>Status page</title> </head> <body style="font-family: sans-serif;"> <p style="font-size: 1.2em;font-weight: bold;margin: 1em 0px;">Unauthorized</p> <p>The request requires user authentication</p> <p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2">here</a>.<br> Please continue your visit at our <a href="/">home page</a>. </p> </body> </html>

I did some more digging and I figured, let me see what happens if I use MY credentials as a Global Admin. If any account is going to work, it'll be mine. IT WORKED!  While riding that surge of endorphins I figured I'd change the name= back to using the $logpath variable. It failed with that curl error 26. I figured there must be something up with a pure variable name so I added a cp $logpath /tmp/  right after the zip file creation and before the curl command. Then I changed name= to name=@/tmp/$auditlog and it worked! I see the files uploaded to the Attachments section of the computer record. Now the real question is why isn't the server account I created solely for API working? 

The account has Full Access, Custom, Enabled. set under the Account tab.

On the Privileges Tab > Jamf Pro Server Objects:

Advanced Computer search READ

API Integrations CREATE READ UPDATE

Computers READ UPDATE

File Attachments CREATE READ UPDATE

But it still gives that darned "Unauthorized" error. Do I have to do something to the account so it can be allowed to use bearer tokens?

 


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • Answer
  • March 9, 2023

SOLVED!! It turns out there were a couple things working against me on this.

1. I had some issues with the name= part of the curl command. To simplify it I copied the zip file to a known location "/tmp/" and then used a single variable for the file path. It turned out to be easier than trying to layer multiple variables into the curl command. This solved the curl 26 error.

2. I thought I had issues with using basic authentication so I decided it was time I jump in with Bearer Tokens anyway. Turns out that basic auth wasn't the problem (because I could basic authenticate with my global admin account), but I needed to move forward anyway.

3. I had to do some deep digging to discover that the API-only account I am using needed CREATE rights on Computers in addition to all the other obvious rights. This solved the http error I was getting.

Here is the final working (and obfuscated) script that I verified in CodeRunner, Terminal, and Self Service.

 

#!/bin/bash # this script is used to collect a full system profiler dump in txt format # as well as user accounts, groups, crowdstrike info, open ports # it will put all the files in /Library/company/PCI/<computername>/ # for later retrieval and submission for PCI audit # Set variables for the JAMF API jamf_url="https://company.jamfcloud.com" jamf_user="APIuser" jamf_password="APIpassword" #Variable declarations bearerToken="" tokenExpirationEpoch="0" getBearerToken() { response=$(curl -s -u "$jamf_user":"$jamf_password" "$jamf_url"/api/v1/auth/token -X POST) bearerToken=$(echo "$response" | plutil -extract token raw -) tokenExpiration=$(echo "$response" | plutil -extract expires raw - | awk -F . '{print $1}') tokenExpirationEpoch=$(date -j -f "%Y-%m-%dT%T" "$tokenExpiration" +"%s") } checkTokenExpiration() { nowEpochUTC=$(date -j -f "%Y-%m-%dT%T" "$(date -u +"%Y-%m-%dT%T")" +"%s") if [[ tokenExpirationEpoch -gt nowEpochUTC ]] then echo "Token valid until the following epoch time: " "$tokenExpirationEpoch" else echo "No valid token available, getting new token" getBearerToken fi } checkTokenExpiration computer_id=$(jamf recon | grep '<computer_id>' | awk -F'[<>]' '{print $3}') echo "Jamf computer ID: $computer_id" # Path and timestamp variables SERIAL=`ioreg -l | awk '/IOPlatformSerialNumber/ { print $4;}' | sed 's/"//g'` echo "serial number: $SERIAL" PCI_PATH="/Library/company/PCI/$SERIAL" echo "Path to log file: $PCI_PATH" timestamp=$(date "+%Y.%m.%d") year=$(date "+%Y") auditlog="$SERIAL-$timestamp.zip" echo "log file name: $auditlog" logpath=/Library/company/PCI/$auditlog echo "the full path to the zip file is $logpath" # if there is already a folder in the target location (probably from an earlier capture) this will delete it if [[ -d /Library/company/PCI/$SERIAL ]] ; then rm -Rf /Library/company/PCI/$SERIAL fi # Creates the ./PCI/<serial number> folder mkdir -p $PCI_PATH #### START OF AUDIT COLLECTION #### # All user and system accounts that do not start with _ dscacheutil -q user | perl -00ne 'print if !/name: _/' > $PCI_PATH/Users.txt echo "user and system accounts saved " # All groups that do not start with _ dscacheutil -q group | perl -00ne 'print if !/name: _/' > $PCI_PATH/Groups.txt echo "all non invisible groups saved" # Generates a plain text list of applications and version info system_profiler SPApplicationsDataType > $PCI_PATH/Applications.txt echo "list of all applications saved" # Crowdstrike Status /Applications/Falcon.app/Contents/Resources/falconctl stats > $PCI_PATH/crowdstrike.txt echo "Crowdstrike status saved" # Lists Services in a similar manner to what is listed in the computer record in Jamf cat << 'EOT' > /Library/company/PCI/$SERIAL/SERVICES_NOTE.txt the list of services was generated by the following command: /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' EOT echo "" > $PCI_PATH/Services.txt /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' >> $PCI_PATH/Services.txt echo "list of services saved" # full system profiler dump system_profiler -xml -detailLevel full > $PCI_PATH/system_profiler_full.xml echo "The system_profiler_full.xml file can be renamed with .spx extension and can then be opened directly with the System Information.app found in macOS of all versions" > /Library/company/PCI/$SERIAL/SYSTEM_PROFILER_NOTE.TXT echo "System Profiler dump saved " # Open ports listening lsof -i -P | grep -i "listen" > $PCI_PATH/openports.txt echo "list of open ports saved" # evidence of time sync touch /var/db/ntp-kod chmod 666 /var/db/ntp-kod sntp -sS time.apple.com > $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "commands used to generate this data: touch /var/db/ntp-kod ; chmod 666 /var/db/ntp-kod ; sntp -sS time.apple.com" >>$PCI_PATH/timesync.txt echo "time sync info saved" # Zips up all the files generated in the /PCI/<serial number>/ folder cd /Library/company/PCI/ zip -rj $auditlog $PCI_PATH echo "Zip file created" # copy the zip file to an easier path to insert in the curl command cp $logpath /tmp/ echo "uploading zip file to jamf server" # Upload file to computer record using JAMF API curl -H "Authorization: Bearer ${bearerToken}" $jamf_url/JSSResource/fileuploads/computers/id/$computer_id -F name=@/tmp/$auditlog -X POST echo "Evidence gathered for the $year Audit. Done"

 

I am so glad JamfNation exists. Even if I don't get the exact solution I need many times the replies I get spark something in my brain to get me thinking about other possibilities. I am also glad I have CodeRunner. This app has made script writing so much easier as far as finding where things break. I am glad that bash -x script.sh is a thing to let me see every step of the script in action as it runs. And I am glad removing the -s option for curl lets it tell me what's happening (sometimes).

I am pretty sure there are better ways of accomplishing this task of gathering lots of system information in various text files, zipping them up, and uploading to JamfPro, but this finally works for me.


mm2270
Forum|alt.badge.img+16
  • Legendary Contributor
  • 7880 replies
  • March 9, 2023
AVmcclint wrote:

SOLVED!! It turns out there were a couple things working against me on this.

1. I had some issues with the name= part of the curl command. To simplify it I copied the zip file to a known location "/tmp/" and then used a single variable for the file path. It turned out to be easier than trying to layer multiple variables into the curl command. This solved the curl 26 error.

2. I thought I had issues with using basic authentication so I decided it was time I jump in with Bearer Tokens anyway. Turns out that basic auth wasn't the problem (because I could basic authenticate with my global admin account), but I needed to move forward anyway.

3. I had to do some deep digging to discover that the API-only account I am using needed CREATE rights on Computers in addition to all the other obvious rights. This solved the http error I was getting.

Here is the final working (and obfuscated) script that I verified in CodeRunner, Terminal, and Self Service.

 

#!/bin/bash # this script is used to collect a full system profiler dump in txt format # as well as user accounts, groups, crowdstrike info, open ports # it will put all the files in /Library/company/PCI/<computername>/ # for later retrieval and submission for PCI audit # Set variables for the JAMF API jamf_url="https://company.jamfcloud.com" jamf_user="APIuser" jamf_password="APIpassword" #Variable declarations bearerToken="" tokenExpirationEpoch="0" getBearerToken() { response=$(curl -s -u "$jamf_user":"$jamf_password" "$jamf_url"/api/v1/auth/token -X POST) bearerToken=$(echo "$response" | plutil -extract token raw -) tokenExpiration=$(echo "$response" | plutil -extract expires raw - | awk -F . '{print $1}') tokenExpirationEpoch=$(date -j -f "%Y-%m-%dT%T" "$tokenExpiration" +"%s") } checkTokenExpiration() { nowEpochUTC=$(date -j -f "%Y-%m-%dT%T" "$(date -u +"%Y-%m-%dT%T")" +"%s") if [[ tokenExpirationEpoch -gt nowEpochUTC ]] then echo "Token valid until the following epoch time: " "$tokenExpirationEpoch" else echo "No valid token available, getting new token" getBearerToken fi } checkTokenExpiration computer_id=$(jamf recon | grep '<computer_id>' | awk -F'[<>]' '{print $3}') echo "Jamf computer ID: $computer_id" # Path and timestamp variables SERIAL=`ioreg -l | awk '/IOPlatformSerialNumber/ { print $4;}' | sed 's/"//g'` echo "serial number: $SERIAL" PCI_PATH="/Library/company/PCI/$SERIAL" echo "Path to log file: $PCI_PATH" timestamp=$(date "+%Y.%m.%d") year=$(date "+%Y") auditlog="$SERIAL-$timestamp.zip" echo "log file name: $auditlog" logpath=/Library/company/PCI/$auditlog echo "the full path to the zip file is $logpath" # if there is already a folder in the target location (probably from an earlier capture) this will delete it if [[ -d /Library/company/PCI/$SERIAL ]] ; then rm -Rf /Library/company/PCI/$SERIAL fi # Creates the ./PCI/<serial number> folder mkdir -p $PCI_PATH #### START OF AUDIT COLLECTION #### # All user and system accounts that do not start with _ dscacheutil -q user | perl -00ne 'print if !/name: _/' > $PCI_PATH/Users.txt echo "user and system accounts saved " # All groups that do not start with _ dscacheutil -q group | perl -00ne 'print if !/name: _/' > $PCI_PATH/Groups.txt echo "all non invisible groups saved" # Generates a plain text list of applications and version info system_profiler SPApplicationsDataType > $PCI_PATH/Applications.txt echo "list of all applications saved" # Crowdstrike Status /Applications/Falcon.app/Contents/Resources/falconctl stats > $PCI_PATH/crowdstrike.txt echo "Crowdstrike status saved" # Lists Services in a similar manner to what is listed in the computer record in Jamf cat << 'EOT' > /Library/company/PCI/$SERIAL/SERVICES_NOTE.txt the list of services was generated by the following command: /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' EOT echo "" > $PCI_PATH/Services.txt /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' >> $PCI_PATH/Services.txt echo "list of services saved" # full system profiler dump system_profiler -xml -detailLevel full > $PCI_PATH/system_profiler_full.xml echo "The system_profiler_full.xml file can be renamed with .spx extension and can then be opened directly with the System Information.app found in macOS of all versions" > /Library/company/PCI/$SERIAL/SYSTEM_PROFILER_NOTE.TXT echo "System Profiler dump saved " # Open ports listening lsof -i -P | grep -i "listen" > $PCI_PATH/openports.txt echo "list of open ports saved" # evidence of time sync touch /var/db/ntp-kod chmod 666 /var/db/ntp-kod sntp -sS time.apple.com > $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "commands used to generate this data: touch /var/db/ntp-kod ; chmod 666 /var/db/ntp-kod ; sntp -sS time.apple.com" >>$PCI_PATH/timesync.txt echo "time sync info saved" # Zips up all the files generated in the /PCI/<serial number>/ folder cd /Library/company/PCI/ zip -rj $auditlog $PCI_PATH echo "Zip file created" # copy the zip file to an easier path to insert in the curl command cp $logpath /tmp/ echo "uploading zip file to jamf server" # Upload file to computer record using JAMF API curl -H "Authorization: Bearer ${bearerToken}" $jamf_url/JSSResource/fileuploads/computers/id/$computer_id -F name=@/tmp/$auditlog -X POST echo "Evidence gathered for the $year Audit. Done"

 

I am so glad JamfNation exists. Even if I don't get the exact solution I need many times the replies I get spark something in my brain to get me thinking about other possibilities. I am also glad I have CodeRunner. This app has made script writing so much easier as far as finding where things break. I am glad that bash -x script.sh is a thing to let me see every step of the script in action as it runs. And I am glad removing the -s option for curl lets it tell me what's happening (sometimes).

I am pretty sure there are better ways of accomplishing this task of gathering lots of system information in various text files, zipping them up, and uploading to JamfPro, but this finally works for me.


Yay! Glad you got it working!


Forum|alt.badge.img+4
  • Contributor
  • 13 replies
  • June 7, 2024
AVmcclint wrote:

SOLVED!! It turns out there were a couple things working against me on this.

1. I had some issues with the name= part of the curl command. To simplify it I copied the zip file to a known location "/tmp/" and then used a single variable for the file path. It turned out to be easier than trying to layer multiple variables into the curl command. This solved the curl 26 error.

2. I thought I had issues with using basic authentication so I decided it was time I jump in with Bearer Tokens anyway. Turns out that basic auth wasn't the problem (because I could basic authenticate with my global admin account), but I needed to move forward anyway.

3. I had to do some deep digging to discover that the API-only account I am using needed CREATE rights on Computers in addition to all the other obvious rights. This solved the http error I was getting.

Here is the final working (and obfuscated) script that I verified in CodeRunner, Terminal, and Self Service.

 

#!/bin/bash # this script is used to collect a full system profiler dump in txt format # as well as user accounts, groups, crowdstrike info, open ports # it will put all the files in /Library/company/PCI/<computername>/ # for later retrieval and submission for PCI audit # Set variables for the JAMF API jamf_url="https://company.jamfcloud.com" jamf_user="APIuser" jamf_password="APIpassword" #Variable declarations bearerToken="" tokenExpirationEpoch="0" getBearerToken() { response=$(curl -s -u "$jamf_user":"$jamf_password" "$jamf_url"/api/v1/auth/token -X POST) bearerToken=$(echo "$response" | plutil -extract token raw -) tokenExpiration=$(echo "$response" | plutil -extract expires raw - | awk -F . '{print $1}') tokenExpirationEpoch=$(date -j -f "%Y-%m-%dT%T" "$tokenExpiration" +"%s") } checkTokenExpiration() { nowEpochUTC=$(date -j -f "%Y-%m-%dT%T" "$(date -u +"%Y-%m-%dT%T")" +"%s") if [[ tokenExpirationEpoch -gt nowEpochUTC ]] then echo "Token valid until the following epoch time: " "$tokenExpirationEpoch" else echo "No valid token available, getting new token" getBearerToken fi } checkTokenExpiration computer_id=$(jamf recon | grep '<computer_id>' | awk -F'[<>]' '{print $3}') echo "Jamf computer ID: $computer_id" # Path and timestamp variables SERIAL=`ioreg -l | awk '/IOPlatformSerialNumber/ { print $4;}' | sed 's/"//g'` echo "serial number: $SERIAL" PCI_PATH="/Library/company/PCI/$SERIAL" echo "Path to log file: $PCI_PATH" timestamp=$(date "+%Y.%m.%d") year=$(date "+%Y") auditlog="$SERIAL-$timestamp.zip" echo "log file name: $auditlog" logpath=/Library/company/PCI/$auditlog echo "the full path to the zip file is $logpath" # if there is already a folder in the target location (probably from an earlier capture) this will delete it if [[ -d /Library/company/PCI/$SERIAL ]] ; then rm -Rf /Library/company/PCI/$SERIAL fi # Creates the ./PCI/<serial number> folder mkdir -p $PCI_PATH #### START OF AUDIT COLLECTION #### # All user and system accounts that do not start with _ dscacheutil -q user | perl -00ne 'print if !/name: _/' > $PCI_PATH/Users.txt echo "user and system accounts saved " # All groups that do not start with _ dscacheutil -q group | perl -00ne 'print if !/name: _/' > $PCI_PATH/Groups.txt echo "all non invisible groups saved" # Generates a plain text list of applications and version info system_profiler SPApplicationsDataType > $PCI_PATH/Applications.txt echo "list of all applications saved" # Crowdstrike Status /Applications/Falcon.app/Contents/Resources/falconctl stats > $PCI_PATH/crowdstrike.txt echo "Crowdstrike status saved" # Lists Services in a similar manner to what is listed in the computer record in Jamf cat << 'EOT' > /Library/company/PCI/$SERIAL/SERVICES_NOTE.txt the list of services was generated by the following command: /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' EOT echo "" > $PCI_PATH/Services.txt /bin/launchctl list | /usr/bin/awk '{print substr($0, index($0, $3))}' | /usr/bin/sed '1d' >> $PCI_PATH/Services.txt echo "list of services saved" # full system profiler dump system_profiler -xml -detailLevel full > $PCI_PATH/system_profiler_full.xml echo "The system_profiler_full.xml file can be renamed with .spx extension and can then be opened directly with the System Information.app found in macOS of all versions" > /Library/company/PCI/$SERIAL/SYSTEM_PROFILER_NOTE.TXT echo "System Profiler dump saved " # Open ports listening lsof -i -P | grep -i "listen" > $PCI_PATH/openports.txt echo "list of open ports saved" # evidence of time sync touch /var/db/ntp-kod chmod 666 /var/db/ntp-kod sntp -sS time.apple.com > $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "" >> $PCI_PATH/timesync.txt echo "commands used to generate this data: touch /var/db/ntp-kod ; chmod 666 /var/db/ntp-kod ; sntp -sS time.apple.com" >>$PCI_PATH/timesync.txt echo "time sync info saved" # Zips up all the files generated in the /PCI/<serial number>/ folder cd /Library/company/PCI/ zip -rj $auditlog $PCI_PATH echo "Zip file created" # copy the zip file to an easier path to insert in the curl command cp $logpath /tmp/ echo "uploading zip file to jamf server" # Upload file to computer record using JAMF API curl -H "Authorization: Bearer ${bearerToken}" $jamf_url/JSSResource/fileuploads/computers/id/$computer_id -F name=@/tmp/$auditlog -X POST echo "Evidence gathered for the $year Audit. Done"

 

I am so glad JamfNation exists. Even if I don't get the exact solution I need many times the replies I get spark something in my brain to get me thinking about other possibilities. I am also glad I have CodeRunner. This app has made script writing so much easier as far as finding where things break. I am glad that bash -x script.sh is a thing to let me see every step of the script in action as it runs. And I am glad removing the -s option for curl lets it tell me what's happening (sometimes).

I am pretty sure there are better ways of accomplishing this task of gathering lots of system information in various text files, zipping them up, and uploading to JamfPro, but this finally works for me.


Hello,
Sorry to bother on an old post.
First of all glad that you found the solution, it seems really smart and effective. Congrats!
I'm new to this whole Jamf stuff and to coding in general and I'd like to ask you Where these files go when you say "uploading to JamfPro" and how to enable the API to let this work correctly.
Do you know if the user can access in any way to the script above and "find" the API user login info?
Thank you very Much! Have a nice day :)


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • June 7, 2024
DonCascone wrote:

Hello,
Sorry to bother on an old post.
First of all glad that you found the solution, it seems really smart and effective. Congrats!
I'm new to this whole Jamf stuff and to coding in general and I'd like to ask you Where these files go when you say "uploading to JamfPro" and how to enable the API to let this work correctly.
Do you know if the user can access in any way to the script above and "find" the API user login info?
Thank you very Much! Have a nice day :)


The Jamf admin can find the files in the computer record > Attachments. 


Forum|alt.badge.img+3
  • New Contributor
  • 3 replies
  • August 2, 2024

Thank You @AVmcclint this is Great!

Is there a way to modify the script in a way that it will delete the files older than 1 month ad example? 

Both on Device itself and on the Device Page on JAMF portal.

 

Thanks Mate


AVmcclint
Forum|alt.badge.img+21
  • Author
  • Esteemed Contributor
  • 1043 replies
  • August 2, 2024
nicho2312 wrote:

Thank You @AVmcclint this is Great!

Is there a way to modify the script in a way that it will delete the files older than 1 month ad example? 

Both on Device itself and on the Device Page on JAMF portal.

 

Thanks Mate


In the script, it deletes the local capture folder if it already exists.  As for removing it from the server, I don’t know. That’s why I have the zip file saved with the timestamp in the name so anyone accessing the files in the computer record will know which file is most recent. 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings