Update your Extension Attributes - Often

mm2270
Legendary Contributor III

Hi fellow JAMFNationers.

I wanted to introduce a project I've been working on after hours to help address a specific need for some of you here in the JAMF community.
I've developed a method of being able to update Extension Attribute values in a computer record in the JSS through the JSS API and more specifically, without the need for a full recon.

As has been discussed at length in this feature request - https://jamfnation.jamfsoftware.com/featureRequest.html?id=78 many of us would like a way to do specific inventory collection, rather than a full inventory collection at every recon, especially if we only need a couple of updated values.
This Feature Request is marked as “Not Planned” and I do understand the concern JAMF has expressed regarding incomplete inventory collections. Partial inventory can be risky business if not used properly, so its a valid concern for sure.
That said, I think many of us understand the implications of doing something like this, but would still like the ability despite the risks.
While what I’ve developed doesn’t provide a way to do only targeted inventory updates, such as only collecting updated Application data, and skipping everything else, as an example, it does allow you to update either all or part of your Extension Attributes.

The high level overview of how it works is, a script called “update-extension-attributes.sh” will loop through a folder of Extension Attribute scripts located locally on a Mac, and populate a final xml file with each scripts results, and finally, upload the xml file to the computer’s JSS record using the API.
For the most part, the scripts you have as your EA scripts will work as is, but some minor edits need to be done on each script for it to work. In order for the “update” script to correctly set up the xml file, it needs to know the EA’s Display Name as it appears in the JSS, so it can be properly paired up. To have this happen, all you need to add is a single commented out line to each script. For example. say you have an EA script called “FlashPlayer Version”, you would add a line, anywhere in the script that looks like this

#ea_display_name    FlashPlayer Version

The space between “#ea_display_name” and “FlashPlayer Version” is a single tab character. The readable name must match what you named your Extension Attribute in the JSS. That’s all you’d need to do in 99% of cases. (See below for some known issues)
You do not need to escape spaces, dashes, colons, etc. The best way to set it up is to copy the display name directly from the JSS for each Extension Attribute and paste that to the script file.after typing in #ea_display_name + tab.

The procedure would be repeated for each script. Once you have them all ready, the script files need to be copied to a location on the Mac, or packaged up and deployed like with any policy. My script uses a default location of /Library/Application Support/JAMF/extension_attributes/, but can be changed.
That folder can be protected by making it owned and viewable only by root. In fact, you could easily deploy this folder of scripts to a location like /Library/Application Support/JAMF/bin/extension_attributes/

In addition to the above, you of course need to have an API account and password with API read and write privileges to the JSS.

How it works
The script runs through each EA script in the target directory, just as a normal recon might do, and captures the scripts results as well as the script’s EA display name from the script file itself. As long as the ea display name doesn’t come back blank when it runs it adds these values into the xml file as it loops through the scripts, in the end it completes the xml, then uses the credentials and JSS URL you supply to it to upload the xml to the computer’s record. It grabs the MAC address for en0 from the Mac to do this, but it could be modified to use other information, like the Serial #, or UDID, etc. I used the MAC address for en0, because its the most compatible identifier across the Casper Suite 8 and 9 series. The serial # and other items only started appearing as valid Identifiers for API operations in version 9.x

Once the above is all set up, you could easily have a policy triggered on every check in to run the script, which in turn runs each EA script in your folder, collects the data and uploads it to the Mac’s record.
Some good uses for it would be to capture Uptime on a regular basis from your Macs, or any other piece of information that might change on a frequent basis, that would normally be stale from only collecting it once a day, as an example.
Another example use would be to include it as an “After” script in a policy that dropped a file identifier on the Mac that needs to be captured in an EA script. You could then potentially uncheck the “Update Inventory” box in the policy.

So, going back to the piece on getting the scripts downloaded and modified. I also built a companion script, called “download-extension-attributes.sh”. For anyone on Casper Suite 8 series, like us, you won’t have much choice but to manually recreate or collect each Extension Attribute script and make the simple modifications to them.
For anyone on Casper Suite 9 though, the companion script I built can be run to pull down all EA scripts using the API, and name them with the EA display name and even add the appropriate #ea_display_name line to each file. It even runs through all the scripts at the end and confirms they work. Any scripts that exit with an error code will get moved into a sub directory. The script provides output on everything as it runs so you can see what its pulling down, which script its testing and what the result was.
I built this companion script to accept arguments on the command line OR Casper Suite script parameters, your choice, so you don’t need to hardcode your JSS URL, or API username and password into the script. It even includes a ‘help’ switch you can call up to see how to use the script..

Known issues and caveats
In testing this out, I’ve discovered a few interesting things.
1 - Casper Suite 8 and recent versions of Casper Suite 9 will properly display multi line EA results. The dev JSS 9 server we have is still on 9.22 and there was a bug in that version that wouldn’t respect any line breaks. Even though the xml would properly set up the line breaks for a result. Once uploaded it would display the result in one long unbroken line. (This was true even when running a regular recon) Working with our TAM, we confirmed this was addressed at some point. I only know it displays correctly in 9.31, so YMMV on that depending on the JSS version you’re on.

2 - Any EA results that use certain characters will cause the upload to fail. The following are some characters I’ve found will cause issues and make the API PUT command fail:
< and > Such as a result like <some value> This confuses the API as it thinks they are xml tags and refuses to upload the file since it believes it to be malformed.
% Not sure why, but results with a % sign can cause the upload to fail as well.
, An odd one, but anything with a comma in the result seems to be also seen as part of the xml and the upload fails.
* Asterisks are also no good. Any result with an asterisk will not upload correctly.[/ul]

There may be several others I haven’t run into so some of this is going to be trial and error to weed them all out. Report any others you find to me if you please so I can update the script. The original EA scripts may work perfectly fine as part of the JSS, but in this more manual process, it can’t accept these characters.
As such, I designed the “download-extension-attributes” script to check the result of each script and if found to contain these characters, will move the script into a sub-folder so you can check on them later.
If you do have any scripts that would give results using an illegal character, you’ll have to decide if you want to modify the script to use a different character, or just skip using that script as part of the process.

Some other stuff I’ve found
I’m not sure if this was just abut in 9.22, but in testing, I found that some of the scripts would download in an incomplete way, meaning the script would get cut off at a certain line, making the script invalid. These can be fixed manually, but on first run of downloading them with the companion script, it will find these and also move them into a sub-directory.

What about with a Limited Access JSS?
So, for those of you who have a Limited Access JSS set up, one issue you’ll run into is that API access is disabled on the externally facing web app server, because it requires Tomcat to work.
To combat this, I built the “update” script to accept a flag that would tell it to check an internal site or server you can specify in a script parameter. If the script sees that an internal check is required, it will run either a ping or curl command against the INTERNAL address you specify. If successful, it continues to run the script and collect the data and upload it. If it fails, it means the Mac is on the outside and it exits gracefully.
If you only have an internal JSS, then the internal check isn’t required and you can remove the flag so it won’t try to do that check.
Conversely, if you have a JSS that is open to the outside (not a Limited Access JSS) then you also don’t need to set the flag since API uploads should work fine even from outside the organization walls.

OK, so, how to get this?
Go to this Github page I set up. It has the scripts and some details. The Read Me needs to be updated with a little more information once I have some time, but it has the basics. Most of what I described in this post is also on the github page:
https://github.com/mm2270/UpdateExtensionAttributes

You might be asking yourself how much faster this will be than just doing a normal recon. Well, valid question, so I’ve done some basic benchmarking. I ended up with a total of 104 Extension Attribute scripts after weeding out some that were causing issues and removing a few others from our list. In testing the “update” script against a few different varieties of MacBooks, I found that it generally took anywhere from 15 to 30 seconds maximum for the script to loop through and run all 104 EA scripts, collect their results, generate the xml file and finally upload it to the computer’s record. That’s not bad, and I’d say, considerably faster than a full inventory collection.

If you decide to use these, best practice would be to only test it on a couple of scripts to start with, just to be sure you have everything set up properly. You can then begin adding in additional scripts to the mix as desired. I should just also note that its your responsibility as a Casper administrator to be sure that you aren’t updating EAs that are also paired up with other regular inventory data being used in Smart Group(s), or you could end up with inaccurate Smart Groups, just as JAMF expressed concerns about. I'm not responsible if you mess up your Smart Groups :)

This process if far from a perfect solution, especially since it requires the scripts to be on the Mac to be used with this process. In some ways though, this can actually provide some granularity in what you want updated, rather than all Extension Attributes, which may not always be necessary.

Lastly, given I’ve been the only person doing any tests with these scripts, I’d love to get any feedback on problems or errors you encounter, or improvements on it if you make any, if you decide to try this out. But before you report problems, please just make sure you’ve followed the instructions I outline on the Github page or in the scripts themselves.

Enjoy.

5 REPLIES 5

Chris
Valued Contributor

Wow, i like this, awesome work!
I've just spent some time myself working with EAs through the API,
but this will definitely come in handy.
Thanks for sharing this!

loceee
Contributor

Cool! This is a very interesting approach and it's given me an interesting, slightly related idea. Thanks!

seansb
New Contributor III

This is great! Not sure if this is the correct place to ask, but I need to do something very similar to your update-extension-attributes script. I'm not too familiar with the REST API and Casper, but I'd like to be able to provide an XML file with hostnames as well as extension attributes. Your example, you pull an attribute from a single local machine and upload it. I'd like to do a single curl statement that uploads information for multiple machines (hopefully that makes sense). For example, my XML document would look like:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<computer>
<general>
    <name>test1</name>
</general>
<extension_attributes>
<attribute>
    <name>TestAttribute1</name>
    <value>Info</value>
    <name>TestAttribute2</name>
    <value>Info</value>
</attribute>
</extension_attributes>
</computer>
<computer>
<general>
    <name>test2</name>
</general>
<extension_attributes>
<attribute>
    <name>TestAttribute1</name>
    <value>Info</value>
    <name>TestAttribute2</name>
    <value>Info</value>
</attribute>
</extension_attributes>
</computer>

Your code uploads to a specific machine using the mac address, here:

curl -skfu "${apiUser}:${apiPass}" "${jssURL}/JSSResource/computers/macaddress/$MacID" -T "${xmlPath}" -X PUT

I'm wondering if there is some way in the curl command or through another attribute to have the JSS read the XML document to get the hostname specified in the XML document and update the extension attributes accordingly.

I hope I've explained that well enough - please let me if you need clarification.

Thanks!

NightFlight
New Contributor III

Here is a fantastic REST API wrapper and I have it working great with the JSS. It should be noted I enjoy coding in PHP much more than BASH for many reasons. I haven't yet implemented EA PUTs but its on my next ToDo.

http://phphttpclient.com/

It should also be noted that while the JSS will give you JSON by default, it will fail if you try to PUT with JSON. This took me days to discover this lack of support / documentation. Yours free for reading this message. :)

alexduffner
New Contributor III

Hello @mm2270 , great work!

Even today with Jamf Pro 10.25.1 the download script helps to check the existing EAs for compatibility with Big Sur (regarding xpath). It only needs a small addition at the beginning for xpath, if your Mac is running already Big Sur (like mine):

# Kudos to Armin Briegel for sharing this snippet on his blog
xpath() {
    if [[ $(sw_vers -buildVersion) > "20A" ]]; then
        /usr/bin/xpath -e "$@"
    else
        /usr/bin/xpath "$@"
    fi
}

Best regards
Alex