Enforcing Apple software updates in the year 2021.

[sdamiano]

While this is, arguably, a feature that Jamf should be able to handle at its core, what is the best method in 2021, across the gamut of supported OS's and architectures, to enforce Apple software updates?

I understand that theres resistance to using any terminal commands for software updates. As stated in the Nudge channel, Since 2018 this has been a problem since Apple doesn't test softwareupdate commands that are triggered by a script. In addition to that, this requires a password if the computer is Apple Silicon.

I recently rolled out the UEX-Tool-For-Jamf, and two of the components in it are no longer developed and throw gatekeeper issues in Big Sur.

I would like to use Nudge, but, my confidence in my users is low enough to not expect Nudge to work fine on its own. Nudge cannot force the update to happen.

What is the best way to handle software updates? Is there MDM magic I am missing? What is the downside to having the checkbox "keep my mac up to date" checked via a config profile?

[bwoods]

For me it's between 20-40 minutes, but I haven't pushed this to production yet. More testing to do. I can just let the user know that the update can happen within that timespan with Jamfhelper.

[bbarciz]

I am sending the commands through the Jamf GUI and just planning for a delay.  So far, it appears that the commands start with in maybe 10 or 15 minutes for me, and the download time can be a factor depending on the number of machines that I tell it to update at the same time (we have our bandwidth for updates limited as not to take everything else to its knees at the same time).  To get a feel for when it starts, I will be logged into a system and watch activity monitor and wait for network traffic to start, that is how I have gauged the delay in my environment.  I find that I can do roughly 20 systems in about 2 hours for an OS upgrade (MacOS11 to MacOS12).

[bwoods]

@bbarciz do you know if it's possible to chain the "download updates" and "download and install updates" commands? that would simulate an install cached workflow and speed things up a bit.

[bbarciz]

@bwoods  I do not know the answer to that.  It seems like a good logic and when I would deploy an install macos full installer in the past, I would pre-cache the installer on the machines for this exact logic.

I just looked at my jamf instance (version 10.35) and see 3 options for install action now:

• Download the update for users to install
  • I seem to recall reading that this one required users to have admin rights or for their accounts to have something set with them.  I don't recall the details and I also assume the user would get notified which may not be what you want.
• Download and allow macOS to install later
  • This has a setting for "max user deferrals" to allow users to defer a certain number of times.
  • This seems to imply the user will be notified since it could be deferred as well.  Not sure if that would be ok in your concept to cache the installer.
• Download and install the update, and restart computers after installation
  • This is the do it now type action with no warning that the computer will restart.

Not sure if that really helps you out at all or not. 

[rocketman]

@Hugonaut presented on this on our last meetup and created some resources to assist with these issues. Check it out: https://community.jamf.com/t5/virtual-mac-admins-meetup/updating-macos-via-mdm-command-with-deferral...

[tzeilstra]

So now that the new features in 10.35 and later have been available for a little while, what kind of results have people been seeing?  From what we've seen, if we send out an MDM command telling computers to update to a specific OS version and allow 1 day deferral option, we might get about a 50% upgrade rate after a few days.  For some folks the popup message in the Notification Center either doesn't appear or isn't noticed, others see it and ignore it, some say Try Tonight but never take any action themselves and the computer just decides not to apply the update after all.  Sure, it's one more tool in our bag when it comes to ways to chip away at the number of unpatched devices but actual "enforcement" this is not.

Are others seeing anything similar or is my experience an outlier here?

[AJPinto]

@tzeilstra its still a mixed bag. There is a lot that can go wrong in the Software Update process, and in JAMFs infinite wisdom they still are not using the StatusUpdate key. So, JAMF has no idea what a Mac is doing with Software Updates. Seems like the problem still persists after all these years, JAMF is still half a!@ing software updates on macOS.

I had a case open with JAMF that got up to the engineers a few weeks ago. The best answer they had for me was to make a smart group for OS versions (which I do already), and contact users for devices that fail to update. You know, because we have JAMF to do things manually and not automate them.

To be fair, Apple moved to MDM command based Software Updates way too fast and did not let the kinks get worked out first. I'm still waiting for a policy to be able to actually schedule OS updates which was hinted would be coming in this post.

[atrystan]

We manage a computer lab and a device checkout program that has no dedicated users. As a result, using the built-in functionality will never work for us--nobody is going to give up their computer time to run an update. We have tried using it but I don't think that we got a single machine to update itself. 

This is totally broken for cases in which there is not a 1:1 computer to user relationship.

[emilh]

This is exactly what we're facing as well. Users of computer labs simply cannot be incentivised to install an update on their limited time.

The only workaround for this that I've found is to distribute the complete OS installer of the latest version of macOS to all the lab machines and then triggering startosinstall during the night. If you ask me, thats too many loops to jump through for every point and security update.

MDM-triggered updates for 1:1 machines is really hit or miss, after about a month we're looking at about a 65% update rate.

During the "old days" when the softwareupdate could be triggered (and trusted to do its job) by jamf, we were at 90%+ within a few days.

[atrystan]

This is precisely the issue we are facing. We've previously used an update policy that we could manually invoke, but that breaks on the latest M1 machines that use the APFS Volume Owner as the only person who can perform an update. We are having a great deal of difficulty installing any updates to these machines because for some reason, Sophos creates a user during install that becomes the machine's owner. We've been working on this problem for weeks and can't get around it, so some machines are stuck on Big Sur until further notice...

[jtrant]

My workaround for this until Jamf add policy-based MDM commands was to set up a limited access local account in Jamf, and script an API call stored on another server located on the same subnet. The local account only has CRU for Computers/Mobile Devices, and can only send an update MDM command (e.g. if the credentials were compromised, they can't be used to remote lock devices, for example).

The server containing the API script has a scheduled task that runs at the desired time, and targets a Jamf smart group with an MDM command to install updates and restart. I'm not thrilled at having to use the API but since the script is not being downloaded to endpoints and run from there (e.g. the server with the script calls Jamf which is on the same subnet, and the credentials are not passed in plain text) it's a happy medium for now.

[atrystan]

In our case, we would do periodic imaging and used eraseinstall to accomplish it. If you kept the latest build packaged in a policy with an eraseinstall script attached, you could upgrade a computer and refresh it at the same time. It truly made the entire thing a zero-touch experience. I miss the old days (of about a year ago).

[dstranathan]

On a related note...

I'm not a fan of the Software Update deferments living in the Restrictions profile, alongside other settings (that are static and never change). I wish these settings were located in the (duh!) Jamf Software Update profile.

I'm considering breaking our Software Update settings into their own discreet profile(s) so that I can edit them without worrying about other Restriction payloads being affected. Since Software Deferments live in the com.apple.applicationaccess preference domain, I dont think this should be a problem.

Is anyone else doing this (or similar)?

[tzeilstra]

https://www.youtube.com/watch?v=tqF4ls823ig

This will totally change the way you look at Profiles.  Short answer is yes, we deploy a profile for deferring software updates and that's ALL that profile does.  No extra stuff from the restrictions payload.

[dstranathan]

Update: In July 2022, I broke out my Deferment restrictions from the other Restrictions. So now my scope look like this:

Restrictions: All Computers (does NOT contain any SU deferment payloads - I disabled them all)
Deferments SU Restriction - Production: All non-IT Macs (30-day minor updates and 90-day major updates)
Deferments SU Restriction - IT: All IT Macs (7-day minor updates and 30-day major updates)

For me and (1) other Mac admin at my org, we are in a scope that excludes us both from any SU deferments so we see Apple updates (Major and minor) at zero-day and thus we can start testing ASAP.

[jtrant]

Check out iMazing Profile Editor too, does a great job of building custom profiles (that you can sign using the app) without including extra garbage, or messing around with PLISTs.

[perryd84]

So this is still a massive issue I see and no one has a decent solution for managing updates still😂😂😂

So I used to use one script to manage both updates and upgrades but have split this into 2 scripts to trigger version upgrades and updates.

For version upgrades I use the softwareupdates command with the fetch full installer switch which is easy on Intel but requires a few pop ups and admin elevations on M1 macs but its working fine.

For updates I use the softwareupdate command for intel macs and then MDM commands from the API for M1 macs. The issue I have with the MDM command is that it will never reboot the device and relies on the user to reboot but it doesn't tell the user you need to reboot?!?!?! This is the line I use at the moment and the forceRestart doesn't seem to do anything. Anyone got a fix for this?

https://$YOURJSS/JSSResource/computercommands/command/ScheduleOSUpdate/action/InstallForceRestart/id/$ID

Currently I have a line at the end of my script to open the software updates system prefs panel so the user can see the progress and the reboot button. But if they close this then they never know they need to reboot!??!

[dlondon]

Is @perryd84 's way of accessing the API deprecated?  Shouldn't this be done with a Bearer Token now?

[perryd84]

I use the bearer token to authenticate this is just calling the MDM command. The old API calls can use the bearer token as well.

[ianatkinson]

After a few weeks of the API approach I think it's working fairly well though not 100% reliably.

I think the structure I've put into JAMF is OK it's just that sometimes machines get the management command but don't seem to actually do the updates even with all the bootstrap tokens in place. install.log isn't a very friendly file to read and I'm yet to spot some obvious reason for when this happens.

Other times the delays between the commands getting to the machine and the update actually installing are very long which from a staff member's point of view is a bad experience. For example I tested one this morning and it got the commands and did the scan at about 9.05am, it installed them and rebooted at 10.45am (this is a machine just sat there with nobody logged in and a fast Internet connection).

2022-04-13 09:06:18+01 BW-UNION-04 softwareupdated[334]: Removing client SUUpdateServiceClient pid=15458, uid=0, installAuth=NO rights=(), transactions=0 (/usr/libexec/mdmclient)
2022-04-13 09:06:18+01 BW-UNION-04 softwareupdated[334]: SUOSUServiceDaemon: Connection invalidated!

2022-04-13 10:46:27+01 BW-UNION-04 softwareupdated[334]: SUOSUMobileSoftwareUpdateController: Download finished: (null)
2022-04-13 10:46:27+01 BW-UNION-04 softwareupdated[334]: SUOSUServiceDaemon: Successfully downloaded & prepared, proceeding to apply & reboot

Quite what it was doing for that 1h40m is a mystery to me.

I do think that JAMF desperately needs the ability to fire off the MDM commands from within a policy though so that it can just be triggered it would be a lot simpler than everyone needing to leverage the API especially since it's getting more complex with the tokens etc.

The API to me should be for people doing specific things that they need, if there's something everyone has to use the API for then it needs putting in as a core feature.

[bbarciz]

Just a thought from looking at the log lines that you provided, the 2nd line says that the connection was invalidated.  I wondered is that 90 minute is some kind of retry delay?

I do agree that some policy (or similar) for updates will be critical going forward.  I really need the ability to schedule machine updates at a certain time over-night to update labs and employee machines while they are not being used.

[ianatkinson]

I've seen that line a lot, I think it just means that it closed the session to the app store or something like that.

You can update your machines on a schedule this way, you need a script which uses the API to call the updates, wrap it in a policy then set the time/day limits on the policy in JAMF. This is how I do our students ones it's limited to Sundays which is the only day we're shut, the machines aren't on through the night as part of the sustainability policy we have.

[bbarciz]

It seems to me like this should be something so easy to accomplish through the basic GUI and not have a million hoops to jump through to make it all work lol.  Thanks for the info!

[jtrant]

It's definitely a lot of work, but totally worth it.

In my case we developed an MDM-based patch schedule using a jump host on the same subnet as my Jamf Pro server, rather than a script within Jamf Pro which runs locally on an endpoint, potentially exposing credentials. It did allow me to support headless Macs on Big Sur and above (including M1) where the softwareupdate binary is either broken or doesn't work at all.

The account I'm using also only has the ability to issue SWU commands to computers and mobile devices, so if the credentials were ever compromised they can't be used for any other purpose.

[bwoods]

Force a Computer Restart to Install macOS Updates - Jamf Nation Community - 265982