Enforcing Apple software updates in the year 2021.

[sdamiano]

While this is, arguably, a feature that Jamf should be able to handle at its core, what is the best method in 2021, across the gamut of supported OS's and architectures, to enforce Apple software updates?

I understand that theres resistance to using any terminal commands for software updates. As stated in the Nudge channel, Since 2018 this has been a problem since Apple doesn't test softwareupdate commands that are triggered by a script. In addition to that, this requires a password if the computer is Apple Silicon.

I recently rolled out the UEX-Tool-For-Jamf, and two of the components in it are no longer developed and throw gatekeeper issues in Big Sur.

I would like to use Nudge, but, my confidence in my users is low enough to not expect Nudge to work fine on its own. Nudge cannot force the update to happen.

What is the best way to handle software updates? Is there MDM magic I am missing? What is the downside to having the checkbox "keep my mac up to date" checked via a config profile?

[snovak]

And I just tried this yesterday, and it prompted for my user password before it installed..

[fgonzale]

Was this on an Intel or M1 Mac?

[jamfjosh21]

I have spent the last couple of days working with a colleague testing out various methods of trying to update Big Sur on our Intel iMacs since we noticed that the Software Update payload policy we had setup was doing nothing. These are iMacs in a higher education shared lab environment where our users are not logged on at the time we would like to action updates (ideally run automatically overnight).

We have tried the following...

1. Deploying the ‘softwareupdate -i -a -R’ command as a script via a policy in Jamf which was the initially advised workaround by Jamf Support. This was unsuccessful.
   
   
2. Deploying the ‘softwareupdate -i -a -R’ command in the Files & Processes payload in a Jamf policy. This was suggested by Jamf Support during a support call and we are advised this is the working “workaround”. From the logs this states that it completes and from the log details looks successful as it downloads the latest update and advises it’s restarting but this is actually a false positive the restart never occurs. The update downloads but the restart does not occur unless a user is logged in at the time the policy runs. The only way to kick off the update is by logging in and restarting manually from either software updates or running the ‘softwareupdate -i -a -R’ command in the terminal.
   
   
3. Deploying an additional policy with the Restart Options payload configured to ‘Restart if a package or update requires it’ and is set like this for both the No User Logged In Action and User Logged In Action. Hoping that with the software update being downloaded to the machine this would issue a restart and install the update that requires it. This did not work.
   
   
4. Deploying an additional policy with the Restart Options payload configured to ‘Restart Immediately’ for both the No User Logged In Action and User Logged In Action. Again hoping that the restart would trigger the installation of the downloaded update. It did not.
   
   
5. Resetting the NVRAM of some of the devices and trying to deploy the ‘softwareupdate -i -a -R’ command in the Files & Processes payload again. This was a bit of a shot in the dark but seeing some other admins advise that their devices were stuck on the black screen with the Apple logo looking like it was installing and doing this resolved it and installed the update. The Macs still would not restart automatically and required a user to be logged in.

It is pretty clear that updates for Big Sur cannot be managed automatically without some user involvement. On intel Macs a user needs to be logged in for Jamf's advised workaround (I use this term loosely...) of using the 'softwareupdate -i -a -R' command. M1 devices are another kettle of fish that we are going to have to deal with soon as well.

This is quite ridiculous to say the least. This OS came out 8+ months ago with advertised zero-day support yet here we are. Surely we can't be the only higher education organisation that will have to stick with macOS Catalina for the start of term unless we can get a working solution.

[dmichels]

Thank you for your detailed account of exactly what is happening in our
environment as well.
--
Debbie Michels
Apple Specialist
ITS Information Technology Services
Bergen Community College
Pitkin Building - Room A236
400 Paramus Road, Paramus NJ 07652

dmichels@bergen.edu
(O) 201-879-1694

*STAFF/FACULTY* Working Remotely Instructions at Bergen:
https://bergen.edu/REMOTE

*STUDENTS* Self Service
https://bergen.edu/self-service/

[dmichels]

Thank you for your detailed account of exactly what is happening in our environment as well. It has been too long for there not to be working fix. 

[AJPinto]

@jamfjosh21 "This OS came out 8+ months ago with advertised zero-day support yet here we are." Lets not forget that the ScheduleOSUpdates management commands were originally put in macOS 10.11, not macOS 11. Apple and JAMF have had 6 years at this point to get them working and have failed to do so. 

I am sure this is more of a problem on Apples side than JAMFs, but JAMF has actively avoided doing anything with macOS updates so far. Apple should have never killed the softwareupdate terminal commands functionality before getting ScheduleOSUpdates working right. Apple is so desperate to get Macs to function like iOS devices that they are not making sure things work before making changes. Apples QA is really garbage right now.

[James_Fred]

Has anyone tried this yet?

https://docs.jamf.com/best-practice-workflows/jamf-pro/managing-macos-updates/Running_Software_Updat...

It seems to work with Catalina, but I'm not sure if it works with Big Sur.

[dmichels]

This does not work.

--
Debbie Michels
Apple Specialist
ITS Information Technology Services
Bergen Community College
Pitkin Building - Room A236
400 Paramus Road, Paramus NJ 07652

dmichels@bergen.edu
(O) 201-879-1694

*STAFF/FACULTY* Working Remotely Instructions at Bergen:
https://bergen.edu/REMOTE

*STUDENTS* Self Service
https://bergen.edu/self-service/

[dmichels]

You are correct this no longer works for Intel based Macs running macOS 11 Big Sur any version.

[taugust_ric]

This is a long thread, and I've been following along as well with multiple bags of popcorn...

It looks like there's a bug in Big Sur where the softwareupdate tool will not execute a restart if a user is not logged into the system.  I've been playing around with softwareupdate commands using --verbose and the output is quite different when executed with a user logged in versus when a user is not logged in.  It looks like a lot of changes to softwareupdate were made under the hood to accommodate Apple Silicon based OS update processes which operate more like iOS than previous versions of macOS thanks to differences in lineage.  I don't think this is by design, but since we're at version 11.5 at this point and Monterey 12.0 is around the corner, probably not worth going through the effort to open a support ticket with Apple support about this.

In any event, I think I've gotten around this for the labs I manage by creating a combination of policies that during our maintenance windows overnight, will boot our systems automatically with an account setup for automatic login, then runs a policy to run softwareupdate and install and download all updates.  That softwareupdate script also removes the automatic login flag so that at next restart after the updates it's back at the normal login window.

I'm still testing this, so I'm hesitant to post any script/code examples yet, but figured I would just post this as a more "thinking in writing" kind of post for anyone else that's stumped on how to push these updates reliably to computer lab like scenarios where there won't be someone in front of the keyboard and mouse to say yes.

[grayw]

I am very interested in where you're going with this. Please keep us updated!

[taugust_ric]

I did more testing after I posted and it seems to be working as I hoped.  I'll post more details when I get a chance.  The basic requirements:

- A script, executed by a policy, that writes the proper values to allow an account to automatically login at next restart.  The policy also executes a restart at the end of the policy run.

- A script, executed by a policy to run at startup, once per day, to execute the software update command when the system is logged in.  This script also removes the auto-login values so that after software updates run, the system is back to booting to a login window.  The policy also has a reboot built-in, in the event no updates are found, so the softwareupdate tool doesn't force the restart first.

So far this has worked with two of my labs pretty well.  These are Intel iMacs.  I don't believe this scheme will not work for M1 Macs - but I don't have any of those in a managed lab setting yet.

[anuj530]

This is exactly what I was looking for! Thank you. My only issue is, how are you waking up the sleeping machines? I have a few conference rooms and classrooms machines that I would like to update automatically (atleast minor software updates) and the biggest issue I am running into is waking them up remotely. Any suggestions? Also do you have your script shared on github? 

[taugust_ric]

I actually had to abandon the strategy.  Most of it worked except sometimes they got stuck in a loop and were never able to remove the automatic login, even though it had been scripted.  I'm a novice at scripting and ran out of time to get it working without causing class disruptions so I backed out of the policies.

That being said, someone with more advanced scripting skills than myself could most likely get this to work reliably.  When the scripts did work the systems did download and install Big Sur updates on their own and then restart to complete them.

[atrystan]

A bit late to the game, but I have separately realized that I need to do pretty much this process in my own environment. Did you get it working?

[AJPinto]

We have already made the call to skip Apple Silicon on Big Sur. It is really not a finished product. Things like the forceDelayedSoftwareUpdates and enforcedSoftwareUpdateDelay keys for the ScheduleOSUpdates policy not being added until Monterey despite Apple breaking any way to do this with softwareupdate on Apple Silicon is just proof apple is releasing incomplete products. We have not had any reliable luck on getting OS updates to work on Apple Silicon. They will run with the Management command, but when they run is totally in the air. Also the Management command not canning a FileVault approval is making lab testing really annoying while remote. 

I honestly think Apple Silicon was released before it was ready, probably Apple trying to get back at Intel.

[scottb]

100%

[James_Fred]

I'm glad to know that I'm not the only one struggling with macOS patch management. 

Has anyone tried using Nudge + M1? Or is another hair pulling experience?

[MrRoboto]

Nudge will nag your end users to update but it is up to them to click Install. I'm doing a similar thing using OSUpdateNotifier.sh.

[daniel_ross]

We love it personally.  Our security team signed off on it after the last few zero days.  I think the biggest is key is communicating to your users what it is.  Also, have documented policies for how long a patch can go without being applied.  We have a set amount of days before we enforce it.

[ianatkinson]

I haven't done any M1 testing yet, I have 1 test macbook which is due to come out of the cellophane this week but that's in advance of hundreds of M1 iMacs turning up so I need to crack on with it.

I see in the release notes for 10.31 is says that you can reboot an M1 mac using a management command from within the policy which removes the need for authentication. It seems to be suggesting that if you make a policy which does the jamf updates payload followed by that it should work?

Has anyone tried this and if it's not working where does that fall over? I'm guessing the management command might not always work. Also if they release M1 MBPs which still have a touch bar (I wouldn't be sad to see them get rid of it I don't see the point) I wonder if the management command would know when to do the shutdown rather than reboot if needed.

[AJPinto]

The ScheduleOSUpdates management command is self contained. I am not sure if an external reboot from any source whether it be another management command or scripted would trigger updates to run. With 10.29 (I think) JAMF changed the default action on ScheduleOSUpdates to install ASAP (I forget what the key is). We are still running JAMF 10.26 so I have not tested that yet.

However any enforcement of macOS updates will come with Monterey, however I am still not sure how reliable that will be. If the Mac does not get the command, or something borks in the command I bet updates still wont run regardless what you set the Management Command to do. Lord only knows when JAMF will update to add all the new keys and features for ScheduleOSUpdates, I am figuring 1st quarter next year. I very highly doubt it will be with their day 1 "support" of macOS Monterey which we wall know is just sales talk. 

Only One M1 MacBook to test on before an entire fleet of M1 iMacs. Yikes. I have had a MacBook Air and 2x Mac Mini's to test with since January and this is still a mess. Apple Silicon and managing updates is a total mess. If you have not tinkered with Apple Silicon at all, look in to queueing Rosetta to install via script; odds are you will need Rosetta.

[ianatkinson]

I don't think they're suggesting you can do the MDM install via a policy, just the reboot part? This is what the notes say:

You can now create a policy to use the RestartDevice MDM command to restart computers in your environment. This includes the option to rebuild the kernel cache with specific kernel extension (kext) paths.

To use the RestartDevice MDM command, create a policy with a Restart Options payload. If you want to rebuild the kernel cache, select MDM Restart with Kernel Cache Rebuild and specify the kext paths.

Computers with Apple Silicon (i.e., M1 chip) must have a Bootstrap token escrowed to Jamf Pro in order to leverage this command. Computers running a version of macOS prior to 11.0 cannot leverage the the kernel cache rebuild functionality of the RestartDevice MDM command.

I take that to mean you'd have a policy which says

1) install all SWUs

2) Reboot 

and then the reboot bit would use an MDM command if it knew the computer was an M1? I may be totally wrong though!

And I am very late with starting the M1 testing yes, things have been quite up in the air still due to COVID and being in education we've been unsure exactly what we'd be buying and so on.

[dmichels]

I am talking about Intel chips, not M1 chips. I know we have to boot M1s manually, however, since upgrading to Big Sur on Intel Chip iMacs I also have to boot manually which is not how it is suppose to work. Jamf should have resolved this issue by now. 

[AJPinto]

Apple changed a lot of things with softwareupdates between 10.14 and 11.1. JAMF needs an escrowed bootstrap token to run software updates for one. There are also dependencies on the version of JAMF you are running. If you are using mobile accounts that do not have admin access they will not have a secure token to approve updates. My guess is something is off with your environment and configuration. Have you opened a ticket with JAMF? 

Using a script to run software updates is functionally no different than typing the commands in to terminal. The only real difference is authentication, does your JAMF instance have the bootstrap token for all your Macs to tell macOS to run updates. I am assuming you dont have some random SSL redirection/inspection tool like Forcepoint or Netskope doing inspections and redirections which will kill Apple packets. 

I agree JAMF should have dome something formal a long time ago with MacOS updates by policy. However, Apple has killed macOS updates by policy so there is no sense in JAMF developing this channel any further at this point in time. That ugly Management Command is the future and where I would spend my effort figuring things out.

[ianatkinson]

I've had to update our big sur stuff twice in the last couple of weeks, once to fix an Apple bug (11.4 -> 11.5.2) and once to fix a scanner bug (11.5.2 -> 11.6).

The only way we can achieve this at the moment in an educational environment is manual updates on each machine (not happening) or pushing out the entire Big Sur installer into /tmp then using that to do a minor update which is a lot of network traffic but does work OK.

The MDM commands are unpredictable and unreliable. I bulk sent it to a room of 20 machines the other weekend, 1/20 of them updated!

Plus I can't be sending MDM commands on a Sunday (the only day we're shut) if it might mean machines randomly update during classes on Monday in the middle of a lesson. That's even if I did want to spent my Sundays sending MDM commands which I most certainly do not.

Still, I did try softwareupdate on my Monterey beta test laptop the other day and it worked OK, so will hopefully work again when that comes out?

[dmichels]

Please test! But this may be the answer must be run as a script:

/usr/sbin/softwareupdate --install --all --include-config-data --restart --force

[ianatkinson]

I'm not sure what difference the --include-config-data flag is supposed to make (not seen that before) but in testing that doesn't make any difference for me.

On an 11.5.2 machine which can see it has the 11.6 update available it still fails to restart and you can see

softwareupdate[14222]: SUOSUUpdateController: Failed to issue software update restart

in install.log the same as it was doing previously

Ian

[dmichels]

Try adding this to script: 

#!/bin/bash

# get the currently logged in user
currentUser=$( echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }' )
echo $currentUser

# get the current user's UID
uid=$(id -u "$currentUser")

# convenience function to run a command as the current user
# usage:
#   runAsUser command arguments...
runAsUser() {  
  if [ "$currentUser" != "loginwindow" ]; then
    launchctl asuser "$uid" sudo -u "$currentUser" "$@"
  else
    echo "no user logged in"
    # uncomment the exit command
    # to make the function exit with an error when no user is logged in
    # exit 1
  fi
}


# main code starts here

# run the softwareupdate command as logged in user
echo Downloading software update now...
echo Running policy as "$currentUser"
runAsUser softwareupdate -d

#give time for software update to download (30 minutes)
sleep 1800

# run restart as root
echo Running restart...
/usr/sbin/softwareupdate --install --all --include-config-data --restart --force

[dmichels]

I think adding the top part of script makes sure the Root user is running it. Please let me know if it works. 

[ianatkinson]

The test I did was as root, doesn't make any difference I'm afraid. It's not down to permissions the binary just can't do the reboots any more for some reason.

[AJPinto]

The concept of @dmichels script may work, but you could not use Root. It would have to be a local admin account, and you would probably need to use apple script to can the 2 required password entry's. 

Apples direction is to use management commands to handle softwareupdates. JAMF is really behind in supporting these. I recommend opening reaching out to your JAMF CA and raising hell to get JAMF to properly support this new work flow. We are upgrading to JAMF 10.26 tomorrow to add more features but from what I am reading it is still way behind where it should be with software updates.

[dstranathan]

There are more granular deferral options starting in Jamf Pro 10.32. Many require macOS 11 Big Sur or higher (i.e.; macOS 10.15 Catalina still requires the deprecated softwareupdate --ignore XXX command via script/policy to block major macOS updates such as macOS Monterey 12.0.)

[harsha]

Hi dstranathan,

If we enable the defer updates, that means the latest version for software updates will be install after 30 days? or we will get notification after 30 days?

[AJPinto]

If you enable defer updates then macs will not see updates for xyz days after they become publicly available. Keep in mind if you use a MDM command to install updates it does not respect this deferral so plan accordingly. 

In practice. If apple releases 11.7 today, and you have a 7 day deferral end users will not see any notifications about it until 11/1. If you tell a mac to run any updates between now and the 1st they will pull down 11.6, unless you use a MDM command which for some reason bypasses this and will install 11.7 (working as intended its just stupid). 

[dmichels]

It must be run as a script. Please try and test this... 

#!/bin/bash

# get the currently logged in user
currentUser=$( echo "show State:/Users/ConsoleUser" | scutil | awk '/Name 😕 { print $3 }' )
echo $currentUser

# get the current user's UID
uid=$(id -u "$currentUser")

# convenience function to run a command as the current user
# usage:
# runAsUser command arguments...
runAsUser() {
if [ "$currentUser" != "loginwindow" ]; then
launchctl asuser "$uid" sudo -u "$currentUser" "$@"
else
echo "no user logged in"
# uncomment the exit command
# to make the function exit with an error when no user is logged in
# exit 1
fi
}

# main code starts here

# run the softwareupdate command as logged in user
echo Downloading software update now...
echo Running policy as "$currentUser"
runAsUser softwareupdate -d

#give time for software update to download (30 minutes)
sleep 1800

# run restart as root
echo Running restart...
/usr/sbin/softwareupdate --install --all --include-config-data --restart --force

[Musicmaker]

 @dmichels  This script looks nice, but are you using this in combination with a policy that can be deferred? It's doing a forced restart and I don't want my users 'surprise' with a forced restart without any notification(s).

[scottb]

Please pardon me today as I'm in a crappy mood, but look at this thread (not to mention many others) and tell me we don't have a problem.  This is insane!  We're using paid for products on an OS that is made by the most valuable company in the world, and here we all are trying to figure out how to update said OS?  This is nuts, and it's driving me nuts as well.

I just don't get why it takes so much wrangling to do a basic task in managing computers.  Again, apologies - just seems like a circle jerk at this point...praying Monterey brings some sanity to this topic (and others).

Thanks to all who have contributed to this - lots of good stuff here to ponder and try.

[AJPinto]

Wait, are we forming a choir? Sounds like you are preaching to the choir here lol.

I am actually going back and forth with JAMF over this right now in email. They keep saying installASAP is a great way to manage updates, and an engineer admits it cannot force updates if something prevents a reboot. So, not a great way to manage updates. Also if you only use installASAP in Monterey users get a dialog box they can use to abort. YAY.

It is beyond my why JAMF refuses to add InstallForceRestart, the engineer just kept stressing how this key can cause data loss. Like dude, we don't care we need to force updates. If you use MaxUserDeferrals it looks like Apple automatically enables InstallForceRestart when deferrals are exceeded. However lord only knows when JAMF will add MaxUserDeferrals to their framework.

So in short, no you don't manage OS updates in 2021. At least there is no forcing compliance on Apple Silicon devices (not going in to the issues others are having with intel devices after 11.4 not wanting to update). Apple makes the tools, JAMF is just not supporting them. Not to free apple from blame, this has been a horrible transition managed by them from the terminal command to MDM commands.

[taugust_ric]

This issue may become worse for us shortly.  The workaround for many of us was to deploy the full macOS Big Sur installer via Jamf policy and install the latest macOS updates that way.  With Monterey's release imminent, I'm curious to if the "Install macOS Big Sur" app/InstallAssistant gets updated going forward.  If you download the latest Catalina installer, it has none of the latest security updates applied.  I'm curious to see how Apple numbers security updates (11.6.1, 11.6.2) going forward with their new numbering scheme, and/or, if 11.6 remains the final OS version and just the build number changes.  The strategies for numbering may affect the strategies for keeping their OS installer current...