macOS Updates Fail To Install - Potential Workaround / Resolution

[cainehorr]

SOFTWARE UPDATE PROBLEM HISTORY
When macOS 10.13 "High Sierra" was first released, many in the Apple community may recall that macOS updates (ie 10.13 to 10.13.1, etc.) would fail during deployment.

Apple Software Update would download the necessary updates and begin installing. macOS updates require a reboot. During that reboot cycle, the macOS update installer would kick off and in many cases, would hang indefinitely in the neighborhood of 60%-75%.

The typical fix was to boot into Recovery Mode and install macOS on top of itself OR hold down the Option key on the keyboard and boot to "Macintosh HD", get a Desktop, go to System Preferences, go to Startup Disk and select the proper volume then reboot... and if you were lucky, the problem would go away.

Around macOS 10.13.3 (or was it 10.13.4?) Apple "said" they knew about the issue and they fixed it. So they said.

My team still sees this issue to this day with updates for any version of 10.13 and even when upgrading to Mojave.

A LESS THAN STELLAR WORKAROUND
So we ended up writing a script and put it in Jamf and we started blocking Apple Software Updates for macOS 10.13 and 10.14 updates. This suppressed the softwareupdate command line utility as well as the Apple App Store "Updates" for macOS revisions.

Users could still download the full macOS installer from the "Featured" section of the Apple App Store.

This was a time consuming process and most people don't want to spend 20 minutes downloading an installer file just to spend another 20-30 minutes not having their device while the installer does its thing.

But it was a necessary evil.

A BETTER WORKAROUND - THE BACKSTORY
Fast forward to this morning. I walked in through the front door and one of my Junior Admins told me about a user who was borked from a macOS update - the latest Security Update 2018-002 - https://support.apple.com/en-us/HT201222

So I decided to crack my knuckles and dive head first into this one myself. I tried the aforementioned workarounds - nothing worked.

I then attempted Single User Mode "just to see what I could see" - this went pretty far into the boot process before the software update installation kicked off, failed, and took me to what looked a bit like a Recovery mode.

I was presented with the option to view the logs for the software update. This was a first!

I started examining the logs and was able to save them to the desktop folder of the user in question.

In the logs I realized that the files associated with /Library/Updates/041-** were missing! Perhaps a smoking gun?

I performed an Option boot, chose Macintosh HD, had the user log in, got her Desktop, opened the log file I had saved and dropped into a terminal to examine /Library/Updates/ a bit closer...

Performing an ls -la revealed several older software updates that had been downloaded but never installed... all related to 091-**

I had a wild idea... let's purge this folder and manually run software updater and let's see what happens!

So begins my journey to identify a better workflow...

A BETTER WORKAROUND - THE PROCESS
Here are the steps I took...

# Deletes all files and folders contained within /Library/Updates/
sudo rm -rf /Library/Updates/*

# Clear the catalog URL back to defaults (requires privileges)
sudo softwareupdate --clear-catalog 

# Runs Apple Software Updates - install all updates and forces a reboot
softwareupdate -i -a -R

After updates were downloaded and began the initial installation kickoff, the device rebooted and the macOS security updates and other updates began running.

I have to authenticate at FileVault (expected) then saw a familiar macOS update splash page.

Then it went black for a minute or so before seeing the Apple log with a status bar.

I was then prompted to authenticate again before it wrapped up installation.

Around 20-30 minutes I was fully updated with a desktop.

I have since tried this technique on 6 machines with 100% success.

IN CONCLUSSION
Is this a guarantied workflow? I can't answer that. I've only run it on a handful of devices thus far. But it's a good sign.

That being said, I have plans to wrap this process/workflow up into a neat Jamf workflow/policy and put it into Self Service so user's who are prompted to run updates can go to Self Service and run this workflow on their own.

I'll be testing the Self Service workflow out with my local team first before opening the flood gates to my corporate environment.

Anyway - Thought I would share my findings. Hope it helps you out too!

WHAT ABOUT MOJAVE?
Since we have been seeing upgrade issues with macOS 10.13.x "High Sierra" to macOS 10.14.x "Mojave", I am planning on testing my prescribed workflow to see if this resolves our upgrade woes as well.

I'll post my findings as they become available!

Cheers!

[ppinter1]

Great catch! I'm looking forward to your update re: Mojave.

[joshuapinter]

Hi Cainne,

Thanks for posting this! I recently tried updating our Mac Pro with the latest High Sierra update, Security Update 2018-002, and ran into the exact same issue, even including the same error message in the Installation Log.

I'll give this workaround a go on an evening this week or over the weekend and see if I have the same success you do.

Cheers!

Joshua

[ukspvmalapati]

@cainehorr -- Good one. If this is handful of machines we can do this manually. How can we achive if we have multiple machines with policy?

do you have any suggestions on this?

[cainehorr]

@ukspvmalapati

You should be able to scope it any way you like or put it into Self Service.

[Lotusshaney]

FYI this won’t work on 10.14 as the /Library/Updates folder is now under SIP control

[cainehorr]

@Lotusshaney - INDEED!

We've found Mojave to be way more stable overall.

[tlarkin]

There are known issues with softwareupdate that were introduced in 10.13 (that worked in 10.12) and also exist in 10.14 and Apple has been addressing them. If you have Appleseed access there should be some mention of the fixes in the release notes.

However, running updates manually through the App Store or through System Preferences in 10.14, has always worked. This is why a lot of Orgs have shifted to a dialog box and nag workflow to nag their users to run the updates themselves. Apple has been fixing the issues, but a few still may exist. Apple is aware as well, but feel free to point this info to your Apple rep. The dev portal and Appleseed release notes should have these bug fixes listed.

Note that running softwareupdate -iRa will have mixed results, especially on T1 and T2 chipped Macs. You may want to drive your users to run their own updates via dialog box or some other mass comms. Of course, test everything out first.

[McGinn]

I am having issues with Security Update 2019-002 10.13.6

The fixes here did not help. I have tried to install the updates via the App Store and its still not working.

The computer will download the update and install everything. When you reboot you get the apple logo and process bar followed by FileVault authentication. Then the computer goes back to another apple logo and progress bar but ends up at the attached image.

I have multiple machines that have this problem....Is anyone else having trouble installing this security update?

EDIT: Manually downloading and installing the DMG from Apple's website worked for me
https://support.apple.com/downloads

[rnoureddine]

I've had a similar issue where software updates would, download, then reboot to install, it goes thru the entire process but then comes back at the same version. This has happened on some machines and on multiple mojave versions.

My first fix was to have a USB installer of Mojave's latest version and basically boot to the installer and run it from there.
My latest fix which i'm not sure it applies to the previous updates was when I updated to 10.14.6, supplemental update 2 was made available, and I ran to the same issue as before with that update.
from terminal, i ran softwareupdate -iRa and that applied the update, rebooted and all was good.

I hope this helps

[diradmin]

@rnoureddine We've observed the same, and now rely on the softwareupdate reboot mechanism. However, we're now observing that this does not always initiate the reboot, especially when machines are sitting at the Login Window.