Security Update 2019-001 (Mojave)

[cbd4s]

Just to share the chaos caused by the Security Update 2019-001 (Mojave) released by Apple on the 29th. Once it is installed, the OS build is 18G1012.

We've had a number of machines basically bricked as a result of installing this update. There are others reporting the same problem on Slack. It is likely the direct cause is the user/network interruption during the installation.

This is what we see on our affected machines. When they start we only see our generic support local admin account login but the usual password is not working. These machines are all relatively new with the T2 security chip and default start security setting that stops the machine from external disk booting. I saw people reporting on Slack that the machines with T1 chip are also affected.

So we boot into the recovery mode and found out the disk utility couldn't mount the volume container. In the end, we had to start into internet recovery mode, reinitialise the disk to rebuild the OS. This is a bit stuff-up from Apple as this OS update together with their new OS security protection features is the cause of potential data loss at the user's end.

Currently, we've deployed the policy for 10.14 machines not already on this built to ignore the Security Update 2019-001. It could be an overreaction. I probably should only target the 10.14 machines with T1 or T2 chip.

[franton]

Hi,

I too have been affected by this, and we have an outstanding support case with Apple Enterprise. I've forwarded your post onto them.

[Stevie]

Hi,

I glad I saw this post because we were just about to roll this out to all our Mac users. We currently have 2 options for our users via self service.

1) Install the full "Install macOS Mojave.app" which the application version is now 14.6.06 from 14.6.03. This takes 40 minutes.
2) Install the "SecUpd2019-001Mojave.pkg" update is downloaded from Jamf and takes 20 minutes to install.

We are not allowing our users to download and update via Apple Software Update servers but I wondering if this might be a mistake.

Has anyone seen the problem when doing a full OS install or is it just an update Security Update 2019-001 which is as problematic as the Supplemental update which Apple had a few goes of releasing.

Thanks

Steve

[dlondon]

Hi @cbd4s Any chance you can describe how your policy to ignore the updates works?

[wjc2a]

I had a network problem when i installed the update i am now running 10.14.6.
I fixed my network by downloading and the add to ad profile and installed this into the admin account on the iMac.
just something to look out for just make sure you are picking up all the Profiles.

[Stevie]

Hi @dlondon, We block all new updates using a configuration profile and set the "Defer software updates for 30 days (macOS 10.13.4 or later)" within restrictions. Next we set up the software update so that nothing automatically installs.

So far we have tested install the update on a few computers and one of them took almost an hour to finish the installation which was 3 times longer than the other computers. The update was downloaded to the computer and run locally. This computer had a clean install of macOS 10.14.6 and Office 365 it was also only built a few weeks ago and never used. We are now testing doing a clean install of macOS Mojave 10.14.6 (18G1012) to see if this is a better way to update the users computer. We already know this takes 40 minutes so we will try on a few computers to see it we have any problems.

[mark_mahabir]

@Stevie Where did you get the full installer for 18G1012? I just tried to download v10.14.6 from the App Store and got 18G103.

Also, Greg Neagle's installinstallmacos.py script shows 18G103 as the latest available build.

[Stevie]

I used a macOS 10.15.1 computer and run this command from the command line.

sudo softwareupdate --fetch-full-installer --full-installer-version 10.14.6

This downloaded the installer for me which I then packaged with Jamf Composer, uploaded the pkg for a policy which updates all the users computers.

[Stevie]

This is very strange, someone at another site has run the same command and he got 18G103 download. I wonder if Apple have pulled the update or have different versions on their update servers.

[franton]

If you attempt to install this update manually, there's a good chance the OS will just ignore it as you don't have the same "entitlements" as the softwareupdate framework ;)

[mark_mahabir]

@Stevie I ran that command on a 10.15.1 machine and got v14.6.06 of Install macOS Mojave.app, which I suspect is 18G103.

[brunerd]

We've had some too and there didn't seem to be a common thread... until you start asking the techs to ask the users for details, then you hear something like: "The user was annoyed the update was installing so they tried to shut down their Mac and restart it"

Details like that make all the difference when deciding to file an Applecare ticket -- and you usually find out about them later 😑

[damienbarrett]

Chiming in here. Had one come in yesterday with the same scenario and results. My situation is only unusual because this machine has never had FileVault enabled on it, yet it appears to be stuck at a FV unlock screen after boot and we can't get past it with any known passwords (and there is no recovery key in escrow; remember FV was never enabled).

It's possible that this user got impatient and shut down the computer mid-update but she hasn't confessed to doing this. But it's plausible.

I just emailed our SE and AppleCare rep. I'm opening an Enterprise ticket next.

[brunerd]

@damienbarrett So when you put the Mac in Target Disk mode and attach to another Mac does the drive mount as an unencrypted drive? If so, that is wild! Is this a T2 equipped Mac, BTW?

[damienbarrett]

No, the drive will not mount. I booted from another USB drive. I haven't tried target disk mode yet, but I expect the same result -- an internal drive that will not mount. I'll let you know. To boot from an alternate source (other than the built in SSD or Recovery), you have to disable Startup Security while booted into Recovery.

Edit: Yup, no change on Target disk mode. Same result. "Macintosh HD" from the affected machine shows up in Disk Utility but is greyed out. No mountable. Has every indication that it's FileVault locked (but FileVault was never enabled on this machine).

[carlo_anselmi]

Seems like Apple re-released both Security updates yesterday (at least that's what shows on my NetSUS)

[cbd4s]

@dlondon, it's pretty straight forward once know how coz I didn't :-)
Files and Processes > EXECUTE COMMAND: softwareupdate --ignore "Security Update 2019-001"
They will still be able to see the 10.15.1 upgrade. But I think the policy stops them from seeing the link to the additional update (2019-001). Technically, if the users download the security update from Apple, they will still be able to install it with admin rights. But I guess very few would be that keen and geeky.

[cbd4s]

@carlo.anselmi, on Apple's website, it's still the one released on the 29th: https://support.apple.com/kb/DL2024?locale=en_AU

[cbd4s]

@Stevie, so it sounds like even if the update package is cached locally, it will still take a long time to install and potentially get interrupted by the user. I remember reading it somewhere that the restart process during this update has a pause like 1 minute that there seems to be nothing happening to the machine.

[carlo_anselmi]

@cbd4s
Yes, it has happened the same in the past.
That's the pkg you can download directly from Apple website and it usually takes longer to see a more recent date/pkg there
The update available when running software update (from terminal or System Preferences) should be changed the same way it has changed when you update NetSUS/local Apple Software Update Server catalogs

[MatG]

We also had Mac bricked and require wipe and rebuild and have blocked this update. We have an Apple Enterprise ticket open bit so far not much response.

[jhalvorson]

Also seeing this issue. No accounts can unlock FileVault. The Individual Recovery Key is not accepted. In Target Disk mode, the "Thunderbolt External Physical Disk" and the "Container disk4" appear, but the Macintosh HD is greyed out and clicking on Mount does not trigger a prompt for a FV password. The command: diskutil ap list shows the following for that drive/volume:

Volume disk4s1
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk4s1 (No specific role)
    |   Name:                      Macintosh HD (Case-insensitive)
    |   Mount Point:               Not Mounted
    |   Capacity Consumed:         171872342016 B (171.9 GB)
    |   Encrypted:                 ERROR -69808

We would expect that the Encrypted status line should be:

FileVault:                 Yes (Unlocked) 
or
FileVault:                 Yes (locked)

[donnenright]

@jhalvorson I am seeing exactly the same output on two machines. the machines are not bricked, just the volume the OS was on. You can create a new volume and install an OS. Crazy part is that the FV2 user password unlocks the EFI password utility, just not the actual OS. I hope that: softwareupdate --ignore "Security Update 2019-001" prevents this from spreading...

[cbd4s]

@jhalvorson When you say create a new volume and install an OS, it is on the same disk and the original volume will be gone, right?

[rcurran]

We had about 50 systems we had to reinstall OS X on that seems to be related to this. We noticed the log files filling up in /var and think that was part of the root cause.

[franton]

Hi,

If you boot to recovery, which should still be accessible then run this:

xartutil --list

If you get the magic response:

Total session count: 0

then kiss the data goodbye. We've had this on all of our affected devices. This output means the T2 SEP has "lost" or "something has deleted" the keys to access the in built disk encryption.

[franton]

@brunerd I'm starting to think like yourself, this is user impatience self inflicted issue. " Hey, my laptop has a black screen, i'll just press the power button and break things."

I just didn't think it'd ever result in total data loss on a system.

As I posted above, I've verified all our devices affected lost their quote "anti-replay tokens - basically a pseudo random seed for a nonce / changing fact." With the end result none of the data is accessible or recoverable.

[bradtchapman]

Hi. We have 9 computers affected by this, all T2 machines. It was first reported on October 10. Affected High Sierra Macs installing the Security update on top of 10.13.6. We also had at least two with 10.14.x that were installing the Mojave supplemental updates, which contained similar security content.

[Raiders18]

Hi all, same boat here. The Mac is a 2019 MBP. I've been trying to mount the disk at least via terminal, but also not finding anything promising.

All shows error: permission denied, or shows some information was not available for internal lookup, or errors like below:

sh-3.2# sudo fsck_apfs -n -l /dev/rdisk3s1
mount_apfs: mount: Input/output error
error: mount_apfs exit status 73
sh-3.2# fsck_apfs -n /dev/rdisk3s1

@franton - very curious on the xartutil --list. When running this on an affect machine, the below is reported:

Total Session Count: 1

Entry: 0
UUID: 00000-000..... ' Session Seads: 1

Any thoughts on if this data/Mac is still able to be rev

[mark_mahabir]

Can anybody confirm or deny that the re-released updates are now available on https://support.apple.com/downloads ?

[Mhomar]

@mark.mahabir, I downloaded the package first thing this morning to check. It is still the October 24, 2019 .pkg. that said can anybody point me/us to and updated SecUpd2019-001Mojave.pkg? Very frustrating and I will not push the original patch.

[asiu]

I too have been affected by this, and we have an outstanding support case with Apple Enterprise. I've forwarded your post onto them.

@franton , is there any way we can access that case, or we can all somehow vote in or link all of our reports?

[franton]

@asiu just report in. they'll do the rest.

[jamesandre]

I've had one instance of this issue. Of note, the (VM) volume also had the ERROR -69808.

Link

[ClassicII]

I am tracking the issue.

Hopefully this will help get the issue some attention.

https://mrmacintosh.com/mojave-2019-001-security-update-causing-data-loss-if-interrupted/

If you find any new information, please let me know.

[mark_mahabir]

So can we safely assume that Apple won't be re-releasing these security updates, or have they in actual fact been updated at https://support.apple.com/downloads but have retained their original release dates?

I haven't seen any issues locally as yet (with the updates downloaded from the Apple Support website).

[damienbarrett]

FYI, Apple has acknowledged this and has identified a solution to be rolled out. No ETA was given, but my understanding is that it'll be fixed for Catalina first and they are working to port the solution backwards for 10.14 and 10.13.

I also composed an email to my entire user base alerting them of this problem. We do want our machines up-to-date, but we don't use a SUS, nor do we want to tell the fleet to ignore this update. So, we are taking the educational route, telling people about the problem, and the potential for data loss, but also to be patient and not interrupt the update as it's happening. This email was sent to our entire user base. Let's hope they read it and pay attention.

[carlo_anselmi]

@stevie
I have tried your suggestion but it replies this way

TestComputer:~ admin$ sudo softwareupdate --fetch-full-installer --full-installer-version 10.14.6
Password:
Downloading and installing 10.14.6 installer
Install failed with error: Update not found

Maybe they changed something....

[franton]

@carlo.anselmi that command works with whatever is present in the software update catalogs supplied by Apple. If they remove it, the update download disappears.

[carlo_anselmi]

@franton Sorry my bad, I forgot I had to reset the previously "ignored" updates on that specific test client...
I could successfully download Mojave 10.14.6 with

sudo softwareupdate --fetch-full-installer --full-installer-version 10.14.6

But at least in my case it is still build 18G103 (missing latest Security Update 2019-001 (Mojave)
Thank you again
Carlo