Posted on 01-23-2018 12:19 PM
As an FYI - we have Apple auto-updating and they released the specter/meltdown fixes. I've started getting a few people bringing me machines that are now crashing on boot up after installing, incl rebuilds that ran the updates.
Anybody else seeing similar behavior?
Posted on 01-23-2018 01:01 PM
Same here, but its due to our DLP software that has its digital hooks in the kernel. The 2018-001 update must of changed the kernel once again. So any DLP or tough Virus/Malware software that uses kernel hooks can cause the system to crash. We ultimately have to uninstall prior or go into safemode and remove after.
SMH. Is this installing automatically on your side? Because I have auto install security updates turned off in my enviroment.
Posted on 01-23-2018 01:37 PM
If it becomes epidemic, what I would suggest ignoring it with the softwareupdate command:
softwareupdate --ignore "macOS Security Update 2018-001-10.12.6"
But if you do so, I believe you will have to deploy manually as the softwareupdate will ignore this update.
Posted on 01-23-2018 03:10 PM
I can confirm we are also experiencing the same issues for users on 10.12.6
Yet to test on 10.13.3 ( High Sierra )
Posted on 01-23-2018 03:15 PM
We have 8 users experiencing the same issues.
Posted on 01-23-2018 03:18 PM
Posted on 01-23-2018 04:16 PM
@ShadowGT, they'll actually want to run
softwareupdate --ignore "Security Update 2018-001". Per the man page for softwareupdate:
--ignore identifier ... Manages the per-machine list of ignored updates. The identifier is the first part of the item name (before the dash and version number) that is shown by --list. See EXAMPLES.
You can test it by running
softwareupdate --list command twice, once before the ignore command and once after, and you should see the update not appear on the second run if the ignore was successful.
The nice thing is that then this ignore command covers both Sierra and El Capitan, since the update identifier is the same, without the OS version number after the last dash.
(Anyone else having trouble editing comments? Like it doesn't keep the original text and instead brings up a blank field and I have to copy and paste the original text, reformat it, and add the edit.)
Posted on 01-23-2018 04:19 PM
Any commonalities? We have Macs that have installed this that hasn't had problems, and Macs that have. Any anti-malware software installed? Any kernel extensions mentioned in the kernel panic message (if you're getting kernel panics)?
Posted on 01-23-2018 04:30 PM
Any additional info (specifics from kernel panic messages) is much appreciated. I ran the updates on my 10.12.6 MBP, am now on build 16G1212, have Symantec Endpoint Protection.app version 14.0.2332.0100, and haven't had any issues so far.
Posted on 01-23-2018 06:16 PM
Does anyone know if this affects 10.11.6 clients that have the 2018-001/Safari 11.0.3 patch?
Posted on 01-23-2018 07:58 PM
I've had six 10.12.6 MBP's run the update, all 6 had the issue. One user claims the issue fixed itself after a couple of reboots, two others had to have 10.12.6 reinstalled. The other three we have yet to try the reinstall. We were not running DLP on these machines but run a lot of other security agents that might be conflicting. I'll do more testing tomorrow.
Posted on 01-23-2018 08:06 PM
We were running an older version of Trend AV 3.0.1098, updated to 3.0.3044 and the issue is resolved for us. Likely issue was incompatible kernel extensions. I have tested successfully on two seperate 10.12.6 machines.
Who is using what AV and what version?
Try updating to latest available version.
This is very similar to Microsoft when they released their Meltdown patch which caused BSODs because of incompatible AVs.
Posted on 01-23-2018 08:09 PM
I had 2 users on 10.11.6 who ran this update and doing a kernel panic on their MacBook Pro
Posted on 01-23-2018 08:13 PM
I'm running Trend 3.0.1106. I'll play with that tomorrow.
Posted on 01-23-2018 08:16 PM
we are using Sophos Antivirus ver. 9.6.2, also Carbon Black. going to uninstall it and see if it fixes it.
Posted on 01-23-2018 08:29 PM
Posted on 01-23-2018 08:33 PM
Upgrading Trend to 3.0.3044 didn't work for me :(
Posted on 01-23-2018 08:48 PM
Hey @mcampbel ,
How did you upgrade to 3.0.3044? did you install over the top or did you run Trend's specific uninstaller and then run the installer? I have had issue with just trying to install over the top.
Posted on 01-23-2018 09:26 PM
@PatrickD , I "checked for updates" and let it update itself. Nothing's ever easy I guess. Thanks Pat, I really appreciate the help.
Posted on 01-23-2018 10:51 PM
For those running SEP14:
10.12.6- no kernel panics to report (installed security update and Safari 11.0.3)
10.11.6- will test to tomorrow
Posted on 01-24-2018 01:09 AM
To add a further point, both 10.11 and 10.12 patches change some things in the Kernel (required to patch the Meltdown vulnerability), so possibly causes could be anything that has a kernel extension. This would include AV, but could also include:
- Carbon Black and other deep level security solutions
- Some printer drivers (yea I know, right?)
- Data Loss Prevention software (an example is EndPoint Protector (EPP), but I don't know if that specifically causes, or indeed doesn't cause, an issue).
Safe Boot (hold down "Shift" on startup) should boot without third party Kernel Extensions, but may take longer to boot too. That might get you in enough to start troubleshooting causes
Good Luck, and I hope that helps
Posted on 01-24-2018 05:55 AM
@daz_wallace I can confirm carbon black's Bit9 v7 patch 8 is causing problems - and that version was released a week or two ago.
Posted on 01-24-2018 05:57 AM
Posted on 01-24-2018 06:04 AM
@daz_wallace and @jwojda I can also confirm the Carbon Back agent is causing the crash 10.12.6 latest security patch. Currently testing 10.13.3 to see if it has the issue as well.
Posted on 01-24-2018 06:07 AM
Posted on 01-24-2018 06:21 AM
@rqomsiya Oh that is a relief. I accidentally left my test machines set to install updates overnight and forgot to remove the security updates from our SUS.
I'm hoping the machines that got the update don't crap out...it was only a few thankfully.
Posted on 01-24-2018 06:29 AM
We're running Sophos AV 9.6.6 and Ensilo 220.127.116.11 here and seeing the issue. We do not use any Carbon Black products here.
Will test to see if it's Ensilo.
@sahmed Did you uninstall Sophos - if so, any luck?
Posted on 01-24-2018 06:31 AM
I am running 10.11.6 as well and encountering this same crashing error. Anyone able to resolve the issue?
Update: Reinstalling MacOS just got this error fixed. Thanks to rlee for the solution.
Garry Joshi https://dltutuapp.com/ https://show-box.ooo/ https://tutuappx.com/
Posted on 01-24-2018 06:37 AM
Forgot to mention that re-installing MacOS from the Recovery partition worked for me.
Give that a shot @garryjoshi
Posted on 01-24-2018 06:44 AM
Let's all open cases w/the vendors for confirming this compatibility. I'm hoping the message will be clear - we expect this kind of validation during betas if possible, or day zero otherwise.
Posted on 01-24-2018 07:10 AM
Carbon black has informed us that a patch is a week out. Their recommendation was to block the Apple update. Another solution is safe boot the mac (Hold Shift on startup), log in as an admin, then Uninstall the CB agent via terminal (sudo sh /Applications/CarbonBlack/sensoruninst.sh). Reboot and your Mac's Kernal will be happy again.
Posted on 01-24-2018 07:51 AM
We have Carbon Black running and that caused the crashes here
Boot in Safe Mode and delete the Carbon Black .kexts in /Library/Extensions
Posted on 01-24-2018 07:51 AM
Removing Carbon Black was also the solution for me. Thanks to @daz_wallace, @jwojda, and @mojo21221 ! Mac updated to 10.12.6 with the 2018-001 update for Sierra, rebooted, and could not boot up normally. I was able to get into safe mode, then remove CB using Terminal and Admin account.
Of note: simply trashing the Carbon Black folder does not solve the issue. If you have done this, reinstall CB, then use @mojo21221's solution: Uninstall the CB agent via terminal (sudo sh /Applications/CarbonBlack/sensoruninst.sh) and reboot.
Posted on 01-24-2018 08:03 AM
For me I had to uninstall Carbon Black and Trend to get the machine running again. I have not tried updating Trend to a newer version yet.
Posted on 01-24-2018 08:04 AM
We couldn't boot into Safe Mode or Single User Mode on nearly all of the systems affected by this, perhaps because we have both Carbon Black Response and Carbon Black Protect installed on our Macs. I figured out the files that need to be removed before the Mac could boot normally:
For Response, I deleted:
For Protect, I deleted:
I either booted the affected Mac to Target Disk Mode and connected it to a working Mac, and used the Finder to delete the files, or I booted into the Recovery partition and use Terminal to delete them (remembering to target "/Volume/Macintosh HD" in the commands). After removing those files, the Mac should be able to start up without kernel panicking, and the uninstall scripts for each product still remained, which I then ran in Terminal:
This worked on all affected Macs that I had yesterday.
Posted on 01-24-2018 08:30 AM
I am seeing this on 10.11 and 10.12, kernel panics after patch and restart. Booted in Safe Mode, removed Carbon Black, machine works again.
Posted on 01-24-2018 09:23 AM
I've installed it on about 90 Macs so far and none of them have crashed. We're running McAfee ePO 10.2.2, and all computers are FileVaulted.
Posted on 01-24-2018 09:26 AM
Just FYI, the only anti-virus/malware software my organization uses is Sophos. Our Sophos Central Endpoint clients are on 9.6.6 and we are not experiencing any boot-loops or kernel panics in my testing of our Macs running 10.11/10.12. I will continue testing and update you all if anything changes.
Just wanted to add another data point to the thread since I've gleaned some useful information from it. Thanks, all.
Posted on 01-24-2018 09:39 AM
It is Ensilo on our systems and not Sophos or OpenDNS. Did testing this morning with each software installed prior to installing 2018-001 to isolate which software was the issue.
After a macOS reinstall, the App store does show 2018-001 as being installed. I haven't seen any further issues on these systems. Even Ensilo console is reporting back properly.
Posted on 01-24-2018 11:38 AM
Okay so we have seen the same issues with Carbon Black. We are removing the b9kernel.kext by going into recovery mode, then using disk utility and terminal to remove the offending kext. On reboot this has shown to work well.