After enrolling, shutdown results in kernel panic before the shutdown can finish. (10.13.2)

isterling_goaaa
New Contributor III

Update 2018-03-14: This issue has been resolved with the newest version of the DLP and SEP clients, as I've stated in this comment.

Resolution: With thanks to @lrabotteau, we've been able to see that this issue is directly attributable to the Symantec 14.6 DLP client not being compatible with MacOS High Sierra. My IT Security group has a ticket open with Symantec and they've provided us with steps to remove the agent, as described in this comment. I'm now marking this issue as solved since it's not related to the JSS itself. Now we just need to find a resolution to the DLP issue without requiring a complete upgrade of our DLP infrastructure...

Issue: I have a fresh-out-of-the-box MacBook Pro 13" non-touch strip mac that I've just enrolled. One of my policies requires a reboot – SEP installer. When issuing the reboot or shutdown command, the computer ends up with a Kernel Panic just after the display LED turns off, and the machine will reboot on its own. When the machine finishes rebooting, I can then shut down or reboot the machine without errors.

This machine has macOS High Sierra 10.13.2, and is enrolled with our production Casper server. The server is running 9.98, which I'd like to update, but CAB has put a freeze on changes until after the holidays.

Has anybody else observed this behavior?

UPDATE: I've just filed a support request with JAMF and have attached the following procedures to replicate the issue:

Basically, the problem can be replicated in the following manner:

  1. Boot new or freshly upgraded mac with High Sierra on it.
  2. Upgrade to the latest security release and update all the system software patches.
  3. Enroll our machine in Casper.
  4. Run all policies. Last policy to run is our SEP installer, which issues a timed reboot command.
  5. Watch the machine crash just after the screen goes dark, when it should be going to the bootloader and starting up again.

It seems that the issue might be with the reboot script that gets put onto the laptop. I haven't had this issue with 10.13. I haven't tested with
10.13.1, but it is definitely happening with 10.13.2. I've reinstalled Sierra on my personal work machine and the problem does not replicate.

Any suggestions anybody can provide me would be greatly appreciated. Thank you in advance.

1 ACCEPTED SOLUTION

isterling_goaaa
New Contributor III

@lrabotteau Hey there, sorry for the delayed response. In our ticket with Symantec, they've informed us that we can remove the agent using the following process:

1) Stop the agent services. See related articles for more information.

sudo launchctl unload /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

2) Delete the Manufacturer folder located in /Library/

sudo rm -rf /Library/Manufacturer

3) Delete com.symantec.dlp.edpa.plist and com.symantec.manufacturer.agent.plist from /Library/LaunchDaemons

sudo rm -f /Library/LaunchDaemons/com.symantec.dlp.edpa.plist
sudo rm -f /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

4) Delete com.symantec.dlp.edpa.plist and com.symantec.dlp.bom from /Library/Receipts

sudo rm -f /Library/Receipts/com.symantec.dlp.edpa.plist
sudo rm -f /Library/Receipts/com.symantec.dlp.bom

5) Forget the pkg

sudo pkgutil --forget com.symantec.dlp.edpa

When I started this process, the first command actually caused a kernel panic. The relevant information in the backtrace is:

Kernel Extensions in backtrace:
com.symantec.dlp.fsd(14.6.2)[670315CF-5770-342D-9792-2E57688BA583]@0xffffff7f9bc6a000->0xffffff7f9bc7efff

BSD process name corresponding to current thread: edpa

So you might want to try unloading the agent first using the following the following command:

sudo launchctl unload /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

View solution in original post

19 REPLIES 19

lrabotteau
New Contributor III

Hello ,

I had the same problem when we have deployed 10.13.2 combo and after searching we found that is came from the DLP/SEP

After uninstall and install again SEP/DLP after 10.13.2 install , I had no Kernel Panic on all my iMac.

Edit : It's was only the DLP , actually we haven't install again it , we w8 for new version because its the same problem after install it (sorry for my bad English :-) )

isterling_goaaa
New Contributor III

@lrabotteau Thank you for the reply, I really appreciate it. Which version of the DLP are you installing? We're deploying 14.6 MP2.

I'm currently working on obtaining a more recent version of the SEP client that provides support for 10.13 (as per this matrix). I'm also working with my security team to find out if we can just disable the DLP on our 10.13 systems. I'll update as I find out more.

UPDATE: It seems like we're able to resolve the issue by removing the client from the DLP policy on the server end. I still have more testing to do, but so far things are looking up.

lrabotteau
New Contributor III

@isterling.goaaa I've deployed 14.6 MP2 too and get the same issue so we've removed this from al computer. I will test the next week with the new version (v15) of DLP to see if the problem persists.

isterling_goaaa
New Contributor III

@lrabotteau Thank you for the update. I'll see if our security team is ready to try v15. They did mention to me that they're currently behind the curve and haven't prepared for 15 yet. They are aware that the problem exists in the current release.

lrabotteau
New Contributor III

@isterling.goaaa Have you found a way to uninstall the DLP ? we see with the v15 , uninstall_agent binary is missing , have you the same on your side ?

isterling_goaaa
New Contributor III

@lrabotteau Hey there, sorry for the delayed response. In our ticket with Symantec, they've informed us that we can remove the agent using the following process:

1) Stop the agent services. See related articles for more information.

sudo launchctl unload /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

2) Delete the Manufacturer folder located in /Library/

sudo rm -rf /Library/Manufacturer

3) Delete com.symantec.dlp.edpa.plist and com.symantec.manufacturer.agent.plist from /Library/LaunchDaemons

sudo rm -f /Library/LaunchDaemons/com.symantec.dlp.edpa.plist
sudo rm -f /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

4) Delete com.symantec.dlp.edpa.plist and com.symantec.dlp.bom from /Library/Receipts

sudo rm -f /Library/Receipts/com.symantec.dlp.edpa.plist
sudo rm -f /Library/Receipts/com.symantec.dlp.bom

5) Forget the pkg

sudo pkgutil --forget com.symantec.dlp.edpa

When I started this process, the first command actually caused a kernel panic. The relevant information in the backtrace is:

Kernel Extensions in backtrace:
com.symantec.dlp.fsd(14.6.2)[670315CF-5770-342D-9792-2E57688BA583]@0xffffff7f9bc6a000->0xffffff7f9bc7efff

BSD process name corresponding to current thread: edpa

So you might want to try unloading the agent first using the following the following command:

sudo launchctl unload /Library/LaunchDaemons/com.symantec.manufacturer.agent.plist

lrabotteau
New Contributor III

@isterling.goaaa Thanks a lot , it's work perfectly :) with this !!

ocla__09
Contributor

Looks like this was release on the 24th. Might provide some relief?

https://support.symantec.com/en_US/article.ALERT2519.html

isterling_goaaa
New Contributor III

@ocla&&09 Thanks for the link. My DLP admin had just been notified that Symantec was releasing an update. She sent me a couple of files, but that one was not it. I'll send her an email to ask for that hotfix.

donmontalvo
Esteemed Contributor III

Had to deal with this today...#symatecSucks.

#!/bin/sh
# Uninstall Symantec DLP because the company's "uninstaller" does not work. 20180302 DM

launchDaemon1=com.symantec.manufacturer.agent.plist
launchDaemon1f=/Library/LaunchDaemons/com.symantec.manufacturer.agent.plist
folder1=/Library/Manufacturer
forget1=com.symantec.dlp.edpa

# Unload Launch Daemons
if [ -e $launchDaemon1f ]; then
    /bin/echo "...wow so $launchDaemon1 doesn't show up in 'launchctl list'..."
    /bin/echo "...no wonder its causing Kernel Panics...man Symantec sucks...Adobe flunkies me thinks..."
    /bin/echo "...ugh, we're going to have to force this incredibly badly written Launch Daemon to unload..."
    /bin/echo "...need a couple minutes, the knife didn't work, gotta whip out the flame thrower..."
    /bin/launchctl unload $launchDaemon1f 2>/dev/null
    /bin/echo "...great, finally destroyed that heaping blob of smelly bits...let's bury the body."
    /bin/echo "OK, now let's cover our tracks and bury the body..."
    /bin/echo "Removing $launchDaemon1f..."
    /bin/rm -f $launchDaemon1f 2>/dev/null
    /bin/echo "...#flipsBirdAtSymantec...#symantecBlowsChunks..."
else
    /bin/echo "$launchDaemon1f does not exist..."
fi

# Delete folder
if [ -e $folder1 ]; then
    /bin/echo "$folder1 exists, deleting..."
    /bin/rm -rf $folder1 2>/dev/null
else
    /bin/echo "$folder1 does not exist..."
fi

# Forget recept
if [[ $( /usr/sbin/pkgutil --pkgs | grep $forget1 ) ]]; then
    /bin/echo "$forget1 exists, forgetting..."
    /usr/sbin/pkgutil --forget $forget1
else
    /bin/echo "$forget1 does not exist..."
fi

exit 0
--
https://donmontalvo.com

Sanchi
Contributor

@donmontalvo this script worked for me. However only after disabling the agent in the DLP console first for the device I was trying to uninstall the agent from.

I'm running 10.12.6 and DLP 14.6 whenever I tried to use any of the advice from these sources:
https://www.symantec.com/connect/forums/how-do-you-uninstall-dlp-agent-mac#comment-11965751
http://eddiejackson.net/wp/?p=9022
https://help.symantec.com/cs/DLP15.0/DLP/v96304134_v120691346/Mac-?locale=EN_US
https://www.jamf.com/jamf-nation/discussions/26561/after-enrolling-shutdown-results-in-kernel-panic-...

It would always like clockwork result in the Mac immediately kernel panicking, then rebooting. The agent was still running after this reboot.

What worked for me was:
1. Login to DLP console.
2. Disable agent associated with Mac.
f7f1f7d9770d442c911cb0c9fa5f7a3a
3. Run donmontalvo's script against the now disabled agent Mac as an After policy in the JSS.
4. Stop swearing because theres no kernel panic in sight now.

Ofcourse we don't want people to remove the agent so its good it prevents it - I just wish it wouldn't prevent by executing the Macs its installed on first. What happened to nice dialogue boxes politely explaining you're not allowed to uninstall the agent you naughty user you?

ocla__09
Contributor

Has anyone seen a situation where the /Library/Manufacturer folder starts filling with tons of gigs of data that cannot be deleted?

I tried running the delete script before hearing about disabling in the console first which resulted in a partial uninstall. As a result I could not disable as communication was killed, so we deleted the client from the console and tried the uninstall on the client again which finished and reported that folder deleted, but it appears to still be on the machine.

Has anybody seen this, and have a method to delete that folder? Not sure I have ever run into a folder that is unable to be deleted.

When I try to manually delete I get the following output. It continues to ask if I want to override permissions, then does not let me.
I have tried in a previous attempt to recursively change ownership/permissions etc, all to no success.

override rw-r--r--  root/wheel restricted for /Library/Manufacturer//Endpoint Agent/temp/frbackup/13448_backupd_000290639a1535c9c9d91979a38edc44_13805_backupd_7eac6aceb960eac45879031270816a89_2697_backupd_ae5ffe6c1060866bad7dce8a8608d7dc_o4112-nd? Y
rm: /Library/Manufacturer//Endpoint Agent/temp/frbackup/13448_backupd_000290639a1535c9c9d91979a38edc44_13805_backupd_7eac6aceb960eac45879031270816a89_2697_backupd_ae5ffe6c1060866bad7dce8a8608d7dc_o4112-nd: **Operation not permitted**

isterling_goaaa
New Contributor III

The newest version of Symantec DLP package provided by my security team actually fixes this issue. I've been able to deploy our new DLP package to all managed macs and have eliminated the associated kernel panics altogether.

helpnickplease
New Contributor

@ocla&&09 I am also seeing this issue. Were you able to find a resolution?

ocla__09
Contributor

No, I have not unfortunately.

donmontalvo
Esteemed Contributor III

The latest Symantec DLP, which includes Symantec DLP EndPoint Agent MP2 14.6.0200.01053 and HF 14.6.0205.01006, fixes the problem...would point your Symantec DLP team to this article (hopefully they're engaging the vendor):

Monitoring macOS applications where SIP is enabled

Apparently the /Library/Manufacturer/Endpoint Agent/temp folder is protected by SIP. Based on the vendor's feedback, if a user has a bloated folder, you would need to disable SIP first, then purge the folder. Then the problem shouldn't come back.

We created an script to track the size of that folder in GBs (run via once-per-day, per computer policy), an EA scoops up the value of the TXT file. If we find any computers that were affected by the issue, we send tickets to local teams to resolve (since the steps require a touch).

Script

#!/bin/sh
# Check /Library/Manufacturer/Endpoint Agent/temp folder size. 20180426 DM

tempFolder=/Library/Manufacturer/Endpoint Agent/temp
checkSize=$( /usr/bin/du -sg "$tempFolder" | awk '{ print $1 }' )
resultFileGb=/Library/COMPANYNAME/SearchResults/checkSymantecDlpTempFolder_gb.txt

if [[ -e "$tempFolder" ]]; then
    echo "$checkSize" > "$resultFileGb"
fi

exit 0

EA

#!/bin/sh
# Check /Library/Manufacturer/Endpoint Agent/temp folder size. 20180428 DM

resultFileGb=/Library/COMPANYNAME/SearchResults/checkSymantecDlpTempFolder_gb.txt
readFile=$( cat "$resultFileGb" )

if [[ -e "$resultFileGb" ]]; then
    echo "<result>"$readFile"</result>"
else
    echo "<result>FileDoesNotExist</result>"
fi
--
https://donmontalvo.com

isterling_goaaa
New Contributor III

@donmontalvo Thanks for this. I've resolved this issue with a more recent version of DLP and Symantec that I'm installing on all my High Sierra machines. It's about time I asked for a newer version, so I'll reach out to my security team and have them get a newer version of the installer for me. Cheers!

Mageshg
New Contributor

As per my testing, if you wanted to remove the DLP completely even though DLP is enabled in console, run all the scripts in Safe boot.
It will do clean uninstallation of DLP from Mac machine. But is there any way to run the scripts in safe boot via JAMF. I have tried it manual safe boot uninstallation with success but when it comes to more devices, I like to know is there a way to do the same via JAMF.

donmontalvo
Esteemed Contributor III

Symantec can't confirm Safe Boot will work with SIP enabled.

--
https://donmontalvo.com