High CPU usage from com.jamf.protect.security-extenstion

[tk]

We're using Jamf Now with Jamf Protect enabled and periodically seeing CPU spikes (with the process hanging and eating up resources indefinitely) caused by the com.jamf.protect.security-extenstion. This is actually causing the OS to get unresponsive and overheat, eating up all available CPU. The simple solution is to kill the process, but eventually the problem comes back.

Some basic debug information from the pid on a machine from when the problem occurred:

sudo dtruss -p 337
dtrace: system integrity protection is on, some features will not be available

SYSCALL(args) 		 = return
sigreturn(0x700008F16550, 0x1E, 0x1F99DBCB69B66C71)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x2EECB3AAFCC39E5E)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x5ECF2791121B465B)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xDCFC18327AB19367)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x131DCCD7A886722F)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xA6420414AE3C2D83)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x774160C6BC097B03)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x9CF5D78ADB397C7C)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xC695A61C98B23746)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x2E174C7243C6C3C)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x8475397DD123F821)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xB86A855D5C6D5582)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x33D38C31FCA52252)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x837887A519FD4360)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x57F08AB2F4CE5C4C)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x1452E243428B300B)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x512AD858951CCC8)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xCB123E6E890BB73)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xA74E8C22E5DAB37D)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xE1337532B76B5F4)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xF3D49E3526C825B5)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x9B382C79A3AF143C)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xE3478EE01738A3FB)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x3E9B48F4D3586447)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xE6B16B5E42609B19)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x694937F7D31E87DB)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xEC47F2F19874D6A3)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xF0EF461A890F4794)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x1AD3BBBA94BF6683)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xE59DCD0E9A8C787B)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xBC7110328B3402B7)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xB0D724F06D5A9148)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x2F59C457FDE2291F)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xCB4DF9599A7246A7)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xD7413C2FCF9AED4F)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xBAF1DA1780A03DD)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x3097FF42B964EBFB)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x7401D005F7749F02)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x3E46AF12BE3ACC53)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x27D8CDC1C73788B1)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x4A74F06CB1103776)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x64D67AB482C2EB9E)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x9314F366DD84EC76)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x7E08A312D1A28009)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xB813024A3C5BDB1A)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x1B1F8EBC893B4B0D)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x5A978B423CC387E7)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xEBADEF2959CFF180)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x5C3FAA00D61FB987)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x15CE2C94340BEA3D)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xC98EA9F9E8C84028)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x2010E19527E30C37)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xC6E313CF4AB76641)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x8785416CD1E73DD8)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xEF942E90885B70AC)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xC0841954B7EACEB9)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x32E5D4C3597F97A9)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x3A39DA7639F1D250)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x1D5A71745EBD3E41)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x961C032FCF13926D)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x15B6281324252B5E)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xD2887F5320CB2577)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x90A9473C0A0D6D54)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x901A97CA0EDD0FC8)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xAB2B5CC4850C8064)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x90F4F40655AE2218)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x78B563E2556A909F)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x2AFC246BCC17EC72)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x6AEDB40B20473B94)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x2D4E8E78AF86ABE5)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x71265E2E561FE22)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x505CDD59A51F9DBE)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xD57DAEE899531CB3)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x64D7000B1A3DE68B)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x45A55A7C80C360FE)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xE9867906907587C3)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x65CDD0610F2595A8)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0xE20D2E98FDB18D65)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x3BF047FFF91D0D41)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x925FDE3A4E3B0D69)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xC62ADDD766062425)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x640DC2D247C9E970)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x1530C30DAC96B81)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x76764F51FA9E3348)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0xF13700255B850A65)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x4A36C8B169315FA3)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x463519A381052379)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x8F919933BE986993)		 = 0 -2
sigreturn(0x70000909F568, 0x1E, 0x162C92B5694B0805)		 = 0 -2
sigreturn(0x700008F16550, 0x1E, 0x17C641DD12F93664)		 = 0 -2
dtrace: 238154 dynamic variable drops with non-empty dirty list

> sudo lsof -p 337
Password:
COMMAND   PID USER   FD      TYPE             DEVICE SIZE/OFF                NODE NAME
com.jamf. 337 root  cwd       DIR                1,4      640                   2 /
com.jamf. 337 root  txt       REG                1,4 12433296            28362867 /Library/SystemExtensions/1276F63E-603C-4E34-B5CD-2FA3DE9F5D01/com.jamf.protect.security-extension.systemextension/Contents/MacOS/com.jamf.protect.security-extension
com.jamf. 337 root  txt       REG                1,4    46944            30534913 /Library/Preferences/Logging/.plist-cache.T66NLeyt
com.jamf. 337 root  txt       REG                1,4    32768             7146411 /private/var/root/Library/HTTPStorages/com.jamf.protect.security-extension/httpstorages.sqlite-shm
com.jamf. 337 root  txt       REG                1,4    56384            29063318 /private/var/db/nsurlstoraged/dafsaData.bin
com.jamf. 337 root  txt       REG                1,4   443920 1152921500312329445 /System/Library/Frameworks/Security.framework/Versions/A/PlugIns/csparser.bundle/Contents/MacOS/csparser
com.jamf. 337 root  txt       REG                1,4   234080            28717902 /private/var/db/timezone/tz/2022f.1.0/icutz/icutz44l.dat
com.jamf. 337 root  txt       REG                1,4   120549            30535436 /private/var/db/analyticsd/events.allowlist
com.jamf. 337 root  txt       REG                1,4    32768            30534938 /private/var/db/mds/messages/se_SecurityMessages
com.jamf. 337 root  txt       REG                1,4 14762160            28362877 /Library/SystemExtensions/1276F63E-603C-4E34-B5CD-2FA3DE9F5D01/com.jamf.protect.security-extension.systemextension/Contents/Frameworks/ObjectiveRocks.framework/Versions/A/ObjectiveRocks
com.jamf. 337 root  txt       REG                1,4 30399984 1152921500312794842 /usr/share/icu/icudt70l.dat
com.jamf. 337 root  txt       REG                1,4  2177216 1152921500312782999 /usr/lib/dyld
com.jamf. 337 root    0r      CHR                3,2      0t0                 317 /dev/null
com.jamf. 337 root    1u      CHR                3,2      0t0                 317 /dev/null
com.jamf. 337 root    2u      CHR                3,2      0t0                 317 /dev/null
com.jamf. 337 root    3      PIPE 0x72cc79a3fe975f22    65536
com.jamf. 337 root    4w      REG                1,4    15802            30535355 /Library/Application Support/JamfProtect/db/LOG
com.jamf. 337 root    5r      DIR                1,4      608             7146258 /Library/Application Support/JamfProtect/db
com.jamf. 337 root    6      PIPE 0x71a00588162061ef    16384
com.jamf. 337 root    7u      REG                1,4        0             7146338 /Library/Application Support/JamfProtect/db/LOCK
com.jamf. 337 root    8w      REG                1,4       62            30535356 /Library/Application Support/JamfProtect/db/MANIFEST-000611
com.jamf. 337 root    9w      REG                1,4        0            30535358 /Library/Application Support/JamfProtect/db/000612.log
com.jamf. 337 root   10u      REG                1,4     4096             7146407 /private/var/root/Library/HTTPStorages/com.jamf.protect.security-extension/httpstorages.sqlite
com.jamf. 337 root   11u      REG                1,4   852872             7146410 /private/var/root/Library/HTTPStorages/com.jamf.protect.security-extension/httpstorages.sqlite-wal
com.jamf. 337 root   12u      REG                1,4    32768             7146411 /private/var/root/Library/HTTPStorages/com.jamf.protect.security-extension/httpstorages.sqlite-shm
com.jamf. 337 root   13   NPOLICY
com.jamf. 337 root   14u     unix 0xa9584682f9389fdf      0t0                     ->0xa9584682f9387a5f
com.jamf. 337 root   15u    systm 0xa958467e2dac6897      0t0                     [ctl com.apple.netsrc id 6 unit 3]
com.jamf. 337 root   16      CHAN             flowsw                              60EA3EE6-3AE3-4378-A931-5372928353F0[2] user-packet-pool

[MattT]

Hey @tk , just on the chance you haven't already done so please be sure to touch base with Jamf Support through the usual avenues to ensure they're able to investigate and work with you to resolve this.  Definitely not the experience we expect nor want to see!

[AJPinto]

I agree with @MattT , this is something I would open a ticket about. If there is a product issue, this is not where it will be noticed. 

[yourmindrewind]

Hey @tk Did you get this resolved?  We've experienced the same issue on select machines.

[MattT]

As a follow up to this, there are certain circumstances or use-cases where high velocity, expected activity can be ignored from detection workflows to ensure only the necessary analysis is taking place.  The Exceptions feature can be used to effectively achieve this for those users and use-cases where it makes sense, such as a software developer compiling code in a very specific directory with a very specific application.  Documentation can be found here.

In either case, discussing this with the Jamf Tech Support team is still recommended to ensure we're either finding and squashing any potential issues or helping implement Exceptions successfully.

[yourmindrewind]

@MattT Thanks for taking the time to reply :-)  I've also raised a case.  I presume Exceptions can only be added in the full version of Jamf Protect not the version that is enabled via Jamf Now

[MattT]

More than happy to @yourmindrewind !  That's why we're here :) You're correct, the Exceptions feature can only be leveraged with the full version of Jamf Protect.  As such, definitely recommend continuing to work with the Jamf Tech Support team to investigate further 👍

[chrissnyder]

I've experienced this issue many times. I have to kill the com.jamf.protect.security-extension several times a week to keep my laptop from turning into a hot plate.

[yourmindrewind]

@chrissnyderWe are still experiencing the issue as well.  Seems to be affecting more and more of ours machines. 

[jbutler47]

As a follow-up, curious about what may have been put into place to exclude MS updates. Can anyone make/share a recommendation? 

Thanks.

[dontmakememac]

@tk checking to see if you have any new insight on this issue?

Additionally, do you have a Jamf Support case # I could reference if I open my own?

I'm receiving reports of similar behavior across our fleet and am trying to get a grasp on where to begin troubleshooting - most machines are macOS Ventura still. 

Thanks in advance.

[tk]

@dontmakememac The response from support was that this might happen when there are multiple file changing in a short period of time. Today I saw this affecting two computers, which recently were updated to MacOS Sonoma. This gets picked up by the users as their computers become hot and sometimes loud (ventilation), and we ask them to kill that process to get back to a "normal" state.
What I want to do is to get one device affected, which I will just keep on high CPU load indefinitely to see if that spike actually ever gets back to normal levels (so is there really something happening in the background that just completes at some point).

Overall I wasn't able to pinpoint any specific circumstance that caused this problem to appear.

[tk]

I was able to once again reproduce the issue and collect some basic debug info, which I provided to support today. Will circle back if I hear back.

[HeyVyner]

Hi all,

Can recommend making sure macs are in low power mode for the time being on battery.

This will stop any apps that have high processing to be limited while Jamf look into the issue.

[maxlevine]

I am also seeing this issue affecting my users. I would interesting in learning about any mitigation steps. Thanks

[tk]

Hey, an update from my side - I was able to capture diagnostic information while the issue was occurring (as instructed by support) and submitted it to the Jamf Team. Waiting for their response.

If you see this issue happening, please use this command to collect the diagnostic information:

sudo protectctl diagnostics

This will generate a zip file containing verbose log from the Jamf Protect process. This file should then be submitted to support.

[nb]

Hi @tk keep the updates coming, also experiencing this on my mac (2019 MacBook Pro 16, i9, Radeon 5500M)

Interested to know what the Jamf team reply with.

[dontmakememac]

I also have a case open w/ Jamf Support and we've been collecting logs over multiple devices. Without having hard evidence, we're receiving more and more reports of this issue. 

So far, Jamf Support has reviewed our Analytic Sets (nothing substantial found) and is also advising us to generate logs using the following command:

protectctl diagnostics -d 10 -l debug

That command will generate a 10 minute log collection. Hoping to try this on the next machine running hot for long duration (seen some 54hr+ situations recently).

[tk]

Yesterday I received this from support:

I can confirm that this is related to an ongoing issue and we are expecting to release a fix by the end of the week or beginning of next. The fix will be included in the latest client update, please keep an eye on the release feed.

So the issue is confirmed and let's hope for a quick fix ;)

[nb]

Anyone been able to confirm whether or not the fix has been released and how to update the client?

[32432jklsfd]

@tkDid you get an update on that timeline being it's past the beginning of the week? Support won't provide a timeline to us and it's impacting many in the org.

[dontmakememac]

I received an update from Jamf Support yesterday evening, instructing me to make a few Plan changes && deploy a 'fix' .pkg. The package wasn't actually included in the message, so still waiting on that part. Wondering if this could be the fix aforementioned by @tk 

I'll keep everyone updated on the results. Like others have said, this is occurring on more and more machines it seems like (at least I've been receiving more reports).

[MattT]

Hi folks.  Thank you for the communication here and your patience as the Jamf Support and Engineering teams have been digging into the issue and a resolution.  We have successfully validated a fix with several customers and are expecting to release an agent update with that fix, possibly as soon as tomorrow.  I'll update this thread once released, I'd also recommend keeping an eye on the release notes.

We appreciate the impact this has had on your end-user's Macs and thank you again for helping us isolate and resolve the issue so quickly.

[HeyVyner]

Awesome news! Appreciate it.

[32432jklsfd]

Hi, @MattT , any updates on timeline for a fix?

[MattT]

Hey @32432jklsfd, a new version of Jamf Protect was released late last week that contained a fix for a known issue causing degraded system performance in some select environments.  Please see 5.1.0 (2023-11-02) for more details.  Apologies for not updating this thread as I had commented!

[nb]

Hey guys. We use JamfNow in our organisation and we have the option to enable malware protection on our Blueprints. 

I've taken a screenshot here of the option in question. To confirm, when enabling this option, our macOS profile is updated and I can see the com.jamf.protect.security-extension process in the Activity Monitor.

I was wondering if anyone knew whether or not JamfNow would automatically update itself if this option is enabled? It would save me from manually having to update all our machines.

I was reading the documentation and it doesn't mention whether or not updates to Jamf Protect would be automatically updated on target machines in JamfNow if this option is enabled.

Any help would be greatly appreciated.

Thanks

[MattT]

Hey @nb customers using the Jamf Now feature you've mentioned will indeed see the Jamf Protect agent deployed update automatically once released.  So, in this case, your devices should already be running the latest version containing the fix.