Jamf Configuration Profiles Stuck Pending

[dthompson1]

I am having a handful of systems with Configuration Profiles stuck in pending state. Does anyone know of a fix for this?

These systems are all Automatic Device Enrolled, they are checking regularly to jamf. I have tried having them reboot, sending blank push from management commands. 

These commands have been pending for months.

[spotmac]

@dthompson1  you can troubleshoot MDM from client side with log command. 

log stream --info --predicate 'processImagePath contains "mdmclient" OR processImagePath contains "storedownloadd"'

Immediately after clicking on "Send Blank Push" the client should return a log.

[dthompson1]

@spotmac thanks I will give that a go.

[anuj530]

Did you find a solution for this? I am also running into the same issue with a few of my devices. 

[user-bzSTfhwTIy]

We are experiencing the same issue with around 20 laptops in our environment across 4 different profiles. Anyone found any workarounds? 

[anuj530]

Is it pending only on some devices or all of your devices? 

[dthompson1]

For mine it is only some. About 8 out of 175 systems. All are checking into jamf with no issues. Just config profiles are stuck in pending.

They are all Automatic Device Enrollment but not all ADE systems are having the issue.

[anuj530]

yeah, I had that issue too. its possible one of your other config profiles is causing the issue? In my case, removing all profiles and running the profiles renew command did the trick. 

[wlevan]

Is the profiles renew command sudo jamf policy?

[anuj530]

No. Once you remove all profiles, it will also remove the MDM profile. You will basically need to re-enroll the computer. This article goes into more details about it -- https://docs.jamf.com/jamf-now/documentation/Re-enrolling_a_Computer_Using_Automated_Device_Enrollme... 
Please make sure you read the requirements carefully before removing the profiles. 

[dthompson1]

I think I asked jamf about this before and they said only way to re-enroll a device that was joined with Automatic Device Enrollment was to wipe the system and setup again.

[Brent_Ranke]

I’ve used these with some success on failed enrollments or squirrely machines

sudo profiles renew -type enrollment

sudo jamf reenroll -prompt

sudo jamf enroll -prompt -noPolicy

[tharr00]

Hey Brent , When utilizing the sudo Jamf reenroll -prompt what are the user credentials (JSS user name)?

[wlevan]

My problem ended up being with the keychain.  I deleted too many keychains troubleshooting another issue.  I did sudo jamf removeFramework command and reenrolled.  My Profiles are coming down now. Not ADE on this test system.

[RyanMilliron]

I was able to resolve the issue on one of the effected computers by doing the sudo jamf removeFramework and then using a User-Initiated Enrollment to re-deploy the MDM framework.  It is a less than Ideal solution for a larger group of users, but was helpful in getting the device in front of me resolution.

[wlevan]

I am having this problem too.

[whiteb]

This might sound dumb, but I had the same issue and what it ended up being with me was that even though the computers that had Config Profiles stuck on 'Pending' were online and actively checking-in, they didn't have people logging into them (I work in K-12 so we have the majority of staff gone for the summer). Every computer that showed 'Pending' that I remoted into, then logged into, the commands all installed. My thinking was that it shows it checked-in recently, it should get the config profiles... nope. A user has to be logged into the computer to get them.

On your computers that have stuck pending profiles, look up one (or a couple) and go to History > Application Usage Logs. Is there anything there? If not, no one is using the computer, so that's actually expected behavior.

If Application Usage Logs are populated after the config profile tried to get pushed and it's still pending... that does sound like an issue.

[BCPeteo]

Having the same issue on about 40 systems. All are checking in fine and most have users logged in. I have tried Jamf recon, launchctl kickstart -k system/com.apple.softwareupdated, renew MDM profile. No of these worked. Rather not have to re-enroll these systems, also sudo profiles renew -type enrollment forces user interaction which is also not ideal. Anyone find why this happens? Seems to be ongoing and happening randomly.

[whiteb]

There is a different way to re-enroll using the API that requires 0 interaction.

https://www.modtitan.com/2022/02/jamf-binary-self-heal-with-jamf-api.html

I've used that to fix a few computers that were not checking-in. The only requirement is the devices still need to be capable of getting MDM commands, which do get send the same way config profiles do I believe, but still worth trying. Definitely fixed a few for me. 

[BCPeteo]

Thanks. Yeah the binary on these systems is fine, they are checking in and doing inventory updates. They are not getting MDM commands that is the issue.

[krbbass]

Seeing the same thing here, over 75 devices.  Checking in, user logged in, MDM Profile Healthy and approved.  Varying OS's. 

Seems to just be more recent profiles that are stuck in pending, but these same profiles have gone out to hundreds of other devices just fine.

Surely someone has a better solution than re-enroll?

[BCPeteo]

I'm working with Jamf on this, will post if we find a solution 

[dthompson1]

The only resolution we found was wiping the system(s) and setting it backup. Even trying to re-enroll the system in jamf did not work.

I also worked with Jamf on our issues.

Dan Thompson
Systems Engineer
HealthStream, Inc.
500 11th Ave. North - Suite 1000
Nashville, TN 37203
Daniel.Thompson@healthstream.com

[pgy_jamf_help]

We have this issue with about 200\1000 devices. Policy and checking are working but configuration profiles are showing as Pending.

[user-DjYuMaLfIG]

In my case, the only thing that works is to restart the computer.

[BCPeteo]

Restarting does not seem to work on a lot of these. Some of them we need re-enroll with the sudo profiles renew -type enrollment which is a pain as the user needs to click on the enrollment notification and approve  

[connor_johnson]

Any update on this issue? We're experiencing this exact same problem and I need to make sure if it's a problem with Jamf or if it could be an issue with ABM that we could fix ourselves.

[Zandreae]

We're seeing the same issue as well. We primary use UIE and see no issues with those computers. We're currently experiencing issues with our PreStage computers not receiving commands.  

[joshuakessel]

Has anyone figured out a way to create a smart group to track these down? I usually run across them when I find another issue, like Crowdstrike is not working (missing profile), or App Installers are not updating installed apps. It would be helpful to have a smart group of all computers with pending management commands. 

[jonw]

I'd like a better way too!  But for now a search/smartgroup like so should provide a list of stuck pending:
• Profile Name does not have 'whatever.is.your.latest.deployed.profile' AND
• Last Inventory Update after 'whatever.is.your.latest.deployed.profile' deployed date

[jonw]

I've been running into this a LOT lately, and not much luck getting to the bottom of it.  However in many cases I'm able to get things going again (at the very least temporarily) without rebooting by kicking the mdmclient.daemon.  It's the next best option I've come up with short of the hassle of 'profiles renew -type enrollment'

launchctl kickstart -k system/com.apple.mdmclient.daemon

[BCPeteo]

So after working with Jamf for months on this issue, they said that it is a know issue with apple and told me to contact enterprise support. Specifically they said the apple issue numbers are PI104712 / PI108400.

Contacted Apple enterprise support and got a good engineer who acknowledged there is an issue and apple is working on a fix for new and older OS's. Basicly he said that the device needs to be assigned to a prestage at all times. Here's how they explained what is happening :

The computer tries to talk to Jamf MDM, sometimes it can not communicate with the MDM (they said that this is rare, and kind of a "perfect storm" must happen). At that point the computer talks to Apple School Manager and asks hey what mdm am I assigned to. ASM will respond with the MDM. But if the device is not assigned to a PreStage in Jamf then it responds with NO MDM. Computer will then tries to contact its originally assigned MDM again, if it gets no response it will try ASM again. At some point it stops trying and if it does not get in touch with the MDM, the trust between MDM and the computer is lost. Now the computer will not be able to ever talk to the MDM (through the MDM protocol, it is still able to to communicate using the Jamf binary).

To fix the issue you either need to wipe the device or try sudo profiles renew -type enrollment (the user will need to accept the new MDM profile and make sure the device is assigned to a prestage)

The profiles renew command has worked for me, but user interaction is needed which is annoying.

I also found a script on jamf nation that can detect the issue, I use it as an Extension attribute: https://community.jamf.com/t5/jamf-pro/configuration-profiles-not-pushing-to-macos-devices/m-p/26346...

[BCPeteo]

Contacted Jamf to see if they had any update on this Apple issue. They said that it still not fixed but Apple said they are getting close to resolving the issue. So hopefully we will hear something soon 

[azimmer1984]

I've encountered what might be a variant of this: Config profiles not updating. We had labs with a prior version of the config profiles last year, that installed successfully. This year, we had a machine behaving badly that clearly had the last year's version of the profile installed, even though after every tweak I've made, the machines say "completed" as if they'd gotten the new one.

Has anyone encountered this outside of prestage or initial setup?

[JeffBugbee]

I've noticed some Macs that still check in and submit inventories but have stuck MDM commands.

Here's a Jamf EA that can assist:

https://github.com/jamf/Jamf-Nation-Extension-Attributes/blob/master/Verify%20MDM%20Enrollment.xml

#!/bin/sh
mdmEnrollmentProfileID="00000000-0000-0000-A000-4A414D460003"
enrolled=`/usr/bin/profiles -C | /usr/bin/grep "$mdmEnrollmentProfileID"`

if [ "$enrolled" != "" ]; then
	echo "<result>Enrolled</result>"
else
	echo "<result>Not Enrolled</result>"
fi

Create a Smart Group looking for devices that show "Not Enrolled." When you find some, they should show MDM commands stuck in a pending state. Generally, re-enrolling this devices will fix the issue. But at least this helps find these broken devices.

[howie_isaacks]

I have been seeing the same thing a lot. I’m going to try using this EA to better detect them. Thanks for posting this.

[BCPeteo]

Wont the computers already have the JSS profile installed which this script is checking for?

[JeffBugbee]

All the computers this finds in my environment do have the JSS profile, but somehow are showing as "not enrolled." I have not found the root cause but this does find some (though not all) Macs with stuck commands.

[jonw]

Not necessarily. The above may alert if the root issue is as dramatic as a botched enrollment (if the Jamf binary is still communicating), but won't detect stuck profiles caused by other issues. The option I prefer, posted further up instead checks for mdm communication errors, thus casting a wider net. Here's my variation:

#!/bin/bash

### mdmclient communication status

### check for failed mdm identity - if failed, may need re-enrollment.
### 2023.06.29 -JW


result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1d | grep "Unable to create MDM identity")

if [[ $result == '' ]]
then
echo "<result>MDM is communicating</result>"
else
echo "<result>Unable to create MDM identity or no communication</result>"
fi

In most of our cases however, stuck pending profiles are not related to this... so I mostly use it as an alert for bigger troubles.  

My kludgy method to detect stuck profiles is usually to create a smart group or saved search to check for workstations that have been online for x amount of time, but don't have some recently deployed profile.  Seems like Jamf could provide a better way...

Fixing an issue is another matter, sometimes a kickstart of com.apple.mdmclient.daemon or reboot is all that's needed.  Other times it's another profile 'clogging the pipes'.  Clearing all pending/failed often gets things moving again.

[howie_isaacks]

You're thinking the same way I am on this. A couple of days ago, I started testing using log show to see all of the mdmclient activity but I couldn't find anything in the log that would indicate that the computer is having problems with MDM. I took my test Mac offline and ran the command. It generated a lot of output. This made me try to look for incoming MDM commands or the lack of them. I just couldn't find an example of an incoming MDM command to grep. So from your EA above, not being able to create an MDM identity indicates that the Mac has profiles stuck at pending?

Jamf needs to make detecting these problems and fixing them easier.