+3I am having a handful of systems with Configuration Profiles stuck in pending state. Does anyone know of a fix for this?
These systems are all Automatic Device Enrolled, they are checking regularly to jamf. I have tried having them reboot, sending blank push from management commands.
These commands have been pending for months.
+2Has anyone figured out a way to create a smart group to track these down? I usually run across them when I find another issue, like Crowdstrike is not working (missing profile), or App Installers are not updating installed apps. It would be helpful to have a smart group of all computers with pending management commands.
+10I've been running into this a LOT lately, and not much luck getting to the bottom of it. However in many cases I'm able to get things going again (at the very least temporarily) without rebooting by kicking the mdmclient.daemon. It's the next best option I've come up with short of the hassle of 'profiles renew -type enrollment'
launchctl kickstart -k system/com.apple.mdmclient.daemon
+10Has anyone figured out a way to create a smart group to track these down? I usually run across them when I find another issue, like Crowdstrike is not working (missing profile), or App Installers are not updating installed apps. It would be helpful to have a smart group of all computers with pending management commands.
I'd like a better way too! But for now a search/smartgroup like so should provide a list of stuck pending:
• Profile Name does not have 'whatever.is.your.latest.deployed.profile' AND
• Last Inventory Update after 'whatever.is.your.latest.deployed.profile' deployed date
+11I've been running into this a LOT lately, and not much luck getting to the bottom of it. However in many cases I'm able to get things going again (at the very least temporarily) without rebooting by kicking the mdmclient.daemon. It's the next best option I've come up with short of the hassle of 'profiles renew -type enrollment'
launchctl kickstart -k system/com.apple.mdmclient.daemon
So after working with Jamf for months on this issue, they said that it is a know issue with apple and told me to contact enterprise support. Specifically they said the apple issue numbers are PI104712 / PI108400.
Contacted Apple enterprise support and got a good engineer who acknowledged there is an issue and apple is working on a fix for new and older OS's. Basicly he said that the device needs to be assigned to a prestage at all times. Here's how they explained what is happening :
The computer tries to talk to Jamf MDM, sometimes it can not communicate with the MDM (they said that this is rare, and kind of a "perfect storm" must happen). At that point the computer talks to Apple School Manager and asks hey what mdm am I assigned to. ASM will respond with the MDM. But if the device is not assigned to a PreStage in Jamf then it responds with NO MDM. Computer will then tries to contact its originally assigned MDM again, if it gets no response it will try ASM again. At some point it stops trying and if it does not get in touch with the MDM, the trust between MDM and the computer is lost. Now the computer will not be able to ever talk to the MDM (through the MDM protocol, it is still able to to communicate using the Jamf binary).
To fix the issue you either need to wipe the device or try sudo profiles renew -type enrollment (the user will need to accept the new MDM profile and make sure the device is assigned to a prestage)
The profiles renew command has worked for me, but user interaction is needed which is annoying.
I also found a script on jamf nation that can detect the issue, I use it as an Extension attribute: https://community.jamf.com/t5/jamf-pro/configuration-profiles-not-pushing-to-macos-devices/m-p/263468/highlight/true#M242423
+3I've encountered what might be a variant of this: Config profiles not updating. We had labs with a prior version of the config profiles last year, that installed successfully. This year, we had a machine behaving badly that clearly had the last year's version of the profile installed, even though after every tweak I've made, the machines say "completed" as if they'd gotten the new one.
Has anyone encountered this outside of prestage or initial setup?
+11So after working with Jamf for months on this issue, they said that it is a know issue with apple and told me to contact enterprise support. Specifically they said the apple issue numbers are PI104712 / PI108400.
Contacted Apple enterprise support and got a good engineer who acknowledged there is an issue and apple is working on a fix for new and older OS's. Basicly he said that the device needs to be assigned to a prestage at all times. Here's how they explained what is happening :
The computer tries to talk to Jamf MDM, sometimes it can not communicate with the MDM (they said that this is rare, and kind of a "perfect storm" must happen). At that point the computer talks to Apple School Manager and asks hey what mdm am I assigned to. ASM will respond with the MDM. But if the device is not assigned to a PreStage in Jamf then it responds with NO MDM. Computer will then tries to contact its originally assigned MDM again, if it gets no response it will try ASM again. At some point it stops trying and if it does not get in touch with the MDM, the trust between MDM and the computer is lost. Now the computer will not be able to ever talk to the MDM (through the MDM protocol, it is still able to to communicate using the Jamf binary).
To fix the issue you either need to wipe the device or try sudo profiles renew -type enrollment (the user will need to accept the new MDM profile and make sure the device is assigned to a prestage)
The profiles renew command has worked for me, but user interaction is needed which is annoying.
I also found a script on jamf nation that can detect the issue, I use it as an Extension attribute: https://community.jamf.com/t5/jamf-pro/configuration-profiles-not-pushing-to-macos-devices/m-p/263468/highlight/true#M242423
Contacted Jamf to see if they had any update on this Apple issue. They said that it still not fixed but Apple said they are getting close to resolving the issue. So hopefully we will hear something soon
I've noticed some Macs that still check in and submit inventories but have stuck MDM commands.
Here's a Jamf EA that can assist:
https://github.com/jamf/Jamf-Nation-Extension-Attributes/blob/master/Verify%20MDM%20Enrollment.xml
#!/bin/sh
mdmEnrollmentProfileID="00000000-0000-0000-A000-4A414D460003"
enrolled=`/usr/bin/profiles -C | /usr/bin/grep "$mdmEnrollmentProfileID"`
if [ "$enrolled" != "" ]; then
echo "<result>Enrolled</result>"
else
echo "<result>Not Enrolled</result>"
fiCreate a Smart Group looking for devices that show "Not Enrolled." When you find some, they should show MDM commands stuck in a pending state. Generally, re-enrolling this devices will fix the issue. But at least this helps find these broken devices.
+23I've noticed some Macs that still check in and submit inventories but have stuck MDM commands.
Here's a Jamf EA that can assist:
https://github.com/jamf/Jamf-Nation-Extension-Attributes/blob/master/Verify%20MDM%20Enrollment.xml
#!/bin/sh
mdmEnrollmentProfileID="00000000-0000-0000-A000-4A414D460003"
enrolled=`/usr/bin/profiles -C | /usr/bin/grep "$mdmEnrollmentProfileID"`
if [ "$enrolled" != "" ]; then
echo "<result>Enrolled</result>"
else
echo "<result>Not Enrolled</result>"
fiCreate a Smart Group looking for devices that show "Not Enrolled." When you find some, they should show MDM commands stuck in a pending state. Generally, re-enrolling this devices will fix the issue. But at least this helps find these broken devices.
I have been seeing the same thing a lot. I’m going to try using this EA to better detect them. Thanks for posting this.
+11I've noticed some Macs that still check in and submit inventories but have stuck MDM commands.
Here's a Jamf EA that can assist:
https://github.com/jamf/Jamf-Nation-Extension-Attributes/blob/master/Verify%20MDM%20Enrollment.xml
#!/bin/sh
mdmEnrollmentProfileID="00000000-0000-0000-A000-4A414D460003"
enrolled=`/usr/bin/profiles -C | /usr/bin/grep "$mdmEnrollmentProfileID"`
if [ "$enrolled" != "" ]; then
echo "<result>Enrolled</result>"
else
echo "<result>Not Enrolled</result>"
fiCreate a Smart Group looking for devices that show "Not Enrolled." When you find some, they should show MDM commands stuck in a pending state. Generally, re-enrolling this devices will fix the issue. But at least this helps find these broken devices.
Wont the computers already have the JSS profile installed which this script is checking for?
+10Not necessarily. The above may alert if the root issue is as dramatic as a botched enrollment (if the Jamf binary is still communicating), but won't detect stuck profiles caused by other issues. The option I prefer, posted further up instead checks for mdm communication errors, thus casting a wider net. Here's my variation:
#!/bin/bash
### mdmclient communication status
### check for failed mdm identity - if failed, may need re-enrollment.
### 2023.06.29 -JW
result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1d | grep "Unable to create MDM identity")
if [[ $result == '' ]]
then
echo "<result>MDM is communicating</result>"
else
echo "<result>Unable to create MDM identity or no communication</result>"
fi
In most of our cases however, stuck pending profiles are not related to this... so I mostly use it as an alert for bigger troubles.
My kludgy method to detect stuck profiles is usually to create a smart group or saved search to check for workstations that have been online for x amount of time, but don't have some recently deployed profile. Seems like Jamf could provide a better way...
Fixing an issue is another matter, sometimes a kickstart of com.apple.mdmclient.daemon or reboot is all that's needed. Other times it's another profile 'clogging the pipes'. Clearing all pending/failed often gets things moving again.
Wont the computers already have the JSS profile installed which this script is checking for?
All the computers this finds in my environment do have the JSS profile, but somehow are showing as "not enrolled." I have not found the root cause but this does find some (though not all) Macs with stuck commands.
+23Not necessarily. The above may alert if the root issue is as dramatic as a botched enrollment (if the Jamf binary is still communicating), but won't detect stuck profiles caused by other issues. The option I prefer, posted further up instead checks for mdm communication errors, thus casting a wider net. Here's my variation:
#!/bin/bash
### mdmclient communication status
### check for failed mdm identity - if failed, may need re-enrollment.
### 2023.06.29 -JW
result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1d | grep "Unable to create MDM identity")
if [[ $result == '' ]]
then
echo "<result>MDM is communicating</result>"
else
echo "<result>Unable to create MDM identity or no communication</result>"
fi
In most of our cases however, stuck pending profiles are not related to this... so I mostly use it as an alert for bigger troubles.
My kludgy method to detect stuck profiles is usually to create a smart group or saved search to check for workstations that have been online for x amount of time, but don't have some recently deployed profile. Seems like Jamf could provide a better way...
Fixing an issue is another matter, sometimes a kickstart of com.apple.mdmclient.daemon or reboot is all that's needed. Other times it's another profile 'clogging the pipes'. Clearing all pending/failed often gets things moving again.
You're thinking the same way I am on this. A couple of days ago, I started testing using log show to see all of the mdmclient activity but I couldn't find anything in the log that would indicate that the computer is having problems with MDM. I took my test Mac offline and ran the command. It generated a lot of output. This made me try to look for incoming MDM commands or the lack of them. I just couldn't find an example of an incoming MDM command to grep. So from your EA above, not being able to create an MDM identity indicates that the Mac has profiles stuck at pending?
Jamf needs to make detecting these problems and fixing them easier.
+10Stuck pending profiles are a symptom of MDM identity or mdm communication errors, not the other way around. The EA simply alerts you of such an issue before you stumble across any related stuck pending profile(s) they create. It at least eliminates this as a variable in troubleshooting (even proactively if you implement a smart group with alert). But it's not going to alert you of stuck profiles in general if that's what you're after.
+10Frankly I don't see why Jamf couldn't add a smart group criteria value for 'profile pending'. Then we could at least get crafty with detection. ¯\\_(ツ)_/¯
+23Frankly I don't see why Jamf couldn't add a smart group criteria value for 'profile pending'. Then we could at least get crafty with detection. ¯\\_(ツ)_/¯
It's time for a feature request!
+23I created this feature request to give us the smart group criteria to track down Macs with MDM issues.
+23After I opened a case with Jamf about our MDM communication issues, I got these three extension attributes.
#!/bin/bash
result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1h | grep "Unable to create MDM identity")
if [[ $result == '' ]]
then
echo "<result>MDM is communicating</result>"
else
echo "<result>MDM is broken</result>"
fi
#!/bin/bash
APNS_certificate=`/usr/sbin/system_profiler SPConfigurationProfileDataType | awk '/Topic/{ print $NF }' | sed 's/[";]//g'`
if [[ "$APNS_certificate" = "" ]]; then
echo "<result>"NA"</result>"
else
echo "<result>"$APNS_certificate"</result>"
fi
#!/bin/bash
theIDs=$(security find-identity -v | awk '{print $3}' | tr -d '"' | grep -E '^[A-Za-z0-9]{8}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{12}$')
echo $theIDs
if [ -z "$theIDs" ]; then
echo "<result>ERROR - There appears to be zero keychain identities matching a UUID on this system.</result>"
exit 1
else
echo "At least one keychain identity found on this system, proceeding..."
fi
for i in $theIDs; do
info=$(security find-certificate -c "$i" | grep issu | awk '{print $3, $4, $5, $6, $7}' | tr -d '"')
echo $info
if [[ $info == *"BUILT-IN CERTIFICATE AUTHORITY"* ]]; then
echo "found you!"
expiry=$(security find-certificate -c "$i" -p | openssl x509 -noout -enddate | cut -f2 -d"=")
echo "<result>$theIDs + $expiry</result>"
fi
done
exit 1
For the first one you may want to change the timeframe. I changed it to "--last 1h". Originally, it was "--last 1d". That may take a while to generate results. I figure 1 hour should be enough time to analyze in the log. These have helped a lot. They basically solve the problem of tracking which Macs are having MDM issues. I created two smart groups, one called "MDM Communication Bad" and another called "MDM Communication Good". If a Mac fails any of the three tests, it is "bad". To be "good", it must pass all three tests. I am a Jamf Now customer. I ran these through CodeRunner on my personal MacBook Pro. They work the same as with a Mac enrolled in Jamf Pro. I hope these help others with MDM issues.
I’ve used these with some success on failed enrollments or squirrely machines
sudo profiles renew -type enrollment
sudo jamf reenroll -prompt
sudo jamf enroll -prompt -noPolicy
+8I’ve used these with some success on failed enrollments or squirrely machines
sudo profiles renew -type enrollment
sudo jamf reenroll -prompt
sudo jamf enroll -prompt -noPolicy
Hey Brent , When utilizing the sudo Jamf reenroll -prompt what are the user credentials (JSS user name)?
I created this feature request to give us the smart group criteria to track down Macs with MDM issues.
still under Reviewing since March 2024
re-enroll machine in jamf sudo jamf removeframework , noway around
I have 15 machines with couple of configuration profiles stuck on pending very annoying.
+3After I opened a case with Jamf about our MDM communication issues, I got these three extension attributes.
#!/bin/bash
result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1h | grep "Unable to create MDM identity")
if [[ $result == '' ]]
then
echo "<result>MDM is communicating</result>"
else
echo "<result>MDM is broken</result>"
fi
#!/bin/bash
APNS_certificate=`/usr/sbin/system_profiler SPConfigurationProfileDataType | awk '/Topic/{ print $NF }' | sed 's/[";]//g'`
if [[ "$APNS_certificate" = "" ]]; then
echo "<result>"NA"</result>"
else
echo "<result>"$APNS_certificate"</result>"
fi
#!/bin/bash
theIDs=$(security find-identity -v | awk '{print $3}' | tr -d '"' | grep -E '^[A-Za-z0-9]{8}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{12}$')
echo $theIDs
if [ -z "$theIDs" ]; then
echo "<result>ERROR - There appears to be zero keychain identities matching a UUID on this system.</result>"
exit 1
else
echo "At least one keychain identity found on this system, proceeding..."
fi
for i in $theIDs; do
info=$(security find-certificate -c "$i" | grep issu | awk '{print $3, $4, $5, $6, $7}' | tr -d '"')
echo $info
if [[ $info == *"BUILT-IN CERTIFICATE AUTHORITY"* ]]; then
echo "found you!"
expiry=$(security find-certificate -c "$i" -p | openssl x509 -noout -enddate | cut -f2 -d"=")
echo "<result>$theIDs + $expiry</result>"
fi
done
exit 1
For the first one you may want to change the timeframe. I changed it to "--last 1h". Originally, it was "--last 1d". That may take a while to generate results. I figure 1 hour should be enough time to analyze in the log. These have helped a lot. They basically solve the problem of tracking which Macs are having MDM issues. I created two smart groups, one called "MDM Communication Bad" and another called "MDM Communication Good". If a Mac fails any of the three tests, it is "bad". To be "good", it must pass all three tests. I am a Jamf Now customer. I ran these through CodeRunner on my personal MacBook Pro. They work the same as with a Mac enrolled in Jamf Pro. I hope these help others with MDM issues.
This looks like something I'd like to add to my Jamf Pro instance so I can be proactive in detecting MDM communication issues with my fleet. I created the EA's and the two smart groups. Is there a particular value that would be used in those smart groups for reporting?
+23I made some changes to the EAs that I posted earlier this year to make using them with smart groups easier. They're posted below:
MDM identity certificate check
#!/bin/bash
theIDs=$(security find-identity -v | awk '{print $3}' | tr -d '"' | grep -E '^[A-Za-z0-9]{8}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{4}-[A-Za-z0-9]{12}$')
echo $theIDs
if [ -z "$theIDs" ]; then
echo "<result>"Failed"</result>"
exit 1
else
echo "At least one keychain identity found on this system, proceeding..."
fi
for i in $theIDs; do
info=$(security find-certificate -c "$i" | grep issu | awk '{print $3, $4, $5, $6, $7}' | tr -d '"')
echo $info
if [[ $info == *"BUILT-IN CERTIFICATE AUTHORITY"* ]]; then
echo "found you!"
expiry=$(security find-certificate -c "$i" -p | openssl x509 -noout -enddate | cut -f2 -d"=")
echo "<result>"Success"</result>"
fi
done
exit 1
MDM APNS certificate check
#!/bin/bash
APNS_certificate=`/usr/sbin/system_profiler SPConfigurationProfileDataType | awk '/Topic/{ print $NF }' | sed 's/[";]//g'`
if [[ "$APNS_certificate" = "" ]]; then
echo "<result>"Bad"</result>"
else
echo "<result>"Good"</result>"
fi
MDM client communication
#!/bin/bash
result=$(log show --style compact --predicate '(process CONTAINS "mdmclient")' --last 1h | grep "Unable to create MDM identity")
if [[ $result == '' ]]
then
echo "<result>"Good"</result>"
else
echo "<result>"Bad"</result>"
fi
I used these to create a smart group called "MDM Communication Bad". If any of the criteria using these EAs is not a positive (good) result, then the Mac will be added to this group. I also added the MDM enrollment status found in a Mac's Jamf inventory record. I hope this helps. In most cases, a re-enroll of a Mac in JAMF Pro is what is needed. Sometimes, I have had to also run a "jamf removeFramework" command first. A Mac that is in Apple Business Manager can be re-enrolled by running "sudo profiles renew -type enrollment".
+11If the "MDM is broken" its it enough to run sudo profiles renew -type enrollment to get the mac able to get profiles again?
+23If the "MDM is broken" its it enough to run sudo profiles renew -type enrollment to get the mac able to get profiles again?
That command will work as long as the Mac is in Apple Business Manager. If it's not, this command will enroll the Mac:
sudo jamf enroll -prompt
You will be prompted for a username and password of a Jamf Pro user who has enrollment privileges. Sometimes I have had to run sudo jamf removeFramework to remove the Jamf agent and keychain items. If the Mac enrolled through automated device enrollment (PreStage), and your PreStage is set to not allow MDM profile removal, this won't remove the MDM profile. That's OK. Running the profiles renew -type enrollment command will re-enroll.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.