Macs losing their MDM capability

[mconners]

Hello Everyone,

I have seen multiple discussions on this topic, but I was unable to glean anything that would solve our situation.

We are using DEP and our Macs are coming into the JSS just fine. We can even use the erase install command to remotely wipe and get our Macs back to an out of box state.

Somewhere along the way though, we are seeing Macs losing their MDM capability. We have tried just about everything. Granted, if I do something manual in front of the Mac stuff to restore MDM, we might be able to get things going. Something like removing the framework or taking the Mac out of the JSS. I am trying to find a way to do this remotely or the next time one of these systems checks in.

MOST of the profiles seem to be loading and working. We have tried to remove the keychain pieces and running quick add again. Still no go. We have tried running sudo jamf manage, this didn't fix it. We tried to renew our MDM, still no good.

I do have a support ticket open and they have escalated the cause. As this is the first day back for faculty and I found 124 out of 800+ with this problem, I am little freaked out as this really just blew up.

I am wondering if anyone is seeing this? I did a smart group on Macs where MDM is equal to No and found the 124 systems.

Thanks in advance, kind of frantic here.

[BOBW]

@mconners I am seeing this one too, imaged a lab of devices with DEP only to find 2 days later that MDM capability is no. Same issues trying to fix this as well, We are at 195 of 1300 showing this issue.
I have seen this in the logs:
018-08-26 22:44:54,417 [WARN ] [ina-exec-15] [ComputerHelper ] - Computer: Computer [ID=1888, Name=NX-323702] may require re-enrollment. It has been 66 hours since enrolled without a valid APN Token
2018-08-26 22:44:54,417 [WARN ] [ina-exec-15] [ComputerHelper ] - Computer: Computer [ID=1888, Name=NX-323702] may require re-enrollment. It has been 66 hours since enrolled without a valid push magic
But when I re-enroll it makes no difference.

[mark_mahabir]

Does anything mentioned in this thread help at all?

When we faced this recently, it was an expired Tomcat cert.

[sam_g]

That sounds really bizarre. What actually happens when you run sudo jamf manage or sudo jamf mdm?

[mconners]

Hello @float0n it is very bizarre. We have run all of those and more. We get an error when doing the sudo jamf manage:

To have full MDM Management functionality, the MDM Profile must be manually approved in System Preferences > Profiles.
Error installing the computer level mdm profile: profiles install for file:'/Library/Application Support/JAMF/tmp/mdm.mobileconfig' and user:'root' returned 102 (New profile does not meet criteria to replace existing profile.)
Downloading required CA Certificate(s)...
Retrying the user level mdm profile install.
Error installing the computer level mdm profile: profiles install for file:'/Library/Application Support/JAMF/tmp/mdm.mobileconfig' and user:'root' returned 102 (New profile does not meet criteria to replace existing profile.)
Problem installing MDM profile.
*

However, the MDM profile is set to verified so it appears valid.

When we do the sudo jamf mdm we get the same error.

I am really stumped on this one.

[kendalljjohnson]

I had the same thing and I was told by support the fix is coming in a coming release, hopefully 10.7. Here's the summary my case closed with:

Issue Summary
We are seeing that our MacOS Devices are unmanaging from our MDM Server.

Root Cause
Pending MDM commands remain uncompleted when user logs out, preventing further successful MDM communication

Solution Summary
Product Issue - PI-005133. Clearing pending commands and re-enrolling can be used as a fix until this product issue is addressed in future versions.

That particular PI doesn't really address the experience but they seemed to be aware of it when they looked at my logs and remoted in to affected computers. Temporary fix is to re-enroll, which with non-removable MDM in my PreStage meant erase and install.

[mconners]

Thanks @kendalljjohnson this is helpful. I understand the issue with you experienced with the PI mentioned, but I am curious if it is really related to MDM being flagged as No? If we have to touch 120+ computers, this could make for a long day. I might be coming in Saturday.

[kendalljjohnson]

For my situation, the devices became unmanaged and the MDM Capability flipped to No. It was as if the computer thought everything was fine but Jamf Pro saw it as unmanaged. I had to fully re-image because my PreStage Enrollment doesn't allow the MDM Profile to be removed. So even through System Preferences > Profiles had all my normal profiles listed, including the main MDM profile, Jamf Pro saw it as unmanaged. If your PreStage allows removal of the MDM Profile you should be able remove and then re-enroll.

When I attempted to re-enroll it would get the following error:

The JSS is available.
Enforcing login/logout hooks...
To have full MDM Management functionality, the MDM Profile must be manually approved in System Preferences > Profiles.
Error installing the computer level mdm profile: profiles install for file:'/Library/Application Support/JAMF/tmp/mdm.mobileconfig' and user:'root' returned 102 (New profile does not meet criteria to replace existing profile.)

[mconners]

Thanks @kendalljjohnson that's our problem too. We have a pre-stage enrollment that doesn't allow for removal of the MDM profile. I noticed on a couple of Macs that imaging them over everything was fine.

My longer term worry and fear is this may start to happen without warning after the semester begins and I don't want to be chasing down Macs just to keep these from flipping to NO.

[igdjamen]

@mconners Go to Jamf pro and check on the computer inventory on "General". See if the MDM capable user is the same as the one you are actually log into the machine. If it is not then, that is why you will see the message after entering the "sudo Jamf manage".

[scottb]

Similarly, we have been enrolling macs into a 10.6.2 JSS and when Self Service is launched, the "Approve MDM" message pops up even though it's been done, verified, etc.
Macs running 10.13.6.

Won't make clients happy that's for sure...

[igdjamen]

@scottb Check if your Laptop MDM capable user is the same as the one you are actually log into the machine.

[mconners]

Hello @igdjamen and @scottb I was pretty excited, but it turns out it this isn't the case.

The local administrator name is the same user same user listed as managed by and even though I log in using the admin name, I don't have the ability to accept or verify the MDM profile. The MDM profile is still listed as verified on the client and the JSS sees the MDM capable as no.

[scottb]

@igdjamen It's the same user. Profile approved, recon run, and still a delay.

[mconners]

Correct @igdjamen, Profile is approved. We are using our local administrator account which happens to be the managed by account. We have recon locally and using jamf remote and still, the MDM capable category is listed as no.

[igdjamen]

@mconners @scottb If the MDM Capability & the User Approved MDM both said no, while the MDM Capable Users is the same then it is very weird. Try this (it seems the enrollment didn't properly work) :
- Go to JSS and Remove MDM Profile from the management tab on the laptop
- Ran user level mdm command "sudo jamf mdm -userLevelMdm"
- Go to "Profiles" on "System Preferences" and approve the "MDM profile"
- Double click on "verified or unverified" to view the certificate
- Drag the "Bellese JSS Built-in Certificate Authority" to your desktop, in order to add it to your keychain Access under "system". Make sure to change the trust configuration to "Always trust". - Run command "sudo Jamf manage". It should work.

After your laptop provide an inventory update to JSS, you should see MDM Capability as "yes", then after few hours the user approved MDM should be "yes" as well. Let me know if that is the case or if I lost you :) Sorry I just work on this issue this last night :)

[igdjamen]

@mconners @scottb If the MDM Capability & the User Approved MDM both said no, while the MDM Capable Users is the same then it is very weird. Try this (it seems the enrollment didn't properly work) :
- Go to JSS and Remove MDM Profile from the management tab on the laptop
- Ran user level mdm command "sudo jamf mdm -userLevelMdm"
- Go to "Profiles" on "System Preferences" and approve the "MDM profile"
- Double click on "verified or unverified" to view the certificate
- Drag the "Bellese JSS Built-in Certificate Authority" to your desktop, in order to add it to your keychain Access under "system". Make sure to change the trust configuration to "Always trust". - Run command "sudo Jamf manage". It should work.

After your laptop provide an inventory update to JSS, you should see MDM Capability as "yes", then after few hours the user approved MDM should be "yes" as well. Let me know if that is the case or if I lost you :) Sorry I just work on this issue this last night :)

[mconners]

Hello @igdjamen part of this whole crazy thing is, without MDM being ON or capable, there are no management commands listed under the management tab. This was the first clue to me something was amiss. So we are stuck there.

Because the MDM profile is listed as mandatory and we are unable to remove it, we are sunk there too.

When I log in with the local admin account, the MDM profile is already listed as verified and there is nothing to approve.

Finally the JSS built in certificate authority is already set to always trust.

The good news is, I have an Jamf engineer WebEx session today at 2. The worst case scenario is we have to image these over again, not good, but we can get these back.

Without knowing the root cause though, I certainly don't want to be dealing with this during the semester.

Looking at our computers reporting no MDM capabilities, they are almost all laptops. This is telling me the computers aren't checking in. Could the client be "shutting off" the MDM piece if it hasn't checked in for a while?

[sdagley]

@mconners Maybe a shot in the dark, but is the time set correctly on these Macs?

[kendalljjohnson]

For context of my situation, I had a dozen iMacs I setup with DEP and installed various software installs back in May and they were in storage all Summer waiting for Fall deployment. Six out of the twelve had this issue occur, no power or communication with Jamf.

[mconners]

Hey @sdagley thank you, I was so excited to think it might be that simple, but no go. When remotely connected to the computer, I saw it was off by a minute. I logged in, it quickly changed to the correct time, but I don't think a minute off would make a difference.

In the JSS, I have the maximum clock skew set to No Maximum.

[mconners]

@kendalljjohnson I believe we had the same situation. I am curious after I talk with Jamf in 10 minutes whether or not this could occur. It really seems like some of us are seeing this. I don't recall this happening last year. Maybe something I introduced with both DEP this summer and the upgrade to JSS 10.6?

[igdjamen]

@mconners Crazy! Please, update me on the result of your webEx session.

[scottb]

@mconners - I as well want to know. Seems they were able to replicate some of my issue, so hoping more eyes and all...

[kendalljjohnson]

To add to the complexity, I'm still on Jamf 10.3.0. All affected computers were running macOS 10.13.3 when enrolled with DEP.

[mconners]

Hello @igdjamen, @scottb and @kendalljjohnson I just got off my phone call with Jamf. We have tried a number of things and he is a little stumped too. While we don't have a root cause, I did suggest I can wipe and start fresh which will resolve this for the start of our semester on Tuesday morning. I have also informed our lab coordinators that the fallout really is user specific settings are not being pushed down to the Macs so dock settings and such that we use for config profiles won't be showing up. Just run with it and have users get into the Applications folders to find their apps.

He fully understands the worry on our parts knowing this could come back without notice.

So, we have labeled a single Mac in one lab as do not touch. This will be our test Mac until we can figure out what triggered what.

For now, I am just going to get imaging and start fresh.

[scottb]

Thanks, @mconners - re-imaging, etc. is not an option at all for us.
I really hope that there are some dedicated brain cells allocated at Jamf for this.

[kendalljjohnson]

@scottb I can't remember the exact steps, but support did mention steps about removing the MDM profile when it is not allowed in your PreStage. Found this blog that talks about disabling SIP and then being able to remove the profiles, might be your only other option.

[m_donovan]

Just to throw my hat into the ring. We are moving to a provisioning workflow with DEP. I have a campus that provisioned a lab of computers, reassigned them to the static group. The appropriate profiles were applied life is beautiful. Then the tech changed the building in a mass action from Jamf. All of the computers disappeared from the static group. After investigating the computers were unmanaged, MDM capability no and User approved MDM No. The profiles were still on the computer even though they were no longer in the static group that the profile was scoped to. The MDM was verified and approved on the machine.

[mconners]

Hello All, it appears we have run into a critical product issue, PI-004892 - Enabling User Level MDM on 10.13.2+ Removes 'User Approved MDM Enrollment' Approval. Talking with Jamf this morning, they have identified as being the case.

Well it certainly makes me feel better to know this wasn't anything I could have controlled. It is still upsetting knowing I don't have a work around.

From Jamf this morning, The language is slightly different from the exact behavior we're seeing, but from the logs it's definitely the same cause. Right now the issue is marked as critical, and there's no workaround aside from the two we did talk about (manually touching each machine with an erase or temporarily turning off SIP).

Unfortunately that leaves us stuck where we are until this product issue is resolved.

I think as I run into these computers, I will have to touch them one way or another. I also expect after the fix is in place, I will still have to touch them. Going to be a rocky start to the semester.

[RedWings]

We are now running into this issue.

[miwe01]

Same here. It's alarming since it's more than a year ago that it was discovered according to this thread. We are soon handing out about 350 new MacBooks...

[coachdnadel]

We are also starting to see this issue. I have had to wipe 3 computers in the last week.

[mconners]

After seeing your updates @miwe01 and @coachdnadel I looked and guess what, I am too seeing this on a couple of Macs. I don't get how this is happening. These are showing up the majority are faculty computers so I can't do much in the way of wiping these to reset back to MDM yes. So we will have to wait this out until they have issues or we find a fix.

[akamenev47]

same here, starting seen this issue

[ChicagoGuy1984]

This has been happening for us as well, multiple older computers that were not "imaged" via the New way or via the MDM enroll via the Prestage Enrollment will have this. when i looked at this month ago , the only SOLID way to shake this is to "reimage" the mac completely and re-install the os with all Prestage enrollment stuff already setup, they will install the JAMF binary via the "supported" way and MDM will not be an issue going forward.

For me this is a huge headache, as we can't "Yank out" older 2-4 year old machines just to correct this issue. , on a filp side they are still checking in and are not going 'DARK" But this "RED" MDM is really Vexing.

[mani2care]

The perfect solution is without doing the action

sudo jamf mdm -userLevelMdm
sudo Jamf manage

MDM Capability: Yes will be changed

[jchen1225]

Hi, when I run 

sudo jamf mdm -userLevelMdm

I get: The mdm verb is not available on this version of macOS.

[eos_bebu]

Did you check your ports for Apple Push Notifications (APNs) ? 
If your Apple devices aren't getting Apple push notifications - Apple Support

[whiteb]

We have a very small handful of machines like this. They still have all of our profiles installed and are checking in, but no management commands available in Jamf + MDM Capability shows 'No'.

sudo profiles renew -type enrollment

Running the above, even with an existing MDM Profile installed, fixed the issue.

I tried a sudo jamf enroll -prompt to re-enroll first, which completed without issue, but still MDM Capability 'No' and no management commands for the computer in Jamf.

Only after running the profiles renew command and accepting the little message that pops-up did the computer get fixed. This computer was an M1 iMac on 13.2.1.

Appears some computers lose their MDM Capability for no apparent reason.