DEP Macs don't think they're in DEP

AVmcclint
Honored Contributor

We bought a batch of about 30 Macs a couple months ago and made sure they were all enrolled in DEP at the time of purchase (verified through ABM). I also verified that the PreStage Enrollment listed all the Macs we bought. I've deployed about 10 Macs from that big purchase so far and they have all enrolled and pulled down all the MDM profiles as expected... until recently. We got to a point in the stack where we're starting to see a few Macs (3 so far) that have no idea that they are supposed to be managed. We take the Macs out of the box, plug them in for power and ethernet, then we turn them on. We pick the language and country etc, tell them to use ethernet, then they completely skip over this Remote Management screen that they are supposed to see:
bb6716466f6a458bbe958e24ffd73716
It goes straight into privacy and the rest of the non-DEP setup screens. I've had my techs stop the process as soon as they identify that the Remote Management screen doesn't come up. I still have a bunch more Macs that I need to setup, but so far only 3 have been identified as having this problem.

Is there anything I need to do on my end before calling Apple?

1 ACCEPTED SOLUTION

AVmcclint
Honored Contributor

I ended up calling Enterprise support and they confirmed that the DEP assignments looked good on their end. I told them that 20 other Macs using the same network jack, cable, and ethernet dongle all worked fine, but these last 2 refuse to work. They suggested I reinstall the OS. I booted up in Recovery mode and went to Disk Utility and formatted the Macintosh HD volume, reinstalled, and TADA! it worked. it is now sitting on the workbench soaking in MDM goodness. The 2nd one was more stubborn. I was unable to format the volume - it kept hanging - so I ended up blowing out the drive and let it create a new APFS container and all that jazz. It's installing the OS now. I should know in 30 minutes if it worked. EDIT: It worked!

To sum up: 1) zap the PRAM, it could be an issue with getting on the network
2) reinstall the OS, the OS could be bad straight from the factory
3) completely reformat the APFS container, the drive format/partitioning could be bad from the factory

View solution in original post

44 REPLIES 44

jcline
New Contributor III

I've had that happen a couple times. Usually, I got in and refresh the DEP token from ASM. This normally fixes the problem.

sshort
Valued Contributor

Every once in a while I encounter this issue if for some reason the ethernet adapter isn't communicating with the Mac properly at startup. I use the Anker USB-C to ethernet adapter, and I can see all the link lights working, but the Mac still doesn't communicate that it's in DEP.

A quick reboot or a PRAM reset has resolved the issue for me in the past. On one occasion I had to open a Terminal prompt in Setup Assistant and run this:

rm -f /var/db/Configuration Profiles/Settings/.cloudConfigNoActivationRecord

kerouak
Valued Contributor

I do the same as @jcline

works for me too

AVmcclint
Honored Contributor

So far I did zap the PRAM on one of the Macs in question and it worked. Some of these Macs have sat in the box a few months and the batteries may not be as fresh as they were right out of the factory. I'll try zapping the others to see what they do. thanks for the input, everyone.

AVmcclint
Honored Contributor

Zapping the PRAM worked for one computer but not the other two. I figured I'd try refreshing my DEP token to see if that worked. Nope. The computers are still completely oblivious to DEP. There is no/var/db/Configuration Profiles/ directory on this Mac so there's nothing for me to delete yet. I took the Mac off our network and tried it on my home network (just because our network security can be a little too aggressive at times), but even on my home network it still does not recognize that it belongs in our DEP. I thought DEP was supposed to be a magical service that made imaging completely a thing of the past. I could see that happening if it actually worked all the time. :-( <extreme frustration>

adamcodega
Valued Contributor

A shot in the dark, but the file path sshort referenced needs the space escaped, or when you type it in Terminal you have to wrap it in quotation marks.

So you'd type

rm -f /var/db/Configuration Profiles/Settings/.cloudConfigNoActivationRecord

or

rm -f "/var/db/Configuration Profiles/Settings/.cloudConfigNoActivationRecord"

AVmcclint
Honored Contributor

@adamcodega thanks for the suggestion, but I looked inside of /var/db/ and there was no subdirectory in there by any name even close to that. I rarely delete a file/folder in a path without verifying it exists in the first place. I looked. It wasn't there.

easyedc
Valued Contributor II

Not that it solves your problem, but have you booted all the way to the desktop on one of them and tried a

profiles renew -type enrollment

to see if once running they actually check into DEP and get recognized for management? I am seeing this morning and more frequently from DEP workstations, but I chalk it up to our stringent network access control tools. FWIW - sounds like it's time for a ticket with Apple.

AVmcclint
Honored Contributor

correction When I checked the /var/db/ directory, I was looking on the Recovery partition. Stupid me didn't look at /Volumes/Macintosh HD/var/db/ That being said, I looked in the right location a minute ago and there IS a "ConfigurationProfiles" directory (no space). The only thing in there is a .migrated file that's zero bytes. Looked in /var/db/ConfigurationProfiles/Setup/ and there is a .profileSetupDone file. I deleted that to see if that would kick it, but it did not.

@easyedc That brings up a question I've had since jumping into DEP: If - for whatever reason - a mac is booted up all the way to creating a local account etc and not getting enrolled in our MDM, do I have the ability to manually reinstall the OS and try to get DEP to engage it during setup? Or is this very first time "out of the box" experience the one and only chance I have?

easyedc
Valued Contributor II

@AVmcclint If you get to the desktop and DEP hasn't recognized that it should be managed through a pre-stage, then you can run the above command and it will then go re-check DEP servers and call down any management. The only gotchya (and it's a biggie) is that the profile has to then be manually accepted through a pop-up notification. If the computer has made it into the hands of users, they could, theoretically, decline it.

I've been working on a re-enrollment process for Macs that get deleted out of our JAMF server (if they're inactive for 6+ months, we delete them thinking they've been removed from users hands). I've got some logic in place, but the users ability to decline the MDM is a problem. I submitted a ticket to Apple and was told that there's already a feature request

for a silent re-enrollment method

but as of yet, nothing's come of it. My argument to Apple is that if they're actively assigned to a DEP MDM, then re-enrollments should be able to be performed without any user acceptance.

easyedc
Valued Contributor II

... Follow-up - macOS 10.12 used to use

sudo /usr/libexec/mdmclient dep nag

but I believe it's all been supplanted by

/usr/bin/profiles

AVmcclint
Honored Contributor

I ended up calling Enterprise support and they confirmed that the DEP assignments looked good on their end. I told them that 20 other Macs using the same network jack, cable, and ethernet dongle all worked fine, but these last 2 refuse to work. They suggested I reinstall the OS. I booted up in Recovery mode and went to Disk Utility and formatted the Macintosh HD volume, reinstalled, and TADA! it worked. it is now sitting on the workbench soaking in MDM goodness. The 2nd one was more stubborn. I was unable to format the volume - it kept hanging - so I ended up blowing out the drive and let it create a new APFS container and all that jazz. It's installing the OS now. I should know in 30 minutes if it worked. EDIT: It worked!

To sum up: 1) zap the PRAM, it could be an issue with getting on the network
2) reinstall the OS, the OS could be bad straight from the factory
3) completely reformat the APFS container, the drive format/partitioning could be bad from the factory

tnielsen
Valued Contributor

3 is likely what fixed it. You should get into the habit of partitioning to something other than APFS, then back to APFS in disk utility to completely wipe all containers before the OS is installed. I banged my head on the desk for this one as well.

seraphina
Contributor II

I have had this problem as well; Resetting the PRAM worked on a few machines, but for the other I had to reinstall macOS. We still have one final one that never picked up DEP enrollment after a PRAM reset and a reinstall. I never tried to delete the APFS container so I will keep this in mind if I run into the problem again.

AVmcclint
Honored Contributor

I just wanted to add a new experience I had with this. Yesterday we had 4 DEP Macs (with 10.13.6 on them) pulled out of their boxes and powered on... none of them recognized they were in DEP. So I did the steps outlined above and it worked for 3 of the Macs. The 4th one.... well it kinda sorta worked, but not really. After it was presented with the Remote Management screen with the giant gear acknowledging that the computer will be managed by us, we clicked Continue and it went to the login screen (where our standard practice is to let it sit for 30 minutes while it pulls down config profiles and installs all the post-enrollment profiles.) While it was sitting at the login screen, a message popped up telling us that the MDM profile needed to be approved in System preferences before it could be installed (or whatever it was). When I looked at the computer record in JSS, it said "MDM Capability: Yes" , "Enrolled via DEP: No" , "User Approved MDM: No"

It was very strange. Obviously the computer knows about DEP because it got the Remote Management screen during setup, and it DID enroll. I verified that the PreStage Enrollment was setup properly and this computer was in the scope. I decided to delete the computer record from JSS, wipe the hard drive, and reinstall the OS one more time. This time it worked fully and is properly recognized as DEP Enrolled..

I see where Apple and Jamf are going with this whole "zero touch" thing, but to me it seems that DEP is way too fragile to rely on it to work 100% of the time.

mm2270
Legendary Contributor III
I see where Apple and Jamf are going with this whole "zero touch" thing, but to me it seems that DEP is way too fragile to rely on it to work 100% of the time.

Agreed. While you, as an IT admin were able to eventually resolve these issues, imagine if we were all using the Apple and/or IBM approach of just shipping these Macs to end users and expecting it should auto enroll, and in some cases it just doesn't work? That's a mess on our hands we'd need to track down and clean up later. It is indeed too fragile in my opinion.

That being said, in my own testing, I've had basically 100% success in taking a Mac enrolled into DEP and using the startosinstall command to wipe and reinstall the OS cleanly and have it be recognized and enrolled via DEP into our Jamf instance. So it does seem like a wipe and reinstall is a pretty sure fire way to get it to work.
But that of course falls apart in the above "drop ship to customer" scenario, since you will not have an opportunity to do that. And even if you did, who wants to go through that process for every single Mac? Might as well go back to traditional imaging if you have to go thru that rigmarole to get it to work reliably!

Sandy
Valued Contributor II

I have 600 new MacBook Airs to deploy and so far in my early testing I am seeing about every other one out of the box skip over DEP. I am not using an enrollment customization configuration.
I m using an ethernet adapter which sure seems like works just fine because without joining any wireless, the ones that don't work are able to boot right up to internet restore. I do not get the network picker on the successes or the failures, which to me would imply they are recognizing the ethernet connection
Once they skip over DEP, as mentioned by others I have to wipe them to get them back to where they will recognize my prestage. Once wiped and reinstalled, changing NOTHING ELSE, they work fine

Since half of them do work, it does not seem to be our network, the Token in jamf, DEP services, my prestage.....
Due to the number of failures my deployment strategy is pretty much out the window

blairb
New Contributor III

When this has happened to me, it's almost always turned out to be the case that clock was skewed too far to allow communication with Apple's DEP infrastructure. Had to turn off SIP, boot to recovery and set the clock manually.

tcam
Contributor

@Sandy
Did you try completing applesetup, rm /var/db/.AppleSetupDone and doing it again?

or maybe
sudo rm /var/db/.AppleSetupDone
sudo rm -rf /var/db/ConfigurationProfiles/*
sudo rm /Library/Keychains/apsd.keychain

could also try
completing applesettupe set network date & time
sudo profiles -N
sudo /usr/libexec/mdmclient dep nag
sudo profiles renew -type enrollment

could try checking the log while doing a NAG or during applesetup
CTL OPTION CMD T
log stream --info --debug --predicate 'subsystem contains "com.apple.ManagedClient.cloudconfigurationd"'
log stream --info --debug --predicate 'processImagePath contains "mdmclient" OR processImagePath contains "storedownloadd"
CTL
OPTION CMD T

manully rebuilding KDC isn't supported anymore, but if your going to erase anyway. Might as well try rebuilding it, see if it has any effect before doing an Erase and Install.
rm -fr /var/db/krb5kdc
/usr/libexec/configureLocalKDC

Really just throwing out ideas.

AdamCraig
Contributor III

Sorry if this has already been suggested, but this has fixed it for me 100% so far: Go back to the language choose page. summon Terminal as root with ctrl shift cmd + t. type in sudo profiles renew -type enrollment and click return. Then Proceed with through the setup assistant again.

tanderson
Contributor

@strayer I tried this (had to control+option+command+t) but Terminal opens as the mbsetupuser user and won't proceed because I don't have a password. Control+Shift+Command+T just beeps.

AdamCraig
Contributor III

@tanderson Step 1 is Go back to the language selector screen. I just grabbed a new computer and it loaded for me as root on that screen. (sorry I got the key command wrong) I don't know when it switches over to mbsetupuser.

tanderson
Contributor

@strayer Gotcha, thanks! I'll try it again and see how it goes.

Sandy
Valued Contributor II

@Strayer If I am able to get back to the Language screen, that worked perfectly. Thank you so much!

If I am forced to complete setup and try to get it into DEP manually I ended up with an unmanaged device.
Tomorrow is another day though so will see what happens :)

AdamCraig
Contributor III

@Sandy You can remove /var/db/.AppleSetupDone to re-run the setup assistant. http://www.theinstructional.com/guides/how-to-re-run-the-os-x-setup-assistant
or worst case wipe the computer.

Sandy
Valued Contributor II

Here's where I am do avoid having to nuke

If I can get back/reboot: Return to Language screen
Terminal as root with ctrl opt cmd + t
sudo profiles renew -type enrollment
sudo rm /var/db/.AppleSetupDone

~~~~

If forced to proceed through setup:
complete setup and verify date & time
sudo profiles remove -forced -all
sudo profiles renew -type enrollment
(follow prompts to enroll from desktop)

marcus_ransom
New Contributor II
New Contributor II

we have been having huge issues with this lately - running profiles renew -type enrollment doesn't always pull down the activation record. Interestingly running profiles show -type enrollment on a machine exhibiting this issue gives the response

Error fetching Device Enrollment configuration: (34006) Error Domain=MCCloudConfigurationErrorDomain Code 34006 "The cloud configuration server is unavailable."

It also seems to come in waves - speaking to $fruitco to see if there is some rate limiting happening as it occurs during peak deployment loads. Also looking to see if it is something on our internal network

Sandy
Valued Contributor II

My jamf is on-prem.10.17.1
I am using ethernet adapters. Yesterday after posting my last response, I pulled 5 brand new from boxes. Connected to power and ethernet and started each one. let them all sit on the language screen for 10 minutes. The first one connected perfectly to DEP and got the prestage. The other four skipped. NONE of them asked to connect to wifi which implies to me they have a connection, and when I look, their ethernet connection has DNS and search auto-filled. I was eventually able to get them all to go without restoring, but some required multiple repeats of steps I posted most recently. Any that made it to the desktop (skipped DEP) had correct date and time.
Directions and workarounds get pretty complicated if it requires multiple retries and lots of "If this... do this" . My original plan had been to send out the cartons to each school, since we paid Apple to asset tag and our inventory has been preloaded. This somewhat shoots that plan. My other fear of course is that by not paying attention some might get out into the wild without DEP :(

marcus_ransom
New Contributor II
New Contributor II

our issues have all been with JamfCloud. I don't really think there is an issue with Jamf as the devices don't actually contact the MDM until after they click the "Continue" button on the remote management screen (it's all to Apple at this critical stage). We also realised that the devices were in fact getting a network connection (despite sometimes showing that the ethernet adaptor wasn't connected) as there was no "connect to Wi-Fi" prompt and they had successfully gotten the .cloudConfigActivationRecord from Apple, and have .cloudConfigRecordNotFound instead of .cloudConfigRecordFound - still troubleshooting but it's challenging now that all of the devices have gone out to the end user

tcam
Contributor

@Sandy glad your making progress, at least a terminal command is faster then a erase and install.

Do you have to set date and time when you get to desktop? Or does it work even without setting date and time?

Does setting date & time in Apple Setup Assitence make a diffrence?
ntpdate -u time.apple.com

Also, have you tried checking to see if there is already a profile there?
profiles show

Sandy
Valued Contributor II

Hi!
profiles show reveals none installed
ntpdate -u time.apple.com. command not found
date reveals time is correct for PST (we are CST)

m_donovan
Contributor III

Can you set the date manually using date in the [mm][dd]HH]MM[yy] format?

date 0205112020

Would be today at 11:20am.

AVmcclint
Honored Contributor

For those of you wondering how to sync your clocks in Mojave and Catalina, take a look at this article in short, use the sntp -sS time.apple.com command, but make sure you read the article first.

wmehilos
Contributor

Couple quick question for everyone experiencing this, did you ever see anything like this when Macs were shipping with Mojave or earlier? And can you confirm through the Terminal that the Macs have full network connectivity (ie, they could load google.com or apple.com) I had this happen to half a batch of iMacs back in December, all running Catalina.

On those aforementioned iMacs, the ones that refused to pickup their DEP prestages also refused to talk to any network resource outside of our intranet. Before anyone starts asking me the obvious about firewalls and ports and all that nonsense, this batch of iMacs were all on the same network context. Same subnet. Same switch. If it was something with my network, A, none of them would work, and B, they wouldn't start working the moment Setup Assistant ended. Swapping cables didn't work. My JSS isn't behind a firewall. Basically everything MDM is over 443 these days anyways. The network infrastructure was not the problem. The networking stack ON the Mac seemed to be the problem. Reverse DNS failed for everything on the Internet at large. I couldn't ping or resolve anything outside our Class B blocks. All had valid IPs assigned via DHCP. None were MAC filtered. The moment, however, they completed Setup Assistant, network access to the outside world worked fine (nothing else having changed with the network config, either hardware or backend), profiles renew -type enrollment worked fine and picked up the DEP enrollment immediately and without error.

There is something specifically, and transiently, in the Setup Assistant environment that was causing them to fail to contact *.apple.com. At least that's what my observations lead me to believe. Whatever gets gummed up gets cleared when you wipe the machine or exit Setup Assistant.

As others have stated, this kind of inconsistency makes selling zero-touch a bit hard. Nuking and paving is not a realistic solution when you can't centrally deploy the installer and automate it like we could in the NetInstall days, and calling up remote employees asking them why their Macs haven't shown up in Jamf yet isn't great UX or security either.

AVmcclint
Honored Contributor

I've had more success with DEP enrollment when the Macs are pulled out of their boxes and setup as soon as possible after purchasing. The longer a Mac sits in the box on a shelf, the greater the chance of failure. I don't understand it myself, but the only repeatable solution for an affected Mac I found was to completely wipe the drive (not the volume) and reinstall the OS. I have verified the clocks on the Macs are in sync with available time servers, but it doesn't seem to matter.

mojo21221
Contributor II

We have that issue as well too. Usually just a machine here or there... Depending on your work flow and what you have scoped to install at enrollment. You may want to just continue through the Setup as if it were User Initiated Enrollment. Then run sudo profiles renew-type enrollment from terminal. If the device is DEP and has some how gotten to the desktop, this will pull down the profiles and enroll the devices as if it were DEP.

Sandy
Valued Contributor II

We have not had an OSX purchase in 4 years, except a few 5 packs. All my new macs have at least 75% battery charge out of the box We've had them stored about 1 month. I am doing 5 at a time, connecting to AC and ethernet. During setup assistant they skip over the Network page, they hang at connecting to activation server (prior to any jamf involvement).
Failure rate out of the boxes is anywhere from 1 to 3 in every 5, random order (Usually the first one works, but sometimes it fails). I am now getting them ALL to go without wiping or having to go to desktop, by going back to the Language screen (sometimes this requires a restart but not always) opening Terminal (command option control T) : sudo profiles renew -type enrollment. letting it sit for a couple minutes and that almost ALWAYS works. ALL my devices are on PST and pick up our zone with location services.

temilit
New Contributor II

Same issue for us with catalina, first time running in to this as we usually only deploy batches of macbooks for our new students during summer.

First of, "profiles renew -type enrollment -verbose" returns error 34006 as mentioned before, seems to be related to this great article here "https://nstrauss.github.io/mitigating-mac-enrollment-failures/"

If you don't get the language chooser during setup (ie terminal launches as _mbtsetupuser~~) you can boot to recovery launch terminal and cd into the volume and on to /var/db/ and run "touch .RunLanguageChooserToo"
this way next boot you will get back to the initial language chooser which gives you Root access when terminal is launched.

on to the mitigation, seems to be almost random when the profiles renew will work, sometimes i just leave the computers off for a few minutes and try again, sometimes complete reimage.

This is a annoying issue apple should fix, I recall somewhere reading about this beeing acknowledged by apple enterprise support as an actual bug in Catalina and should be fixed in the next OS verison, in my mind this should be a Major bug and should be hotfixed, but apple does not seem to care about enterprise customers ..

mconners
Valued Contributor

@temilit I just ran into this with our MacBook Airs we received. Apparently, there is something wrong with Catalina 10.15.5? This affects the T2 Macs and once the computer has been updated to version 10.15.6, the problems seem to vanish. I have yet to understand what it is, but with Apple helping me yesterday, I was able to get this resolved.

This also means updating to 10.15.6. Whether you boot to an internet recovery drive and do or use some other mechanism, the bottom line is once you have them updated, you should be good.