Login hanging after Monterey Upgrade - FileVault issue T1 MacBook Pros

JustGoogleIt85
New Contributor III

Ok so we have a very weird issue here in my org and I wanted to see if anyone else has ever run into this. On some of our late-2016 and mid-2017 15" Pros, users are having login issues post Monterey upgrade (12.1 and 12,2). When they attempt to login to their account it accepts their password but then just sits there with the progress bar and never logs in. Sometimes the account will let you in, but the users have no menu bar, extremely limited access to the keyboard, and it just never really logs in all the way. But you can use the keyboard shortcut to logout and then log back in with no issues. So given these experiences and after troubleshooting further, we have pinpointed that this issue is with FileVault. We can also go through a password reset with the recovery key, but never change their password, and the users can login with no problem; they can also login fine if we turn off FV and decrypt the drive. But then as soon as we turn FV on again and the drive encrypts the issue is back after the next reboot. We have almost 40 of our 2018 and newer MacBook Pros on Monterey with no issues at all but out of the 16 of the 2016/17 models we have had 6 experience this...Its scary because 200 of our Macs are one of these 2 models so we paused our upgrade rollout until we can figure this out.

We also have a ticket open and are working on this with Apple but I figured I would throw this out here in case anyone else has run into this. I am waiting for their support to get back to me with more troubleshooting. Anyone have any ideas of stuff to try while I wait to hear back from them? So far they've had us verify that the FV/user password are synced, and that the user has a secure token and its enabled for their account. 

45 REPLIES 45

Hugonaut
Valued Contributor II

How are you deploying the update? Policy, Mdm Commend, etc?

 

Are your end users Standard Accounts or Administrators?

- Local or Mobile?

 

Are you using a 3rd party tool to login? Jamf Connect / NoMAD?

(It sounds like you are using Native macOS Login Window but i just want to be sure)

________________
Looking for a Jamf Managed Service Provider? Look no further than Rocketman
________________


Virtual MacAdmins Monthly Meetup - First Friday, Every Month

Update is being deployed using a script with a policy that calls the startosinstall command to kick it off. But a couple of the impacted Macs were upgraded from Big Sur via Sys Prefs > Software Update before enrolling in Jamf.


We have a mix of standard and admin user accounts.


Our Macs are still AD bound mobile accounts. We will be moving to local accounts later this year (hopefully).


Still using the native macOS login. We do have NoMAD on all our Macs but just for AD password sync and issuing Kerberos tickets.

We also have a fun mix of security agents installed on all our Macs; Pulse Secure, CarbonBlack App Protect, CarbonBlack Cloud Sensor, Forcepoint DLP, Jamf Protect, Zscaler, and Cisco AnyConnect (VPN). However, while I have been testing with a Mac that's in this "broken" state I uninstalled most, and it doesn't appear they are to blame.

Hugonaut
Valued Contributor II

Well **bleep**! Okay, a lot of moving parts here. Best I can recommend (which you've already been doing) - is troubleshooting 1 by 1. Enroll a mac, bound with nomad & upgrade, see if the problem exists & then start over adding 1 at a time your security agents.

 

Upgrading from Big Sur alone is also crazy to see. I could understand an older version.

 

If you haven't done so already, review your Filevault encryption policy / config /etc & verify its setup up correctly.

 

I have a feeling this is because the accounts are mobile accounts sync'd with NoMAD but thats just a hunch.

________________
Looking for a Jamf Managed Service Provider? Look no further than Rocketman
________________


Virtual MacAdmins Monthly Meetup - First Friday, Every Month

I mean, if it truly is FileVault could potentially regenerating a key fix it?

sudo fdesetup changerecovery -personal

ianatkinson
Contributor

Hi,

 

Did you get anywhere with this? I'm seeing it as well. 

 

Freshly flattened macbook with Monterey on, mobile users (non admin). If the macbook is on the network via ethernet they can log in OK but if it's off the network so logging in with cached credentials then the the login stalls with the progress bar at about 50%.

 

Since unlock/login is now unified I'm not sure whether it's doing the unlock but the failing the login or quite what's going on. Also just seen the missing menu bar and half broken keyboard thing when I did get it logged in on the network.

 

Ian

tech90210
New Contributor

Having the exact same problem at De Montfort Uni, has anyone found a solution yet? 

JustGoogleIt85
New Contributor III

@ianatkinson @tech90210  

Still no solution, Apple had me send them SysDiagnose logs from both the stalled login and from when I get in where the menu bar is missing. The also had me run an lsappinfo when I was able to get in where the menu bar was missing (which wasn't easy) and its showing "Installer Progress" as the topmost process running.

 

From what I can tell while troubleshooting is that it appears to be failing during the unlock. I only assume that because disabling FV clears up the issue completely. I can provide users their FV recovery key and they can get into their Mac at the next login window.

 

What model Macs is this happening on for you? What 3rd party clients do you have installed? (ex. NoMad, Jamf Protect, etc..)

 

We ONLY see this on late-2016 and mid-2017 MacBook Pros but I am curious if its happening on other models.

I am so ready for this issue to go away...we have around 500 Macs in our environment and have stalled our Monterey upgrade until this can be addressed. The only Macs we have going out right now with Monterey are new M1 Pros that ship with the OS. 

I've done some more testing on this today.

We're only just looking at Monterey so I don't have a big install base to work with but we've replicated it on 3 MacBook pros which are all model  '15-inch Retina MacBook Pro (Mid 2015)' so that's at least 3 models it affects. Maybe it's any T2 Intel?

I was mistaken in that the one I saw the other day was a flatten it was actually an upgrade from Big Sur to Monterey (done via JAMF so a download of the full installer to /tmp not via app store or software update).

I have fully flattened one of them though and can replicate the same issue so I would say it's nothing to do with the machine having been upgraded.

The no menu bar/broken keyboard thing seems to happen if you power off during the stuck unlocking screen, plug the network cable back in, power back on and then log in as if it breaks the subsequent successful login. After a second reboot it's then back to normal, bizarre....

My conclusion on this is it's a Monterey bug with Filevault + mobile accounts when the domain isn't visible, works fine with the network cable in or with a local account. This is on 12.3 so a current bug.

 

What is your process to reproduce this issue? I have trouble sometimes reproducing this but I found doing a password reset usually triggers that but its not all the time.

This morning I found that if you also hard power off while logged in I can reproduce the no menu bar issue but if I shutdown or reboot by choosing the option in the Apple menu the issue doesn't appear and its back to normal. This is definitely a bizarre issue...it also makes no sense how this just pops up randomly on some of our users weeks after upgrading to Monterey.

The test process that I've used for this is

  • flatten machine (either to Monterey or Big Sur then upgrade to Monterey, doesn't seem to make any difference), as part of this our AD bind happens and will be set to mobile accounts
  • log in as admin and perform the encryption/key escrow to get FileVault on
  • with network cable inserted log in as an AD user, this will create the mobile account for that person
  • reboot and check the AD user can unlock the disk and log in (they can)
  • remove network cable
  • reboot and check the AD user can unlock the disk and log in (they can't)

ianatkinson
Contributor

I've tried this again today with the newest Intel one I can find which is 'MacBook Pro (16-inch, 2019)', although it takes a while it does log in OK in the scenario on this one so it may be model specific.

 

The progress bar sticks about half way like it did with the others, I timed this at ~22 seconds until it got stuck and ~1'22 until it logged in so I'd go so far as to suggest there's a 1 minute timeout in there somwhere when it's off the network that doesn't work properly on some older models?

 

Ian

I really wish my org could just replace the 160 or so 2017 and older MacBook Pros we have so I could walk away from this one since it does seem to be model specific. But MacBooks aren't cheap so.....😆

JustGoogleIt85
New Contributor III

Apple emailed me this in regards to my ticket so it sounds like their engineers are stumped by this just like the rest of us are who encounter the issue...

Apple is making progress in the investigation, narrowing down the circumstances under which the issue may occur. It appears as though when the installer is running in the background while other applications (processes) are functioning in the foreground, the installer needs to regain control and is unable to continue. We are attempting to determine exactly what circumstances are needed to cause this, as this is a rarely reported occurrence across all our customers.

ianatkinson
Contributor

Looking at the list of MBP models over the last few years I think there must be 3 that are affected from the testing everyone has done then, essentially 2015-2017 models:

  • OK - 2021 MacBook Pro (14-inch, 2021) ...
  • OK - 2020 MacBook Pro (13-inch, M1, 2020) ...
  • OK - 2019 MacBook Pro (16-inch, 2019) ...
  • OK - 2018 MacBook Pro (15-inch, 2018) ...
  • Affected - 2017 MacBook Pro (15-inch, 2017) ...
  • Affected - 2016 MacBook Pro (15-inch, 2016) ...
  • Affected - 2015 MacBook Pro (Retina, 15-inch, Mid 2015) ...
  • Monteray not supported - 2014 MacBook Pro (Retina, 15-inch, Mid 2014)

I don't know about anything else like Macbook Airs etc. from the same period? We don't buy anything other than MBP here since they're all heavy Adobe users.

We currently have 55 M1s deployed with Monterey at the moment and its a mix of these 3 models. So far, none of them have experienced this issue.

2021 MacBook Pro (16-inch, M1 Pro 2021)

2021 MacBook Pro (16-inch, M1 Max 2021)

2020 MacBook Pro (13-inch, M1, 2020)

Strannik
New Contributor III

I've also seen this issue a few times - twice it happened on 15-inch Retina MacBook Pro with TouchID (Mid 2017). First time in Monterey 12.2 and again recently on 12.3.1. Suddenly computer won't wake from sleep properly(black screen), user will force it to shut down, after restart no menu bar and trackpad/keyboard don't work properly. Restarting computer again seem to resolve issue. Haven't seen this before Monterey. Computers joined to AD with mobile accounts, have NoMAD installed for Keychain syncing and Kerberos tickets renewal.

jdaileycait
New Contributor

Its happening to our fleet of our 2021 MBPro M1 on Monterey.

When signing in using ethernet-- our mobile accounts take about 20 seconds but eventually login.

But if you unplug the ethernet, restart and try to login again (with the same mobile account), it will begin to login, and when the progress bar gets about 35%, it'll stall for 40 seconds, and then take me back to the login page.

When I try to log in again it works without issue, but its frustrating to have to type in your password twice (when you shouldn't have to). 

We also have FileVault turned on. Running MacOS 12.3


Interesting, that seems like a similar issue but the machine is handling it a little differently. On the older models you don't get returned to login when it's stuck it'll just sit there until you force power it off (in my experience anyway)

That is really interesting and definitely a similar issue but your users do eventually get logged into their Mac, ours can't get logged in unless we go through the one of the processes I mentioned in the original post.

I will pass this along to Apple that others are reporting similar issues with newer models that have the same config we are currently using. I want to demobilize a user having this issue and unbind their account to see if these issues go away. I can tell you the just unbinding from the domain didn't resolve anything in our testing.

JustGoogleIt85
New Contributor III

Hello! Wanted to touch base with everyone on here to see if any of you have resolved this issue. Apple is telling me they can't find anything in their logs so they want me to do isolation testing to narrow it down to one of the agents causing the issue. I've tried removing all of them, with the exception of Zscaler for our proxy and AnyConnect for VPN, but the issue persists. The only thing I haven't been able to test is whether or not moving the user account from mobile to local resolves anything. Anyone else have anything new on this issue to share?

I couldn't get any further on with it, it's a bug Apple need to fix I think. In my testing environment there were no other odd bits of software installed like proxy/VPN stuff.

JustGoogleIt85
New Contributor III

Update:

We finally had a chance to test demobilizing a user who has this issue. After moving from a mobile to a local account the issue is gone! At least so far after a couple reboots it looks like its resolved.

I really hope this is the case but after dealing with this strange issue I wouldn't be surprised if it came back in a few days or weeks. I sure hope this is the final answer though.

I now have this problem to with a 15" 2017 MacBook Pro. I tried this fix and apparently it works. 
But is there a more permanent solution yet or still no updates ? 
thx for your work btw

GrootUser1337
New Contributor

I fixed the Issue by running Onyx on the troubled Mac and the problems were gone.

Oh really? I will have to give that a shot and see if it resolves the issues for our users. Thanks!

ianatkinson
Contributor

Hi,

Did anyone who was dealing directly with Apple on this get to a sensible resolution as in it'll be fixed in an OS update? We're having to hold off installing Monteray for a lot of staff on laptops until this gets sorted out :(

 

Ian

Nope. They still claim they haven't found anything in the logs we provided them and the most recent thing they asked me was to try the Ventura beta... Here's their most recent response from them a couple weeks ago:

I see there have been changes related to this in the seed of macOS 13 Ventura. While we don’t recommend installing pre-release software on production systems if you’re able to try this out on a test system we’re interested in knowing whether you can still reproduce it or not.

I might try this and upgrade the one impacted user I left in this broken state to Ventura, but with how this issue presents itself there's a chance it could be back at some point in the next few weeks. Its crazy how they're handling this issue...or maybe its normal and I just don't have enough experience with them.  ¯\_()_/¯

Ah, the old 'we're too far down the cycle to fix it in this OS' response.

I was told at a recent Apple event by Apple that they basically move engineering staff off one OS team and on to the next as they develop them, the further along it gets the more people get moved. So by the this stage in the year I get the impression there's a full team of crack engineers working on Ventura somewhere then a couple of people sat in a corner somewhere still working on the last OS (possibly as a punishment from the HR department at Apple!?).

I wouldn't be looking at putting Ventura out until well into next year really so if they don't sort this out in Monterey that's going to be a pain, we'll just have to leave those people on Big Sur for now :/

 

Ian

TheStig
New Contributor

I have had this same issue on a Mid 2015 Macbook Pro, upgrade from Catalina to Monterey (Just in Monterey support, I know, but in support).

 

I have wiped the Macbook and ran a fesh Monterey 12.4 install and logins fine until I re-eanbled FileVault.

 

Resetting the SMC or booting in Safe Mode after a boot failure (Stuck with loading bar) and restarting subsequent logins work fine for half a dozen logins and the issue the returns.

 

Disabling Mobile account isn't an option as of course the users do need to login offsite / off domain. :-| 

 

ajufer
New Contributor

Just wanted to chime in here and say I'm running into the same problem on my 2020 MacBook Air fleet after our recent push to macOS Monterey. We have about 190 of those devices on Monterey and I've received (and verified) 3 accounts of this.

Like everyone else mentioned, it doesn't seem to fully 'hang' when it's visible to the Domain Controller, only when off-site. Boot times though are significantly different on FileVault Mobile Accounts vs. FileVault Local accounts, regardless of Domain Controller visibility.

Bootimes with filevault enabled are longer than without. 
Anyway, having a mobile account and having the mac not being able to connect to the DC on boot, you will have a boot-time increase of off very precise 60seconds until it runs in a "timeout" and completes the boot. 
I have experienced and tested this while setting up a new DC infrastructure in my company. 
The solution then was, like apple wants you to do, getting rid of the mobile accounts and dc binds and move to a SSO. In my case NoMAD. 

DannyV1
New Contributor

Hi All,

I Updated to Monterey about 2 weeks ago (Clean install), no problems until today. Macbook Pro mid 2017 Touch bar, locked up, pushed power button and have only seen these problems since.

Cannot use any keyboard except for typing remotely into a terminal window. but can't type passwords in.

Can't see menu bar at top of screen.

Can't access most menus via mouse.

Cant reboot properly (it asks for password which it lets me type but nothing happens) I've got to hold power button to shut off.

SMC & NVRAM reset don't help.

Can't change Firewall or any other settings as I can't type a password (even pressing power (finger scan) button which let's me enter the password still does nothing).

I tried to get the Oxv.. app off my lan, but it locked up trying to mount my network folders.

Can anyone please help?

Regards,

Danny

Hi Folks,

Well I got My machine going again after about 24 hours of pain!

It turned out (I think!) the main offender was 'Dropbox.app' trying to copy the world to my dropbox, but was failing because it was trying to copy Time Machine backups to dropbox at the time, BUT I'd NEVER ENABLED TIME MACHINE! so it was just in an endless loop I think. This app had been always asking me if I wanted to Back up This Or That to dropbox and I kept on saying 'No - Don't ask me again!' when I should have just deleted the **bleep** thing, which I now have.

Anyhow I then started to get things running again and ran Oxy...app which cleaned up a bit but found no errors.

So after a lot more cleaning up and Backing up, I'm starting to trust the machine again - scary stuff isn't it?

Cheers Folks!

Danny

Tonydig11
New Contributor III

I too am having this issue in our fleet. It is a mix of different devices. The latest is a user on an iMac. Was working just fine. Updated to Monterey last week and the issue started. The only common item is macOS Monterey. I will try some of the tricks mentioned above but it seems Apple is aware and do not have a fix and that is a shame.

ianatkinson
Contributor

It seems like this issue isn't actually going to get fixed in Monteray, and unfortunately some of the affected models (e.g. '15-inch Retina MacBook Pro (Mid 2015)' which we have a lot of) are not compatible with Ventura, so this effectively means those machines can't be upgraded beyond Big Sur for us. Big Sur will drop out of security support late next year when MacOS 14 comes out basically rendering all those end of life. The same would go for the 2016 MBP models.

Yes they will be ageing by next year anyway but in the current financial climate we are expected to make things last as long as possible so this is not a good situation to be in.

I need to test if a 2017 one works OK on Ventura if I can find a spare one somewhere.

TheStig
New Contributor

So I went again with a Mid 2015 Macbook just over 2 weeks ago, with the latest 12.6.1. Filevault enabled and so far so good, not quite counting my chickens yet, but will update on any changes. 

Hmm, OK I thought I'd give this another go and I can confirm it's still broken on 12.6.1.

To confirm the testing environment for us which produces this is:

AD bound + filevault on + mobile accounts on + no ethernet connection (so using the cached account)

We don't own any 2016 or 2017 MBPs for me to test if this is OK in Ventura however I have set up a 2017 iMac and can confirm that is exhibits exactly the same issue on Monteray and works fine when upgraded to Ventura.

This suggests to me that this has been silently fixed in Ventura so I don't buy that Apple don't know what the issue is, unless it was somehow introduced into Monteray after that was forked in Ventura.

Either way they should be able to look at the differences in the two in order to identify the issue I just don't think they'll bother at this stage :(

jac807
New Contributor

We found this issue with a similar set up and what resolved it for us is in Directory Bindings in the JAMF policy uncheck "Use UNC path from Active Directory to derive network home location".  For the home drive mapping in Finder we map the users shared drive by go to server and set a favorite link and have them access this way instead of the globe icon on the dock with UNC path from Active Directory.

 

The way I got this to work with an existing user who was having the issue, was to uncheck "Use UNC path from Active Directory to derive network home location", delete the account and home folder for the user. Recreate the mobile account and I have restarted multiple times and have had no issue.

 

Just for a test I rechecked the box, deleted that same mobile account I got working and then recreated the account again. Immediately I had the issue again.