Upcoming change will enforce LAPS on Prestage admin accounts (important!)

[rstasel]

Reposting this from my personal website to gain some visibility. 

UPDATE 1: I'm being told now that this change IS RETROACTIVE. Meaning your dozens/hundreds/thousands of endpoints that have been configured with Prestage enrollment, and an admin account created in that process, will have that password randomized. Which will break Secure Token/Volume Ownership/etc. Maybe you're okay with this, but if you're not, you NEED to reach out to your Jamf Representative and tell them. 

I wanted to bring up that a big change is coming to Jamf soon and they (Jamf) seem to think it’ll be well received all around: forced LAPS for Prestage (ADE/DEP) admin accounts (See bullet 2 here, https://learn.jamf.com/bundle/jamf-pro-release-notes-current/page/Deprecations_and_Removals.html, “Functionality to specify the local administrator account for computers in a PreStage enrollment”).

And to start with, I applaud Jamf for working on a LAPS solution. All we've ever asked in this whole thing is: Make it optional (for those that already have a solution, or want to see this bake longer). If you know this is going to break your fleet, you need to reach out to your Jamf rep and let them know. 

On it’s face, this sounds great. Static passwords are bad. LAPS is good. Right? Right?!

So yes, Good LAPS is good. Bad LAPS is… bad. Very bad.

First, Jamf is using Apple’s inferior LAPS functionality via the MDM “SetAutoAdminPassword” (https://developer.apple.com/documentation/devicemanagement/set_the_local_administrator_password). This basically just pushes a new hashed password into the OS, which breaks Secure Token and Volume Ownership (due to Apple reality). So Prestage configured admin accounts will no longer have any Crypto permissions (no logging into a Filevaulted computer, Applying updates on Apple Silicon machines, etc.

Second, Jamf is also forcing their same 29 character passwords (like Recovery Lock). 

Jamf has said “you can use the management account!” but many of us with Jamf Pro infrastructures that span several years (with thousands of computers) have management accounts that were used for Jamf Remote, never actual login. Jamf remote is no more, and Jamf decided to repurpose this formerly invisible account to be some kind of “admin” account (I honestly don’t know who they’re talking to sometimes). Those accounts are named who knows what (because it wasn’t visible), have no secure token, and still would rely on Jamf’s immature LAPS.

So, you say, “Just ignore Prestage admin account!” Somewhat easier said than done. If you don’t have that account created via Prestage, you cannot skip account creation (which makes sense, why would you want a computer with no accounts). So, you make the Prestage admin account some throw away account name that you can delete, or use in an emergency with the FileVault recovery key. Then you have a policy create your _real_ admin account. Great (assuming enrollment policies actually run properly). You login, but now notice that that account doesn’t have secure token, and bootstrap hasn’t been escrowed (it’s unclear why this is happening, as the first user to login SHOULD get SecureToken, and bootstrap should escrow. However, having tested this several times on a current macOS 14.2.1 machine, I can confirm this is the behavior). So you have to do the whole “no accounts have secure token, so you can grant secure token to the account itself” (sysadminctl -secureTokenOn username -password “password” -adminUser “username” -adminPassword “password”), and then manually escrow bootstrap (profiles install -type bootstraptoken). All good, but more and more work to work around Jamf’s decisions. 

We’ve spent months trying to convince Jamf to make this change optional. As mentioned, we don’t disagree with LAPS, good LAPS is good. We use it across the fleet of several thousand machines (we used a product called EasyLAPS). We just think their (Jamf’s) implementation is sub-par (and certainly below the level of EasyLAPS, which we pay for because it’s a good product). 

I think Jamf has a blind spot on this, and they’re not properly alerting their users this change is coming, and coming SOON! (we’re being told it’s scheduled for 11.3, which is slated for February). And because they don’t hear anyone upset (other than us?), they think everyone is cool and happy with this change. But they’re also not about to alert everyone either (I’m sure many many Jamf admins don’t bother reading the Deprecations and Removals section of the release notes (note! you should read these! You should read ALL the release notes)). We’ve met more than once with the Project Manager that’s working on the LAPS rollout, and it goes no where (instead of listening to us, and our concerns, it turns into “how can we help you implement our LAPS solution?”). We’ve told Jamf this, but it’s honestly making us look at switching MDM vendors, which I’m sure you realize is a HUGE undertaking.

[Deanna · Accepted solution]

Thanks @rstasel 
Jamf is looking forward to bringing LAPS functionality to the GUI and in 11.3 you'll have the ability to view/rotate passwords from the Inventory page. The ability to configure settings via the GUI will be in a subsequent release.  There's been a lot of chatter about the PreStage account and I can share that:

• LAPS will not be on by default for the PreStage account
  
  • The ability to set a static password in the PreStage will remain. 
    • This means that there will be no change to your current static passwords.
• The ability to enable LAPS for the PreStage account will be available in the GUI in a future release.  Admins will have the ability to "opt-in" to automatic password rotation.

Current workflows that rely on a static password in the PreStage will remain unchanged.  More information to follow in an upcoming blog post.  

[cbrewer]

At the very least, rolling this out without the ability to specify the password length would be a huge miss. A 29 character password is fine for a break-glass account, but what about organizations who actually use that account for Help Desk and IT troubleshooting, etc?

[rstasel]

believe the answer we've gotten is "it's under review". but yes, you're right. Our biggest concern honestly isn't this breaking our fleet, it's going to completely break out technicians... lol. 

[jwojda]

Thanks for the heads up on this.  This seems to be the 2nd time Jamf has released important information over holiday breaks where there are fewer people to notice, but could have immediate ramifications. NoMAD going EOL at the end of December '23 and now figuring out LAPS across the fleet.  I've reached out to Jamf for clarification on these changes.

[foobarfoo]

Since this thread is going to be riddled with complaints about making LAPS mandatory, I figured that I'd try to weigh in some counter-arguments. First, no-one likes environments where you have accounts created that serve no purpose, and that you don't know the password for, and yet they are the same for thousands of machines. Yes, I know, that shouldn't be the case, but it was for us due to a "technical debt". And that was true both for the account created in the prestage and the user enrollment created admin account. So we've been running LAPS for some time to remedy that situation. Regarding filevault, we do escrow the recovery key. And for those who don't have a valid key, we use escrow buddy to remedy that. Never had any issues in months. So it's highly dependent on how you work today, but I think there are many alternate ways to solve whatever you used those accounts for in the past.

[rstasel]

Yup, you're right. LAPS is good. Useless accounts aren't good. rotating passwords is fine. But Jamf took a look at this problem and just said "YOLO" and did the bare minimum to implement. Didn't ask Jamf users their thoughts, wasn't interested when we said "Hey, we'd like to provide feedback on this and how it's going to break our setups", etc. 

Yes, in some environments, not having a default useraccount have SecureToken is fine. In our situation, it's not. Technicians do all computer setups, we're not zero touch. 

None of these issues are insurmountable. Our concern is Jamf isn't listening to the community (isn't asking even, or heck, isn't even widely broadcasting this change), and is making it mandatory, which may break lots of existing workflows. As mentioned, LAPS is great when done right. Bad LAPS is bad. Even Microsoft hasn't forced LAPS on their users, and they largely invented the idea. 

[whiteb]

---

@rstasel wrote:
You login, but now notice that that account doesn’t have secure token, and bootstrap hasn’t been escrowed (it’s unclear why this is happening, as the first user to login SHOULD get SecureToken, and bootstrap should escrow. However, having tested this several times on a current macOS 14.2.1 machine, I can confirm this is the behavior). So you have to do the whole “no accounts have secure token, so you can grant secure token to the account itself” (sysadminctl -secureTokenOn username -password “password” -adminUser “username” -adminPassword “password”), and then manually escrow bootstrap (profiles install -type bootstraptoken). All good, but more and more work to work around Jamf’s decisions.

I've been seeing something in the last week where the PreStage admin account is 'stealing' the SecureToken that should be getting issued to the end user in our 1:1 deployments. So far all are on 14.2 / 14.2.1. Administrator gets SecureToken and Volume Ownership even though it hasn't even been signed into yet. And the Bootstrap token doesn't get escrowed. Not quite happening on every deployment. Just put a ticket in. We point to SSO and lock device information to populate from SSO during local account creation.

But yeah, the last time I checked-in with this whole saga it was totally optional for PreStage admin, but being enforced on the management account which we don't even bother creating anymore. I know tech's like to have the option of using our local admin account to approve updates in a pinch so that would suck if it broke that.

I also rolled out macOSLAPS a couple months ago, and am very happy with it. I have absolutely zero need to have this forced on us.

Techs will grill me if there's no way around this and 29 character minimum, that's wayyy too complex. Was that mentioned somewhere?

[rstasel]

I saw something similar to this in testing with and Intel macOS 14.2.1 computer and Jamf 11.1.1. I had a policy create a user account prior to any login after enrollment (prestage account created assumed to just be "throw away" since it wouldn't keep secure token). Logged in with policy created account, didn't get Secure Token, Bootstrap didn't escrow, etc. Was able to self-grant Secure Token and then manually escrow bootstrap... going to test with M1 here soon. If you have AppleCare OS support, would definitely raise this with them. We don't have it, so best we can do is file feedback via Appleseed. =/ 

That said, this does mean Jamf's LAPS plans are tenuous based on Apple's bugs (if that's what it is). I just don't get why Jamf, the premiere MDM, wouldn't get some face time with Apple to hammer out a GOOD solution. Sure, maybe it doesn't come until macOS 15 or 16, but at least it means a real laps solution and not whatever this mess is. 

[whiteb]

Yeah it sounds like we won't even know what the GUI looks like until it's actually out? I don't plan on using it if I can.

With these recent Secure Token, etc issues I've been seeing with M1's and 14.2.1 and the impending PreStage admin JAMF LAPS fiasco, I think I'm just not going to create a PreStage local admin anymore at all.

Tentative plans for PreStages without local admin creation boxed checked:

1:1 deployments - user signs in with SSO and a local account is created. I'm going to have the local admin get created after the user signs in. Need to test behavior there.

Shared Labs - Instead of SSO I think I'm just going to do Google LDAP for auth (so just username + password, no MFA) - and techs will need to sign into a service account, which will then get promoted to admin. Then NoMAD gets installed and it's ready for shared use. Also need to do some testing.

I've also read that enrollment triggers can be unreliable so I'm going to try and pilot Setup Your Mac. Was testing last week and had an enrollment where not a single enrollment policy ran + the Jamf Daemon wasn't even running (according to the JamfCheck utility), API reinstall of the framework fixed everything. Enrollment logs showed some network errors so I'm not sure yet. But we need to be able to just hand devices out and trust everything will work, can't babysit every enrollment.

[rstasel]

Are you going to do anything to make sure your admin account gets Secure Token, or bootstrap properly gets escrowed? 

fwiw, NoMAD is EoL... https://github.com/jamf/NoMAD

https://www.jamf.com/blog/jamf-to-archive-nomad-open-source-projects/

[whiteb]

The 1:1 user should get a secure token + bootstrap token escrowed. Then, any additional accounts will also get a secure token and volume ownership. That's expected behavior, that's how it's always worked in my experience. This wasn't even remotely a concern prior to recently when I started seeing those issues. All of our users on shared computers have secure tokens and volume ownership across the board (as well as Admin account).

The stuff I've been seeing that isn't expected behavior seems to involve our admin account getting created during PreStage. So I'm hoping taking that out of the equation will ensure the first user reliably gets secure token and bootstrap token gets escrowed. As long as that happens, the admin account we're creating via policy should get secure token + volume ownership as well. If  that doesn't go as expected, I'll be at a loss.

For the new shared deployments I'd mentioned, since the techs will be logging into each computer for the first time with the same (known) service account creds, I was going to add some checks for those deployments/prestages, where I can double-check for a secure token + bootstrap token escrow and fix as needed by securely passing creds through as needed. But really hoping those work the same as 1:1.

And yeah I saw that about NoMAD, but thank you. I have enough Jamf Connect licenses to completely change over (rep saved us money by bundling those with Protect), just haven't had time to tackle that.

[A-bomb]

I have tested this and it works but if you think any of this is unneeded or have a better approach, please let me know.

Current PreStage workflow:

1. admin1 is created at PreStage.

2. A tech logs in with admin1 which gets a Secure Token.

3. The bootstrap token is then escrowed.

4. A policy creates admin2 and a non-admin service account. 

5. A script then has admin1 grant admin2 a Secure Token.

LAPS will retroactively break the Secure Token for all existing admin1 accounts. Not a huge deal because I was expecting something to happen with local admin accounts at some point. Either internal or external. Hence admin2 with a Secure Token.

Here’s the impending problem. If LAPS is activated by Jamf it will break the PreStage admin1 Secure Token so we need to find a way around the PreStage admin account. Sucks.

Alternate PreStage workflow:

1. A throw-away admin1 account is created at PreStage.

2. A tech logs in with Entra ID SSO as the first account and therefore gets a Secure Token.

3. The tech elevates their account to admin, temporarily.

4. A policy creates admin2 and a non-admin service account.

5. A tech manually escrows the bootstrap Token. 

6. A tech manually grants admin2 a Secure Token with their account.

7. The tech account is demoted.

[A-bomb]

The retroactive piece is ridiculous. We are planning to convert to LAPS this year but forcing it retroactively is way too heavy-handed. Luckily I i’ve always been worried about this and setup a secondary admin account now and have the primary PreStage account grant the secondary a Secure Token, just in case. I actually wasn’t aware that an account can grant itself a Secure Token, and then bootstrap escrow. I’ve tried this in the past, and it never worked, but I’ll definitely try it again, reading here that it works for folks. Thanks for that. 

[rstasel]

So caveat about secure token self granting. I tried this yesterday on an M1 with 14.2.1, and it wouldn’t work. No users had secure token. 

I thought I had done this on apple silicon previously, but maybe not. Or maybe Apple moved the goal posts again. It does work consistently on Intel machines though. 
Makes me think we need to highlight problem computers before they go out the door… 

[A-bomb]

That's exactly what I have seen as well.

[A-bomb]

I just wrote my entire Jamf team and they wrote back very quickly with this:

I have thoroughly read through that entire Jamf Nation thread and definitely hear your concern and where you and many other admins are coming from on this topic. With that said, I'm not going to do the typical "I'll relay your feedback to our teams" response. I talked with my manager and we think this Jamf Nation thread needs to be put directory in front of our Product Managers right away. I will be reaching out directly to our Product Management team this afternoon to get this thread of feedback to them. 

[rstasel]

Thanks A-bomb. Good this will go there. I'm relatively confident they're aware since we've been meeting with some of that team, but thank you. =) Glad this is finally getting some attention. =)

[rstasel]

I forgot to originally include my idea that's been posted since August... https://ideas.jamf.com/ideas/JN-I-27528

[jwojda]

I've got a meeting with our Jamf rep today on this too.

[A-bomb]

My team wrote back this morning and said they will get back to me today and that "it looks promising"!

[takayuki]

Thank you @rstasel and others for bringing this critical change to our attention. We have also opened a support ticket with JAMF Support to clarify the change and assess its impact on our Mac management operations accurately.

[A-bomb]

Things are looking good.

[A-bomb]

UPDATE: They aren't going to force LAPS rotation with 11.3. Instead, they are going to introduce the feature in the GUI that you can apply to machines yourself. They aren't using MDM for LAPS, they are using the Jamf agent. You will be able to grant Secure Tokens to other accounts and view the LAPS password in the GUI.

[rstasel]

We still have some questions that are needing answered, but yes, they're delaying rollout at least one version. It's still unclear if it's retroactive, etc. Will post more once we have some clear answers. Thanks for also holding feet to the fire. =)

[rstasel]

Right now they're not forcing in 11.3, but 11.3 will still be MDM rotation. 11.4 is going to change to JMF account doing rotation, but it's still going to be forced, and may or may not be retroactive (unclear based on info). If it's retroactive, it's still going to break secure token if JMF rotates the password for an account it doesn't know the current password for. Also still not sure WHY they want to force LAPS at the same time they implement it. Heck, really, why are they forcing it at all?

[rstasel]

Update on this based on responses from last week and today from Project Manager. We're still working with them, but right now, what we're being told:

• They're delaying implementation (rather than forcing in 11.3, they'll force in 11.4) and are pivoting to using the JMF account to rotate the password for the prestage account rather than using MDM. 11.3 will use MDM, 11.4 will use JMF. We're supposed to see updated release notes (again, deprecations and removals) reflecting updated timetable. 
• 11.4 they will take away the ability to set prestage password, it'll be forced LAPS. There will still be no way to turn off LAPS once it's forced on. So once again, they're forcing the change with the same version they're implementing something. =/
• They're still saying it'll be retroactive, which means if the JMF doesn't know the existing password, it's going to break SecureToken, etc.
• still no ability to set password length, makeup, etc. 
• still no documentation on how Jamf handles failed rotations... at this point, kinda sounds like it'll just do a "reset" which will break Secure Token, etc. 
• Still lack of clear communication to Jamf users about this upcoming change. 
• No clear guidance at this time how to "break" LAPS so that it stops trying to rotate the password. With MDM method you COULD change the GUID, but that caused other issues. 
• Still zero answer, or even acknowledgement of question, when asked "Why are we forcing this?" They completely ignore us when we repeatedly ask/suggest "make this optional..."

I continue to be amazed at how Jamf engineering, or whomever, doesn't seem to understand why this is such a big deal for us... we've been raising alarms about this since August. =(

We're continuing to attempt to engage and impress upon them why this is a problem. Please continue to push on your account reps... they need to know this is bad, and should not be blindly rushed into. 

[A-bomb]

Thanks for the update.

[Mark_Lamont]

interesting thread, one I will also ping to jamf.

[rstasel]

We have a meeting with Jamf again on Thursday, potentially involving engineering. I've also emailed one of our Apple EDU reps to let them know this is coming, and they were also surprised this was mandatory (they assumed it was optional). They have forwarded to their Jamf SE for information... At this point, we're just trying to get this in front of as many eyes as possible. Next step if Thursday meeting doesn't result in any real progress is to start emailing people on Jamf's leadership page. =/ 

[mark_lynch]

I love how I'm just now hearing about this from a 2 week old post being shared in another circle. Whelp, time to ramble a Success. I feel bad for them, because they get to receive a lot of complaints from us that they have no more power than tossing it into the void.

[rstasel]

Good morning everyone. I believe @Deanna will be posting to this thread soon with an update. Will leave that up to her. 

[Deanna · Accepted solution]

Thanks @rstasel 
Jamf is looking forward to bringing LAPS functionality to the GUI and in 11.3 you'll have the ability to view/rotate passwords from the Inventory page. The ability to configure settings via the GUI will be in a subsequent release.  There's been a lot of chatter about the PreStage account and I can share that:

• LAPS will not be on by default for the PreStage account
  
  • The ability to set a static password in the PreStage will remain. 
    • This means that there will be no change to your current static passwords.
• The ability to enable LAPS for the PreStage account will be available in the GUI in a future release.  Admins will have the ability to "opt-in" to automatic password rotation.

Current workflows that rely on a static password in the PreStage will remain unchanged.  More information to follow in an upcoming blog post.  

[A-bomb]

Thanks @Deanna

[rstasel]

No, seriously, thank you Deanna. You've been a great help through all of this, and I'm sorry if I've contributed to any gray hairs for you or any on the team. Thanks for listening to feedback and being willing to adjust. Greatly appreciated. =)

Could we ask that release notes be updated at some point to reflect this change? not sure how you deprecate a deprecation notice... =/ 

[Deanna]

Hi @rstasel - Thanks for your partnership.  As far as the gray, I prefer to think of it as "hair sparkles."

The 11.3 GA release notes will be updated.  The work is on hold as we determine the right path forward.  There are no plans to remove the ability to set a static password on the 2024 roadmap.  The deprecation notice will stay posted for visibility, but the timeline will be removed.  

[rstasel]

Hey Deanna, so 11.3 just dropped, and indeed, the date is gone from the notes, but the language is still... worrisome? discouraging? 

"

Functionality to specify the local administrator account for computers in a PreStage enrollment—

In a future release, the ability to specify or modify a local administrator account password in a PreStage enrollment for computers will be removed from Jamf Pro.

Once implemented, the local administrator password solution (LAPS) will provide equivalent functionality for securely viewing and modifying macOS account passwords on managed computers. For more information, see the Local"

This still reads as "this change is coming, like it or not... we just don't know when". This isn't what any of us want to hear, or read. Kinda thought we all made that clear... =(

[mark_lynch]

Sounds like one or more people need to get on the same page at Jamf.

[Deanna]

Hi @rstasel 
I shared in this post three weeks prior that the date would be removed, but the deprecation notice would remain.  This is to provide visibility to admins of future changes.

The post stated "The 11.3 GA release notes will have the date removed from the deprecation notice.  There is no plan to remove the ability to set a static password in the PreStage on the 2024 roadmap.  The notes will remain for visibility as we determine a path forward."

The immediate goal is to bring LAPS functionality to the GUI and make it accessible for those who want to use it.  Beyond that, additional enhancements will be researched and reviewed before changes to the PreStage are prioritized.  

[mark_lynch]

Just a thought, leaving the deprecation notice in place appears to already only be serving confusion. Putting my ex-Jamf hat back on, there's a high likelihood that the deprecation will be seen by other admins who then raise similar concerns to support or customer success, who likely don't spend all their time in a Jamf Nation thread to get the information that it's still all a big ol' shrug at this point.

Further, removing the date doesn't clarify that it's still in flux, but only serves to add confusion that it could happen next update, or it could happen 2 years from now, but to anyone not in the thread that it's still been determined as happening.

[rstasel]

Hi Deanna,

Right, I saw that response and replied. 

I understand you're trying to paint some nuance here, but the release notes do not have any of that. All the release note message conveys now is "we're doing this. we don't know when, but it's happening". Not "when we do it'll be optional" or anything. Which, brings us right back to ya'll hanging seriously workflow breaking changes over our head, only now we don't know when they'll happen. And worse, because it just lives in the release notes now, that could be seen as a "we gave everyone plenty of notice about this change, so we're doing it next release". 

I thought we all made it pretty clear this must be optional. 

What _should_ happen here is that message gets removed from the release notes, or better, a "reconsidered" note added, and a statement saying that when/if this is re-evaluated Jamf will reach out to admins across their customer base and discuss potential issues/solutions/etc. Or heck, they could even say: "In a future release, the ability to specify or modify a local administrator account password in a PreStage enrollment for computers may be removed from Jamf Pro." (emphasis is mine). 

Does all that make sense? I know time was spent getting all this ready, and we appreciate Jamf backing off on implementation... but now we're just waiting for the other shoe to drop. =(

Thanks.