OS X need to repair your library

@bentoms, this has worked on 10.8.5 and all previous versions -

@jruskey, - I suspect rolling back to 10.8.x will work - it did for us, but we are still trying to get 10.9.4 operational for this upcoming semester

[for reference, I work in the same department as @jake.snyder ]

@bentoms][/url][/url][/url, @agrosvenor][/url][/url, I can't roll back. These all shipped with 10.9.x and not matter what I try, it won't allow 10.8.x to install. So, back to the fun.

@jruskey that's really unfortunate, @agrosvenor and I haven't made any progress in the last 24 hours

we were assigned an Apple engineer from Apple Enterprise Support this morning so we'll see if we can make any progress once they reach out to us

@jake.snyder, There is the obvious problem of the home directories not mapping. We are all experiencing that. Let me know what they say. I have confirmed if I use AdmitMac from Thursby, my home drives mount correctly. Now, that isn't the solution since it is about $70/machine for that plug in. However, even if we paid the money for that, the other issue is that no network users can log in after the Mavericks machine goes to sleep. You either have to reboot or log in with a local account, log out and then network accounts are fine. Are you also lucky enough to have this other problem as well?

Please let me know what your apple engineer says. Thanks so much.

We also have our AD homes on a SMB server, but quite frankly, I don't think this is the root cause. Same for the udld force cache.

I also have seen couple of times this since 1 or 2 years now, and the only thing I could notice when pb happens, is a mismatch between the UID in the Directory service record for that user (dscl . -read /Users/yourADuser ) and the UID seen on the user's Library folder on disk (run a simple "ls -lnr /Users/yourADuser/Library").
Remember that OSX silently checks, and updates if needed, any user's record entries during session logon (99,999% of the time, there is obviously nothing to update but...) : this is what maybe created the permissions mismatch and the annoying popup.

This happens with old users, brand new test users created, etc. It is every network user. It is only on 10.9.x. Any other versions from 10.6.8 and up, no issues whether new or existing user.

We are seeing the same error on our older classroom computers. Our Mid 2009 MacBook Pros and Early 2009 iMacs prompt us after imaging: "OSX needs to repair your library in order to run applications..." Furthermore they fail to bind to the active directory at imaging time. If we log into the Local Admin and repair the library when prompted - we are then able to manually bind the computer to the AD furthermore, the required applications work for the Local Admin and AD users.

Our newer computers (Late 2009 +) do not display the repair message and bind to the AD. Imaging is a flawless process on these iMacs and MacBook Pros.

We are using 10.9.4 build 13E28 images created from AutoDMG - its quite the predicament, we may end up imaging our classroom computers with 10.8 again. This is the first time I've experienced the age of a mac producing different imaging results in such a manner.

@jake.snyder, has your apple engineer been able to resolve anything as of yet?

@jruskey not yet, they've been getting pretty deep into logs and verifying what we're seeing, I'm hoping for a resolution soon. I'll post on here as soon as I get anything.

@ebioit have you tested brand new accounts with network home directories on 10.9.4? I'd be curious to see if you have success or not.

I saw the same things with old user accounts from 10.8 when we moved to Mavericks. We saved the work-data and made a new account/library.
Base OSX 10.9.4 is from AppStore and packed with Composer.
Before that we were also exploring the everybody-deny-delete-road: We removed that ACL from the user template and wrote a launch agent for removing. But OSX "repairs" that - so you always end up with files with everybody-deny-dele-ACL in the Library. The only remedy we found was ExtremeZ-IP (handles the ACL correctly).

@michaelhusar @jruskey I just downloaded the trial of Acronis ExtremeZ-IP and can confirm we didn't have account corruption when using AFP, but it still only created three folders (Desktop, Documents, Library).

I'm still waiting for next steps from the kind folks at Apple. They are very detailed and thorough but I'm afraid I won't have time to wait for their solution.

I'm preparing two options for the scenario where Apple can't figure it out in time for the start of my school year:

1. Roll back to 10.8.5 on iMacs that will support it and run forced local home directories on our 20 brand new imacs - less than ideal for those that will need to use these computers.
2. The other option is to implement Acronis ExtremeZ-IP, but the unlimited client licensing is expensive for a technology that Apple is moving away from.

Michael - Do you have instructions or tips for how to setup ExtremeZ-IP? I followed the instructions I found for version 3, but they seem slightly dated. It seems to be working except that its only creating 3 folders.

Thanks all. If Apple support comes through, I'll post a follow up here.

@jake.snyder - I understand your frustrations. What we did with the labs that can't be downgraded is similar to option 1. We created a generic local login since it is in a lab. Then, we have Aliases on the desktop that allow users to connect to their home directories, shares, etc. Not ideal, but didn't have a lot of options.

I did use AdmitMac from Thursby and it works very well. I think education pricing is about $132 per license($110 for the license and $22 for support). The more licenses you purchase, you get volume pricing. It works very well, but it is expensive. We ended up purchasing a 5 pack for the machines that we are rolling out to staff that won't be in labs.

If you hear anything from apple, let me know. Thanks so much.

sure: Security:
Permissions:
NoCheck Allow Mac clients to change permissions
Check Reset permissions on move
Check Support UNIX permissions
Check Support ACLs on all volumes
Check(ed) Show only accessible Folders + Files
Directory Services:
Check Global Catalog: We have a AD-Account for that

Search:
Check Index volumes for search

Other stuff is default/no game changer I guess.

At the first sync I also see only 3 folders - it seems to build up incremental - as the user needs the stuff.

I use a Profile Manager Config to set what gets synced.
I use the defaults of "mobility" on ~/Library
with these exclusions

~/.SymAVQSFile
~/Documents/Microsoft User Data/Entourage Temp
~/Library/Application Support/SyncServices
~/Library/Application Support/MobileSync
~/Library/Caches
~/Library/Calendars/Calendar Cache
~/Library/Logs
~/Library/Mail/V2/MailData/AvailableFeeds
~/Library/Mail/V2/MailData/Envelope Index
~/Library/Preferences/Macromedia/Flash Player
~/Library/Printers
~/Library/PubSub/Database
~/Library/PubSub/Downloads
~/Library/PubSub/Feeds
~/Library/Safari/Icons.db
~/Library/Safari/HistoryIndex.sk
~/Library/iTunes/iPhone Software Updates
IMAP-
Exchange-
EWS-
Mac-

and also added those:

~/Library/Developer
.fstemp
~/Library/Safari/LocalStorage
~/Library/Mail/V2
~/Dropbox
~/Library/Mobile Documents
~/Library/Messages
~/Library/Application Support/AddressBook/Sources
~/Documents/Microsoft User Data
~/Documents/Microsoft-Benutzerdaten
~/Library/WebKit/LocalStorage

in future probably
~/Library/Containers, ~/Library/Accounts, ~/Library/IdentityServices, ~/Library/iCloud

One other thought: Do you use a Win2012 Server? I had to make sure the SPNs are all low letters (!) otherwise Kerberos does not like it and there are on and off sync problems.

@michaelhusar thanks for your quick response!

My ExtremeZ-IP settings were fairly close to yours except for the "reset permissions on move" and "show only accessible". I made adjustments to my settings to match your settings. It's good to know that you're seeing the three folder thing initially as well.

I will be using folder redirection mostly for Adobe related cache.

We're using Windows Server 2008 R2, but the SPN is already lower case. Good to know though!

Thanks again!

@jake.snyder Jake, I've been following along with the same issue, and eagerly awaiting the response from your Apple engineer.

I also have Everyone ACLs getting explicitly set on new Windows Server 2008R2 home folder items by Mavericks 10.9.4. My only workaround at this point, has been to build my own "User Template", by logging into Mavericks as a local user, logging out, then ditto'ing the created local home folder up to the server.

I then strip off the everyone ACLs: icacls U:Studentsusername /remove:d everyone /t /c /q /l on the home folder, then grant full control to the user.

Interestingly, any folders manually created by the user aren't handicapped by these everyone ACLs, but any folders/items created by Mavericks OS processes are.

Those user home folders that I migrated over from an Xserve file server don't have this problem.

@dhandy @jruskey @michaelhusar @ebioit

I just got this update from Apple, but haven't tried it yet:

After some more testing I have been finally able to reproduce the reported issue and found a solution in order to address this behaviour.

Could you please check in your Home share properties under advanced if you have a Feature enabled called: "Enable access-based enumeration" ?

If this is correct please remove the nsmb.conf ( $ sudo rm /etc/nsmb.conf ) global Configuration again and disable this feature.

After this has been disabled create one more Test user, restart the Client and try to login with this new user account after restarting.

@jake.snyder

FWIW, access-based enumeration is off for me, and has never been enabled on the new share points I set up on a new server.

Here's my scenario to recreate the issue (sorry for the length):
----
Active Directory/Open Directory bindings
Student Network Home folders on Windows Server 2008 R2
Mavericks 10.9.4 clients, OD server is OS X Server 3.1.2

Steps to recreate:

Create a new user in AD.
Active Directory creates the empty home folder with the user's name, when path entered in Home Directory attribute in AD. (Without this, user login denied.)

Permissions on empty home folder automatically set with new user in Full Control over subfolders and files, and System and Administrators Full Control propagated down by inheritance from parent share point.

Then, first login by user on Mavericks client, Mavericks populates the empty home folder with a Spotlight folder, Library and Downloads folder (no use of /System/Library/User Template)

"Downloads" inherits home folder permissions, and sets explicit Everyone Deny Delete this folder only.

Library also inherits home folder permission, and sets explicit Everyone Deny Delete this folder only.
*ALSO sets "S-1-5-88-3-448" Deny None this folder only.*
*** (This SID translates to Everyone) - This is where problems start.

Client user login experience shows "Keychain Not Found". Attempts to Reset to Defaults repeats twice. Then error "Home folder for user isn't located in the usual place or can't be accessed" User can't get to home folder at all.

Logout, and user can't login again. ("Unable to login at this time.")

If I run the following command, to remove the "Everyone" ACLs on the Windows Server, the user can login again.

icacls U:Studentsusername /remove:d Everyone /t /c /q /l

Home folder now accessible to the user, with no keychain error. Desktop folder now available (permissions set again with Everyone Deny Delete). No Documents, Pictures, Movies, etc. yet.

Various other Library sub folders and Library items are created by the client OS, but permissions on these items are all set to explicit "Everyone Deny Delete this folder only" and "S-1-5-88-3-448" Deny None this folder only" on everything.

The user can create a folder in their home folder, but there's no "everyone permissions" set, and looks normal.

*My current workaround is to create the user's home folder as a local home folder, login, logout so it gets populated, then ditto this local home folder to the server. I then delete the everyone ACLs, and grant the user Full Control over their server home folder.*

Hopefully this is aligned with the symptoms you're seeing. Otherwise, I'm in the wrong thread.

@dhandy

I'm seeing something a bit different than you, but the issue is related. Perhaps its because I'm not using Open Directory. My home folders are created on first login too.

@dhandy @jruskey @michaelhusar @ebioit

Significant progress today: My environment seems to be working with Access Based Enumeration off and forcing SMB1 on each client.

In order to enforce SMB1 on a mac client:

1. Create the Global Config: $ sudo -s $ sudo echo "[default]" >> /etc/nsmb.conf $ sudo echo "smb_neg=smb1_only" >> /etc/nsmb.conf
2. Restart the OS X Client
3. create a new AD Test user
4. Login and check if the issue still persist

I'm still seeing these permissions with the above changes, but I'm still able to write:

Deny S-1-5-88-3-448 None <not inherited> This folder only

Deny Everyone Delete <not inherited. This folder only

Forcing SMB1 Only now allows login and creation of Library folders, so that's a great step! But there's no observance of the /System/Library/User Template for new home folders. (But this may always have been true in the auto-creation of new SMB home folders - I don't know.)

For instance, launch iPhoto on a new home folder, and it can't create its own database due to the absence of a Pictures folder.

Guess we can't have everything.

@dhandy I noticed that those folders eventually get created, not sure why its not happening all at once like it used to though

Adobe CC is working on network homes, which is huge. Notes and Photo Booth do not work.