Can't bind to AD in 10.9 Mavericks

[Lincoln]

OK this was brought up in another thread but as was suggested it is probably better in it's own thread so here goes.

I've been testing deployment of Mavericks in our environment. I cannot bind to AD either with the JSS binding or manually. If I upgrade an already bound machine to Mavericks things are fine, logins work, home folders mount etc. If I image a machine with binding in the config it fails to bind. Post imaging binding fails and so does manually binding.

I wondered if the problem related to existing accounts in AD so I tried reseting accounts, still no good. Then I tried deleting accounts altogether. Still same error.

When trying to bind everything looks like it is working as the machine gets the AD details and checks for existing accounts, when it gets to the point where it says "Joining Active Directory domain..." it fails saying, "The plugin encountered an error processing request."

I have machines on 10.6.8 - 10.8.5 with which I have no binding problems.

I know I am not the only one having this problem, anyone got any great ideas?

Regards

Lincoln

[Kumarasinghe]

AD binding with manual and Casper methods work fine here but we have an issue with "Allow administration by" groups with Nested AD group membership.

[curullij]

Is your AD domain .local?

[Lincoln]

yes our domain is .local

[plawrence]

@Lincoln

Are you able to post the command you were using when you were trying to bind manually? I bound a Mavericks machine to AD today using the following:

dsconfigad -add domain.name -username administrator

This doesn't configure any of the other parameters, but its somewhere to start. I'd like to think that if this didnt work you would get a more helpful error message.

[Lincoln]

By manually I mean by resorting to using the GUI. That said I just took a shot with:

dsconfigad -add mydomain.local -username myUserName

dsconfigad -add mydomain.local -username administrator

dsconfigad -add mydomain.local -username administrator@mydomain.local

all of these returned 'Invalid credentials supplied for binding to the server'

I tried each option a couple of times to make sure I wasn't mistyping passwords with the same results. Both my account and the administrator account have the necessary permissions for binding.

I have just confirmed that this works on an identical machine running 10.7.5.

[plawrence]

@Lincoln

That definitely sounds odd. Have you checked that the time on the client and server match? My only other suggestion is to crank up the DirectoryService logging on the client (http://support.apple.com/kb/HT4696), try binding again and look through /var/log/opendirectoryd.log and see if there is more information about the error there. I did that on a client here and noticed that there is a bunch of Kerberos stuff before I see the 'Invalid Credentials' error (when typing in a wrong bind password intentionally) so perhaps there is some Kerberos weirdness going on?

Is this a completely fresh install of Mavericks? or are there other items included in your build?

[Lincoln]

Yes time matches. Debug logging enabled and found the following (sanitised) which looks to be the root of the issue. My boss is wondering whether we may have some kind of DNS issue at present because of another issue he's having and wonders if 'configuration file for realm myDomain.LOCAL not found' may be related. Further down the errors certainly seem to relate to kerberos.

If anyone has any brilliant ideas I'd love to hear them. I am heading off for a long weekend now given that it is Melbourne Cup weekend. I won't be back on deck till Wednesday next week.

Cheers

Lincoln

2013-11-01 15:50:06.991564 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - Bind Step 1 - Searching for Forest/Domain information - 'myDomain.local'
2013-11-01 15:50:06.991778 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - Removing Kerberos reachability info 'Kerberos:myDomain.LOCAL'
2013-11-01 15:50:06.991865 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - Binding using 'myUserName@myDomain.LOCAL' for kerberos ID
2013-11-01 15:50:06.992010 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - new kerberos credential cache 'MEMORY:0x7fc178cb72f0' for 'myUserName@myDomain.LOCAL'
2013-11-01 15:50:06.992052 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: loop 1
2013-11-01 15:50:06.992062 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send 0 patypes
2013-11-01 15:50:06.992076 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - fast disabled, not doing any fast wrapping
2013-11-01 15:50:06.992115 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - Trying to find service kdc for realm myDomain.LOCAL flags 0
2013-11-01 15:50:06.992371 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - configuration file for realm myDomain.LOCAL not found
2013-11-01 15:50:06.992691 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - submissing new requests to new host
2013-11-01 15:50:07.402117 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - connecting to host: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00000001
2013-11-01 15:50:07.402190 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - writing packet: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00000001
2013-11-01 15:50:07.403322 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - reading packet: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00000001
2013-11-01 15:50:07.403366 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - host completed: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00000001
2013-11-01 15:50:07.403411 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_sendto_context myDomain.LOCAL done: 0 hosts 1 packets 1 wc: 0.411298 nr: 0.409322 kh: 0.000558 tid: 00000001
2013-11-01 15:50:07.403449 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: loop 2
2013-11-01 15:50:07.403463 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: processing input
2013-11-01 15:50:07.403482 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: got an KRB-ERROR from KDC
2013-11-01 15:50:07.403513 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: KRB-ERROR -1765328359/Additional pre-authentication required
2013-11-01 15:50:07.403535 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send 4 patypes
2013-11-01 15:50:07.403549 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send PA-DATA type: 19
2013-11-01 15:50:07.403562 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send PA-DATA type: 2
2013-11-01 15:50:07.403575 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send PA-DATA type: 16
2013-11-01 15:50:07.403588 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - KDC send PA-DATA type: 15
2013-11-01 15:50:07.403611 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: using ENC-TS with enctype 18
2013-11-01 15:50:07.403625 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: using default_s2k_func
2013-11-01 15:50:07.410049 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - fast disabled, not doing any fast wrapping
2013-11-01 15:50:07.410099 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - Trying to find service kdc for realm myDomain.LOCAL flags 0
2013-11-01 15:50:07.410540 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - configuration file for realm myDomain.LOCAL not found
2013-11-01 15:50:07.410849 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - submissing new requests to new host
2013-11-01 15:50:07.799250 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - connecting to host: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00010001
2013-11-01 15:50:07.799327 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - writing packet: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00010001
2013-11-01 15:50:07.801387 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - reading packet: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00010001
2013-11-01 15:50:07.801433 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - host completed: udp 10.170.224.90:kerberos (hccstu.myDomain.local) tid: 00010001
2013-11-01 15:50:07.801475 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_sendto_context myDomain.LOCAL done: 0 hosts 1 packets 1 wc: 0.391385 nr: 0.388300 kh: 0.000728 tid: 00010001
2013-11-01 15:50:07.801511 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: loop 3
2013-11-01 15:50:07.801525 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: processing input
2013-11-01 15:50:07.801543 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: got an KRB-ERROR from KDC
2013-11-01 15:50:07.801577 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - krb5_get_init_creds: KRB-ERROR -1765328360/Preauthentication failed
2013-11-01 15:50:07.801591 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - krb5_credential - FAST disabled and got preauth failed
2013-11-01 15:50:07.801625 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - Invalid credentials
2013-11-01 15:50:07.801805 EST - 8534.19356, Node: /Active Directory, Module: ActiveDirectory - ODNodeCustomCall failed with error 'Invalid credentials' (5000)
2013-11-01 15:50:07.805101 EST - 8534 - Client: 'dsconfigad', exited with 0 session(s), 2 node(s) and 0 active request(s)

[PeterClarke]

Hi, I've had similar problems.

I use a Bind Script, to bind to a Read-Only AD-domain controller
-- this works brilliantly in OS X 10.8.5.

But in Mavericks it comes up with an error.

Previously in OS X 10.6.8, I use a bind script, to bind to a Read-Write AD-Domain controller
-- I find that if I use the same credentials, as the 10.6.8 script, then I can bind Mavericks to the AD.

So it seems, that it works - provided that I no longer try to bind to a Read-Only domain controller.

In 10.8.x, the Mac was able to write to the Read-Only domain controller !!
-- Or whatever actually happened the effect was equivalent to that.
(I think it actually searched outside the 'preferred' controller to create the bind, but then used the preferred controller for read operations.

I need to discuss this with our AD admin, but she is on holiday at the moment - so waiting for her to get back.

It's possible that under 10.9, the Mac may behave more like a PC - our PC's have to have their computer records created manually - before they are able to use the Read-Only domain controller.

Where as our Macs were able to create their own computer records in the AD.
- even in the Read-Only controller when using OS X 10.8.x

In OS X 10.9, this no longer seems to be the case.
-- I have not got to the bottom of the mystery yet,
-- But it's clear that Apple has made some change too the way that the AD-Plugin operates..

I tried doing a manual AD Bind, in order to eliminate any script issues
- and that didn't work either with Mavericks and the Read-Only credentials.
- The error dialogue said: "The plugin encountered an error processing request."

Where as using the read-write (that we earlier used for OS X 10.6, did work..

So there are definitely a few clues there..

[nessts]

I have to say in my environment(s) binding to AD has been much faster than with anything before. Literally I am binding in under 30 seconds now where it used to be over a minute. I have my own bind script that i have been using since 10.6 days.

[PeterClarke]

Yes I am using my own bind scripts too. - They predate our use of Casper.
My OS X 10.6.x AD bind script was fairly fast, I think about 15-20 seconds ?

We skipped OS X 10.7 altogether.

My OS X 10.8.x bind script to AD Read Only Domain Controller (AD-RODC) took about 60 seconds.
- which I put down to it needing to hunt around to find a writable system..

Now it remains to be seen what I'll need to do for 10.9.x (since the 10.8 script clearly does not work)
- Neither does a Manual AD-Bind attempt using 10.8 working credentials. So I know it's not a 'script' issue..

[Lincoln]

OK so I got it ... forest for the trees kinda thing.

I don't like making myself look stupid but it might help someone else so ...

IPv6 was enabled in my Mavericks image! Since 10.7 I have as standard had IPv6 disabled in my images because it causes issues with AD binding. I can't believe I didn't think of it earlier. When I created my Mavericks image I didn't disable IPv6 and so I couldn't bind to AD. Of course you can't disable IPv6 from the GUI so it has to be done from the command line or from a script. I have a script that does some network related stuff at imaging time so I just added this line to the script:

networksetup -setv6off Ethernet

Problem solved. Machines bind as expected. :)

Hope this helps someone.

Cheers

Lincoln

[andyinindy]

Nice, glad you figured it out, Lincoln! We will be sure to check on IPV6 in our Mavericks image, as do not want to deal with AD issues.

BTW, you mark your post from 11/6 as the "answer" ;)

[dapopeah]

I had a problem with this as well, my solution was different.
I was not putting the full path for the OU/CN. If you have an OU that you've created in the AD domain, let's say it's JAMF-Computers, and it appears in the AD directory as ad.domainname.type(org/com/edu/whatever) the binding directory will look like the following. ad.domainname.type
>Computers (CN)
>Windows-Machines (OU)
>JAMF-Computers
>>TheComputerYourBinding

the entry in the OU field of the JSS would show reverse order: OU=JAMF-Computers,DC=ad,DC=domainname,DC=type
If you have nested OUs, you need to prefix the OU entry with each nest, closer to the object you're binding.

if you wanted to put it in the Computers container it would look like: CN=Computers,DC=ad,DC=domainname,DC=type

Don't know if it's a container or an OU? Look at the icon, plain folder icon is a container, folder with a computer in it is an OU.

[calumhunter]

hey @PeterClarke were you able to specify a RW domain controller to bind to with 10.9?
I'm facing the same issue, but the IPv6 solution isn't working for me.
I can bind find with 10.8.5, but on 10.9 with the exact same arguments it fails. I believe it is because 10.9 is just grabbing the nearest DC which is RO and can not create a computer record and then fails. I can't seem to find a way to tell dsconfigad on 10.9 to bind to a certain DC

[ClassicII]

@calumhunter

Have you tried to bind through the GUI and under the advanced options then Administrative click "prefer this domain server"? We have tested this and it should lock the machine down to the DC that you specify. I am guessing this is an option on the command line too but don't know the options.

[calumhunter]

Thanks @ClassicII, but yes I have actually got 2x machines right next to each other, one with 10.8.5 and one with 10.9. the 10.8.5 works fine, the 10.9 fails.
The prefer this domain server, doesn't actually specify which DC you bind to, it just prefers a particular DC for authentication/contacts, this option actually gets set after a bind is done.
I have even tried using Centrify Express, which does allow you to specify the DC it binds to. But this also fails but with a different error, it fails with something like unexpected configuration size. It has to be something with our environment, but also something that Apple has changed in 10.9. 10.10 beta also exhibits the same issue. Good old trusty 10.8.5 though works great. I should be used to the break fix break fix break fix nature of AD plugin in OS X releases by now lol

[dlondon]

Hi Calum,

When you say 10.9 do you really mean 10.9 and not 10.9.4?

Regards,

David

[calumhunter]

Sorry i wasn't being very clear there, yes I mean 10.9.4.

[dwhitehead]

@calumhunter Are you by chance running a split domain, or a domain with a .local suffix?

[PeterClarke]

Yes I am able to link to a specified AD-Controller.

With Mavericks (10.9.2, 10.9.3, 10.9.4) all tested.
I need to use a Read/Write AD-Domain controller.

If I try a Read-Only domain controller, it won't work
(Unless the Machine record already exists, in which case I can rejoin to it)

I do see frequent time-out errors in (just a few tests) about 25% of the time.
I dealt with that by retrying, and so far 100% of cases covered, in just two attempts.
- I am using my own AD-Bind script to do this.

Issues are:
1) Client must use "ntp" to time-sync with AD-server (kerburos requirement of within 5 minutes
2) Due to our Network Firewall, AD-Bind only works over Cable-connected ethernet not wireless
3) Have to Retry some Bind attempts
4) Have to use a Read-Write AD-Domain Controller with Mavericks (Where as 10.8 could, due to auto-search, use a read-only DC) (10.8 would find a writable, write to that, then switch to the RODC) 10.9 won't do that.
5) The AD-Bind, to R-W Controller has to happen from inside a login-session (At Re-Imaging I use a auto-login account to do the AD-Bind, and then logout, disabling the autologin)

6) Your AD-Admin, if using a Firewall on the Domain Controller, has to allow the client to connect! -- either by the individual IP, or via the Subnet.
7) You have to use a Unique AD-Name of Max 15-Characters.

8) Always try to do a Manual AD-Bind as part of the debugging process. - If a Manual AD-Bind won't work, then an automated one won't work either. - In this case, you need to investigate what's wrong. -- Usually some detail is incorrect, of a firewall is blocking the connection.

You can test that out by: from Terminal do: telnet YourServerIPAddress 389 -- you should be able to get a connection, otherwise a firewall is blocking you -- or you have the wrong server IP address, or the service is not running..

9) Talk to your AD-Domain admin, and check / ask about any other requirements.

Once you have Manual AD-Binding working, then you should be able to automate the process.

It is fiddly, and not completely straight forward..

[calumhunter]

thanks @PeterClarke but im curious HOW exactly you were able to get the machine to look to the specific RW controller? or was this done through AD sites? I cant find any switches to the dsconfigad command that allows specifying a particular server for binding

no problems with all the other things you listed

@dwhitehead - not .local, possibly split domain, we have different domains on the same flat network i think

[dwhitehead]

@calumhunter You may be able to get some additional insight by setting the open directory log settings to debug (odutil set log debug) and then attempting to bind. Output is written to /var/log/opendirectoryd.log.

To my knowledge, there's no way to 'force' a bind with a specific DC, however you are able to set preferred servers with the '-preferred' flag in dsconfigad (also present in the Casper Directory Bindings config under the administrative tab).

It's also a good idea to check what your DNS is returning for service queries (especially if you're on a split domain). Details here: http://support.apple.com/kb/HT3394

[PeterClarke]

I just specify the preferred controller.

Here is a script fragment:

AD_Binder_R1=$(dsconfigad -add $FQDN
-username $RWD_BindAccount -password $RWD_Password
-computer ${MyComputer} -ou ${MyOU}
-preferred "${MyPreferred}" -force
?packetencrypt require 2>&1)

Where the variables are what you would generally expect.
ie: ${MyPreferred} contains the DNS-Name of the Preferred RW Domain Controller.
$FQDN contains the AD-Domain Name.

[calumhunter]

Thanks @dwhitehead, but we have diagnosed the problem as 10.9.x has different behaviour to 10.8.5 when it comes to RODC's. The service records are fine. The AD environment is working correctly, I have also used the Centrify Express AD Diag tool to double check, everything passes fine.

@PeterClarke, Interesting it is my understanding that the preferred option only sets the preferred server for search/contacts not for binding. Setting that option was one of the first things i tried, but no luck.

Are you sure that if you remove that preferred option you can not bind with 10.9? Perhaps your being lucky and dsconfigad is picking a RW AD server from the DNS pool randomly, that would also explain why you need multiple attempts in order to bind successfully perhaps

[PeterClarke]

Well the unix 'man' page on dsconfigad, says, for the preferred option:

-preferred server Use the specified server for all Directory lookups and authentications. If the server is no longer available, it will fail-over to other servers.

It seems to work for us, but having got our ADS-Bind operational, I didn't experiment with all possible config options, to see what effect they had.

[eohlin]

We were having issues on several Macs that we had upgraded from OS X 10.6/10.7 to 10.9. Each of these machines had a previously used Computer Name in AD. We found that creating a new computer name node in AD (and using Capser to rename the Mac to that new name) allowed the Mac to bind with no issues.

[maloyd7847]

I just spent a bit wrestling with this myself. What I found that was throwing the error for me was a too-long name for the machine in AD. I was using a 21-character machine name, when I lowered that to 18 characters I was able to join without error.