Sierra 10.12.5 Upgrade breaking Corp Wifi

nwiseman
Contributor

With the release of 10.12.5 today, I immediately upgraded one of my test systems. Everything was working fine on said system before the upgrade. After the upgrade Wifi fails to connect. Have attempted to re-configure wifi in order to see if it just corrupted, but no dice.

I've opened a support ticket with Apple, but I'm curious if anyone else is seeing something similar.

87 REPLIES 87

chisox1
New Contributor

Our official response from Apple was as follows
"You can think of the authentication process as a two step process; first it will check to see if the certificates are in the keychain and trusted, then it will check to see if trust has been anchored in the Wi-Fi profile. If it meets these two criteria it will connect.
If trust isn’t anchored in the profile then it will check to see if there is a trusted server certificate name in the profile. If there is, and it is the domain used to sign the certificates, then the machine will successfully authenticate."

@bbot Our current setup is deploying AD certs through the config profile. We also have a Root Cert profile. In my very limited testing so far, having a profile that deploys all certs and having a duplicate "trust" certificate in the wireless profile has not broken anything. We are pushing the profiles via MDM.

dpratl
Contributor II

Hi all,

We encounter the same problem, no connection to our WiFi possible.
The WiFi network is cert based.

What seems to be working: (suggested by @nwiseman )
Set in the Trust Section of that certificate EAP to "Always trust"
8c596a439e6048628a277a5551fda45b

I tried to handle that inside the Configuration Profile, but that wasn't working at all (it even disabled the Workaround). What am I doing wrong here?
6a6935a638f144448df5dc23fb63b86a

Thank you very much
BR
Daniel

jason_d
New Contributor III

@dpratl I deploy my root and intermediate certs as a part of the same profile and they show up under Trusted Certificates and allow me to check them as trusted.
8a1fa3fc1cd548d5b0d33c5098a17bda

bbot
Contributor

Still running into some issues here. On my test machines, I am unable to connect to WiFi completely after checking the Trust box for the intermediate certificate.1e57a8c3b9254bcc86dadbf4c9fc7ac9

Can someone review my config? This is getting installed on the computer level. AD certificate is requesting for a machine certificate to authenticate. Certificates are requesting properly, it creates the com.apple.network.eap.user.identity.wlan.ssid in keychain and assigns the appropriate certificate.

On 10.12.5, I can have the user re-install the current configuration profile (without the intermediate box trust checked), and it'll re-connect to wifi fine. It's just that the wifi disconnects after updating to 10.12.5.

I also have a separate configuration profile that deploys just the root + intermediate certificates to the system keychain. (these have been on the Mac prior to the wifi config profile being deployed)

jason_d
New Contributor III

@bbot you need to trust the root and sub, also I'm trusting by exact cert name where you have 'root' and 'sub' and it's working here.

bbot
Contributor

No dice. I've tried trusting both certificates, and adding the common names into the trust.
I've tried going into keychain, right clicking the certificates and setting the trust to "Always Trust" and it still won't connect to wifi... so I'm not sure if the issue is with the intermediate and root.

dgreening
Valued Contributor II

We trust both the root and the sub (which are included in the profile) in the Network payload and it has been working fine for us on our 10.12.5 testers.

87924bb5e7144fc1b250a113bf0a527e

jason_d
New Contributor III

@bbot are the certs getting installed and marked as trusted?

bbot
Contributor

@jason.desveaux the root is marked as Always Trust all the way down the list. The Intermediate is marked as valid, trust is set to use system defaults and no value specified. This is after marking them both as trusted in the configuration profile.

The messed up part is even after manually trusting the intermediate, it still won't connect.

The only way I can get it to connect to wifi is if none of the boxes are checked in the configuration profile.

jason_d
New Contributor III

@bbot strange... is your identity preference in the keychain pointing at a trusted cert?
7aebe0f6d0d84fc09cf70173c818a288

bbot
Contributor

769c7c071c5a470ca51611bc7c026c58

4f7ddfa519da4eca85a3252f8f5ccf9e

Looks fine to me. I have both system and login keychain.

jason_d
New Contributor III

@bbot it all looks good, any clues on your NPS server?

bbot
Contributor

We use Cisco ISE. Looking at the logs, it doesn't look like it's presenting a certificate. Not too helpful here...

Event 5400 Authentication failed Failure Reason 12521 EAP-TLS failed SSL/TLS handshake after a client alert  12521 EAP-TLS failed SSL/TLS handshake after a client alert  12507 EAP-TLS authentication failed  11504 Prepared EAP-Failure  11003 Returned RADIUS Access-Reject

Here's a successful one
12504 Extracted EAP-Response containing EAP-TLS challenge-response  
12571 ISE will continue to CRL verification if it is configured for specific CA - certificate for ############
 
12571 ISE will continue to CRL verification if it is configured for specific CA - certificate for #######
 
12811 Extracted TLS Certificate message containing client certificate  
12812 Extracted TLS ClientKeyExchange message  
12813 Extracted TLS CertificateVerify message  
12804 Extracted TLS Finished message

MatG
Contributor III

Same issue fo some of our users.

The solution so far has been, open Keychain, find the correct cert and select Trust Always, then manually reconnect to the Wifi using Join Other Network option and add int the correct details of

Network Name
WPA2 Enterprise
EAP-TLS
Select the cert just changed updated Username

Then authenticate as admin and you are good to go.

Nix4Life
Valued Contributor

We pre-populate our certs into our profile. machine/login window. Just upgraded to 10.12.5. No wi-fi issues

AVmcclint
Honored Contributor

I have built an entirely new Config Profile to test with 10.12.5.
1. General: basic name and description. Install Automatically. Computer Level.
2. Network: a. Ethernet. TLS. Trust: our 5 internal root and issuing and proxy certs. Allow Trust Exceptions
2b. WiFi, our SSID (not hidden), Auto Join. Proxy setup Automatic with the URL of our pac settings. WPA/WPA2 Enterprise. TLS. Trust is the same as ethernet. Allow Trust Exceptions
3. Certificate: I uploaded our 5 internal root and issuing and proxy certs
4. AD Certificate: basic name. our cert server address, Certificate Authority name, Cert template name, 15 days expiration notification. Allow access to all applicaitons.

On my test 10.12.5 Mac, I removed the computer from the separate already-existing scopes for 802.1x and certificates. I went into Keychain Access and cleaned up any remnants of com.apple.network.eap and any certs manually added. Rebooted and plugged it into a non-802.1x port. Then I added the computer to this new single config profile and let it push. After confirming that the profile pushed and the certs are all present, I plugged into an 802.1x port and rebooted. It appears to have worked on ethernet. I even crated a test Login trigger policy to say "hello" and it worked. After getting to the desktop I verified that Ethernet has a valid IP and shows 802.1x connected. However, it absolutely will not connect to WiFi. Even if I manually click Disconnect and then Connect in System Prefs > Network. The only error I get is the standard "The Wi-Fi network could not be joined." I can find no errors in Console relating to 802.1x or our SSID. Could this be the still troublesome JAMF/Apple bug with 802.1x?

I have achieved partial success using these steps. Maybe you'll have better luck with your network.

bbot
Contributor

Still no luck here. I'm tried every possible combinations of checking the trust for sub, root, root and sub, typing in the certificate common name of the sub and root and wifi won't connect at all with those settings.

My original configuration profile without checking the box to trust any of the certs or typing the certificate common name allows our users to re-connect 100%. The only issue is that when user's update to OS X 10.12.5, they get disconnected from wifi and need to re-run my bash script that installs the profile from Self Service to get re-connected.

At this point, I'm thinking about just sending out communication to users who haven't updated and give them a heads up that they'll need to re-run the "connect to wifi" policy in Self Service.

bbot
Contributor

Did some more testing. After updating to 10.12.5, i found that the com.apple.network.eap.user.identity.ssid.###### was completely missing on the user's login keychain. The com.apple.network.eap.system.identity.ssid.##### in the system keychain is there and has the correct certificate, but wifi wouldn't connect with just this.

After deleting the wifi profile from network preferences, re-connecting by choosing EAP-TLS and choosing the AD machine cert, it'll reconnect fine.

Retrac
Contributor

We got our Wifi cert working yesterday after trial and error. We can now use it with JSS 9.98 and 10.11 > 10.12.5.

Additional config added to the Configuration Profile included the certificate section, and then uploading the certificate we had configured in the AD Certificate section already, using the full file name including the .cer extension as the display name. Saved the profile and then went into the Network section of the Configuration Profile, clicked the trust tab half way down next to protocols and ticked the box.

Our configuration uses WPA/WPA Enterprise, TLS and a hidden SSID.

dprakash
New Contributor III

think my issue relates to this

https://www.reddit.com/r/sysadmin/comments/52mz6r/apple_sierra_breaks_ssl_w_blank_subject/

I can confirm that I changed the certificate template to include dns name instead of common name as the subject name. It now properly populates the subject name, and macOS Sierra clients are happy.

dprakash
New Contributor III

If i request a cert without a Common Name and trust it manually it works.

  1. Remove your computer from any 802.1x config profile scopes.
  2. Open Keychain Access
  3. From the menu select Keychain Access > Certificate Assistant > Request a Certificate From a Certificate Authority.
  4. User Email Address: your email address
    Common Name: Leave Blank
    CA Email Address: Leave Blank
    Request is: Choose 'saved to disk'

  5. A file named 'CertificateSigningRequest.certSigningRequest' should be generated

open that file with text edit

copy all of the text inside the file (including the ---beginning and ----end parts)

  1. Logon to your Certificate Authority and select Request a certificate

  2. click on 'advanced certificate request.'

  3. Paste in the copied text into the 'Saved request:' box

  4. Choose your Certificate template from the drop down box and click Submit

  5. on the next page Download the certificate

  6. Double click the certificate and it should be added to your login keychain

  7. in the certificate's properties make sure EAP is set to Always trust.

  8. Connect to your Corporate Wifi and select your certificate

any prompts that appear make sure to choose always allow and allow

AVmcclint
Honored Contributor

@dprakash That is an interesting tidbit into this whole mess, but it is extremely labor intensive and not at all practical. :-

dprakash
New Contributor III

@AVmcclint thats the entire process certificate retrieval manually and if someone's desperate it may work

donmontalvo
Esteemed Contributor III

Just tested. We are using 802.1x Wi-Fi device certs, so far connections are fine. #knocksOnParticleBoard

We have some other team members testing as well, in LAB and field, to see if this is going to impact us.

--
https://donmontalvo.com

ClassicII
Contributor III

Wifi and ethernet are no longer connecting. Have a case going :(

bbot
Contributor

What are the benefits of having a wifi configuration profile installed at the computer level vs the user level?

nkalister
Valued Contributor

@AVmcclint that's also entirely scriptable! I do all my certificate provisioning with that workflow, scripted. you can still use profiles with the certs to configure wifi, too.

nkalister
Valued Contributor

@bbot machine wifi configurations can stay connected even when a user is not logged in. user configurations will log out of wifi if a user is not logged in.

bbot
Contributor

I've found with computer configuration, the identity preference "com.apple.network.eap.system.identity.wlan.ssid.__ is created and assigned the appropriate certificate, but machines won't connect to wifi.

If I have a "com.apple.network.eap.user.identity.wlan.ssid._" then it'll work. Any idea why the configuration profile deployed via Self Service creates this entry, but pushing it through a policy does not?

I'm deploying through a script that utilizes "profiles -I -F"

ClassicII
Contributor III

@bbot

I think you are on to something. For my system level wired and wifi after upgrading to 10.12.5 the profiles seem to now be looking for a USER level keychain item and NOT the system level keychain like they should.

Here are the errors that are telling us that.

<WiFiAgent[880]> START USER MODE 802.1X USING KEYCHAIN request received

[CWWiFiClientDelegate __startUserMode8021XUsingKeychainWithSSID:interfaceWithName:error:]: Failed to find 802.1X credentials in the user keychain, returned error code -25300

-[CWXPCSubsystem associateToUserMode8021XWiFiNetworkUsingKeychain:remember:interfaceName:authorization:connection:error:]: User Mode 802.1X failed for network (WIFISSIDNAMEHERE) on interface en0, returned error Error Domain=com.apple.coreWLAN.EAPOL.error Code=-25300 "(null)"

bbot
Contributor

Having the com.apple.network.eap.user.identity.wlan.ssid in either login or system keychain appears to work in my case. I've been trying to find ways to get this value created by running a bash script.

Hopefully someone can find what I'm doing useful and add to it. This is what I used when I first implemented 802.1x wifi a few weeks ago using machine AD certs. I pushed this script to all machines and this worked with over 90% success rate in an environment with 600 machines. The same script continues to work when deployed via Self Service 100% of the time. This works in 10.12.5, 10.11.6 and 10.10.5.

However, If I re-push the same script through a policy at recurring check-in, it'll cause the com.apple.network.eap.user.identity.wlan.ssid identity to lose the certificate and thus causing wifi to not connect. I need to be able to re-push the script for certificate renewal purposes. I commented out some of the commands I've tried to get it to re-assign the machine certificate to no avail. The two "security" commands not commented out is what I'm currently using that works in Self Service and first deployment.

If I delete all the profiles, and the identity preference in keychain, then re-push the script. It'll work again.

The config profile I use is as follows:
Computer level / install automatically
1 root and 1 intermediate certificate
1 AD certificate request (machine cert)
1 WiFi payload that uses EAP-TLS. No trust boxes are checked, no common name entered. Select to use AD Certificate (these creates and assigns com.apple.network.eap.system.identity.wlan.ssid into the system keychain. However this isn't enough for it to connect)

#!/bin/sh

#  Wifi Self Service.sh
#  Script to connect to LC-#
#  Brandon Wong 4/24/2017

function connectWifi (){
sudo /usr/bin/profiles -I -F /Library/LC/LC-#.mobileconfig #-U $currUser
    Cert="$compname.#"
    #su "$currUser" -c "security set-identity-preference -n -s com.apple.network.eap.user.identity.wlan.ssid.LC-#"
    #su "$currUser" -c "security set-identity-preference -c $Cert -s com.apple.network.eap.user.identity.wlan.ssid.LC-# /Library/Keychains/System.keychain"
    #security set-identity-preference -n -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#" /Users/$currUser/Library/Keychains/login.keychain
    #security set-identity-preference -n -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#" /Users/$currUser/Library/Keychains/login.keychain-db
    security set-identity-preference -n -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#"
    #security set-identity-preference -c "$Cert" -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#" /Users/$currUser/Library/Keychains/login.keychain /Library/Keychains/System.keychain
    #security set-identity-preference -c "$Cert" -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#" /Users/$currUser/Library/Keychains/login.keychain-db /Library/Keychains/System.keychain
    security set-identity-preference -c "$Cert" -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#" /Library/Keychains/System.keychain
wservice=`/usr/sbin/networksetup -listallnetworkservices | grep -Ei '(Wi-Fi|AirPort)'`
whwport=`networksetup -listallhardwareports | awk "/$wservice/,/Ethernet Address/" | awk 'NR==2' | cut -d " " -f 2`
hwports=`networksetup -listallhardwareports | awk '/Hardware Port: Wi-Fi/,/Ethernet/' | awk 'NR==2' | cut -d " " -f 2`
wirelessnw=`networksetup -getairportnetwork $hwports | cut -d " " -f 4`
    sleep 10
    if [[ $wirelessnw == "LC-#" ]]; then
    echo "Connected to LC-#"
    else
    echo "Machine is not connected to LC-Authorized. Checking identity preference"
    security get-identity-preference -s "com.apple.network.eap.user.identity.wlan.ssid.LC-#"
exit 1
    fi
}

function bindToDomain (){
#removed bind to domain script to shorten in jamf nation

    dependenciesCheck

}

function dependenciesCheck(){

# Check for an AD computer object
ad_computer_name=`dsconfigad -show | grep "Computer Account" | awk '{print $4}'`
compObj=`dscl /Active Directory/CORP/All Domains read /Computers/$ad_computer_name RecordName | awk '{print $2}'`

if [ $ad_computer_name = $compObj ]; then
    echo Matching AD object exists. Continue script...

    # Make sure mobileconfig exists
    file=/Library/LC/LC-#.mobileconfig

        if [ -f "$file" ]; then
            connectWifi
                else
                echo Profile missing. Downloading profile
                jamf policy -trigger LCAuthProfile
                    if [ -f "$file" ]; then
                    connectWifi
                        else
                        echo Cannot connect to LC-#. Mobileconfig is missing.
                        exit 1
        fi
                    fi
else    
    echo "Missing Domain Binding"
    bindToDomain
fi

}

# HARDCODED VALUES ARE SET HERE
Pass=""

# CHECK TO SEE IF VALUES WERE PASSED FOR $4, AND IF SO, ASSIGN THEM
if [ "$4" != "" ] && [ "$Pass" == "" ]; then 
Pass=$4
fi

# Check to make sure Pass variable was passed down from Casper
if [ "$Pass" == "" ]; then 
echo "Error: The parameter 'Pass' is blank. Please specify a value." 
exit 1 
fi

# Variables
currUser=`stat -f "%Su" /dev/console`
echo "Current user is $currUser"

    # Cleanup duplicate machine certificates
    compname=`dsconfigad -show | grep "Computer Account" | awk '{print $4}' | sed 's/.$//'`
    Certs="$compname.##"
    hashes=$(security find-certificate -c "$Certs" -a -Z|grep SHA-1|awk '{ print $NF }')
        for hash in $hashes; do
        echo deleting duplicate certs $hash
        sudo security delete-certificate -Z $hash
        done

# Check dependencies 
    dependenciesCheck

exit 0

perrycj
Contributor III

@bbot are you on the macadmins slack channel by chance? If so, let me know and we can talk there.

bbot
Contributor

@perrycj I'm not. Fairly new to the Mac scene. Can you send me where I can sign up?

perrycj
Contributor III

Just go to slack.com and download the client for your mac, or whatever device you're using. Then sign in to the macadmins.slack.com team. You'll need to make a username, etc and then you're in. It's free.

bbot
Contributor

@perrycj I'm in there under the username @brandobot

juttmartin
New Contributor II

Just to note our issue was the DC certificates used for client authentication for 802.1x. Looks like after this update our clients are no longer able to validate our DC certificates even though our CA certificates are trusted. We are using certificate base authentication for all our systems. I'm going to do another test and push our DC certificates out through our configuration profile.

juttmartin
New Contributor II

After testing it looks like pushing these certs out through a configuration profile are not being trusted. So I just packaged them up to install them that way. Looks to be working fine authenticating through 802.1x. I'm just going to create a policy to publish out this package install. Here is a the the security command that can be used to import and trust the certs:

security add-trusted-cert -d -r trustAsRoot -k /Library/Keychains/System.keychain /tmp/PATH_TO_CERTIFICATE

MatG
Contributor III

The reply I got from Apple

To resolve this on your end, make sure that you anchor trust to the appropriate Intermediate or Root certificate in the Network payload or your configuration profile. In the profile, this will create a key for “PayloadCertificateAnchorUUID".

AVmcclint
Honored Contributor

@MatG What exactly does "anchor trust" mean and how do we do that in the context of using JAMF Pro?

MatG
Contributor III

I was posting to hope someone else would answer that!