Thunderbolt Enet adapters and Casper Imaging

znilsson
Contributor II

So we have set up a number of USB 3 flash drives with a basic Mavericks install and Casper Imaging set to auto-run. We use the flash drives to boot a new out of the box Mac, Casper Imaging auto-runs and we image the new Macs that way, which has been working great on the new Mac Pros which have actual Gigabit Ethernet ports.

When we get something like a Retina Macbook Pro or a Macbook Air that don't have ethernet ports, we've been trying to use Thunderbolt to Gigabit Ethernet adapters. The issue is that when we boot from our Casper flash drives, Casper Imaging fails with an error saying it couldn't find the JSS, and this is despite the Mac actually connecting through the adapter and pulling a valid IP address from the router. That's the really confusing part, and I'm not sure what the problem is. Even after we have a valid IP if we try to run Casper Imaging we get the same communication error.

But the same flash drives booting Mac Pros that are connected to the same physical network cables on the same network don't have any problem connecting to the JSS and imaging.

Is there a known problem with trying to use casper imaging over a connection that's going through one of these adapters? We have not yet entered the MAC addresses of these adapters into the MAC address exclusion list in Casper, we're doing that tomorrow - but is that the problem, or is it something else?

Thanks for any help.

29 REPLIES 29

pblake
Contributor III

Are there any weird DNS or search suffix showing up on the thunderbolt to Ethernet adapater. The network settings are unique for each device.

were_wulff
Valued Contributor II

@znilsson

This actually sounds like it falls pretty in line with an open defect we have that only affects Thunderbolt Ethernet Adapters.

The defect ID is D-006627, and it is still listed as open.

When we run into it, we can see any (or all) of the following behavior, and it can vary from machine to machine:

- The device shows up in inventory in JSS.

- If the computer was previously enrolled, the machine name in inventory is updated to it's newly imaged name in the JSS - suggesting successful JSS communication. There is not successful JSS communication at this point, however.

- The computer responds to mdm commands - proving that MDM profile is actually installed, suggesting successful JSS communication. There is still not truly successful JSS communication at this point, due to the defect.

- The computer does NOT actually enroll - imaging configuration management account are -not- present.

- Local machine's jamf.log will display Connection Failure: "The host (JSS SERVER) is not accessible".

Sometimes, if we let the computer sit for 5-10 minutes, it will eventually grab a connection that the jamf binary recognizes and the imaging process & enrollment will wrap up, but that's not a given; sometimes it just doesn't happen at all, which is part of the defect behavior.

Currently, there are a couple of workarounds that we have:

1) Use a USB Ethernet adapter or a different method of imaging.

2) Create a custom imaging config and checking the 'Allow Setup assistant to run' will ensure that the user is able to create another account (as opposed to relying solely on the enrollment process to create the JSS management account configured in the Imaging Configuration settings) at the time of imaging. When we run through the setup assistant, at the Network step, we can select Other, then select the Thunderbolt Ethernet adapter, and it will grab a valid network connection. This may not be feasible if we have a lot of computers that need reimaging/imaging, though, due to the extra legwork and time involved.

If you haven't already, I'd recommend opening up a case with your Technical Account Manager so we can get the issue tracked and attached to the defect; they may also know your environment a bit better than I do and may be able to tweak the workaround steps we have to something that might work out a bit better for your specific setup.

Thanks!
Amanda Wulff
JAMF Software Support

mikeh
Contributor II

I've seen this problem once, using one Thunderbolt Ethernet adapter on a 2014 MBAir, but a few days later, using a different 2014 MBAir, using a different Thunderbolt Ethernet adapter, I didn't have any problems. I had thought at the time that I either had a faulty adapter or a faulty Ethernet switch.

One thought is that, over the past couple days, Apple has released a couple firmware updates for the last couple generations of MBAir, with the EFI update dealing with issues "when booting while certain USB and Thunderbolt devices are connected." Maybe try the update and see if that helps at all?

References:
MBAir EFI Firmware Update 2.8 for mid-2013 and early 2014 models
http://support.apple.com/kb/DL1749

MBAir SMC Firmware Update v2.0 for mid-2013 models
http://support.apple.com/kb/DL1748

znilsson
Contributor II

Unfortunately right now I have a small pile of rMBPs that I need to image, and neither the Thunderbolt to Gigabit Ethernet adapter nor the USB to Ethernet adapter are working. And I've added all the MAC addresses to the Casper exception list. And also unfortunately, this is contrary to Amanda's advice above regarding the USB to ethernet adapter because that's not working for me either. And I've verified that everything is correct, it's pulling a good IP address, DNS info is all good... there is nothing I can find that would explain why it's not working.

edit - From my casper install flash drive, I can also ping the JSS and I can even traceroute it, using the FQDN rather than the IP. The DNS stuff is fine and there is no network issue as far as I can tell. But I did notice that I can't mount the casper share at all - using any account. It's refusing login credentials no matter what I put in.

were_wulff
Valued Contributor II

@znilsson

If you haven't already got a case open with your Technical Account Manager, I'd recommend doing so, especially with the mention if the same behavior with a USB to Ethernet dongle; if there's something else going on, your TAM will be able to dig in more than I can on the forums, and if it turns out the same behavior does happen with the USB to Ethernet dongles as well as the Thunderbolt to Ethernet dongles, we'll want to get our defect notes updated and let development know that we're seeing the behavior in both USB and Thunderbolt dongles.

Thanks!

Amanda Wulff
JAMF Software Support

znilsson
Contributor II

We did open a case with our TAM but unfortunately this is looking a bit like an unsolvable problem - or more accurately it's not something Casper can solve because the problem is on Apple's end. The base issue here is that for whatever reason, when booted from my flash drive or even the recovery partition on the built in HDD or SSD in this case, I cannot mount any drive via SMB when using a Thunderbolt or a USB ethernet adapter. It's not even just CasperShare, it's literally ANY SMB share.

As a result, Casper Imaging can't mount CasperShare over SMB, through no fault of its own. When we imaged a batch of 25 Mac Pros recently, Casper Imaging worked absolutely flawlessly, because Mac Pros have ethernet ports built in. Our TAM suggested we enable HTTP sharing, but that's not a viable option for us right now, and obviously this is going to be a problem for any Macbook we order, since we're pretty much only ordering rMBPs and Macbook Airs, neither of which have a built in Ethernet port.

So now I'm asking for suggestions on how to get around this problem. We have a lot more MacBooks coming in, and I need a way to get them imaged that doesn't require mounting an SMB share. I'm still new to Casper so I'm just not intrinsically aware of all my options, you know? Thanks for any advice.

were_wulff
Valued Contributor II

@znilsson

The issue with SMB seems to be centered around how Mavericks’ SMB2 deals with Windows SMB shares.

This thread has a few workarounds to try (the issue in that thread is also related to how Mavericks’ SMB2 interacts with Windows SMB shares), but otherwise, if HTTP isn’t an option, did your TAM discuss setting up an AFP share on a Mac in your environment instead of switching the SMB distribution point to use HTTP?

Amanda Wulff
JAMF Software Support

jkuo
Contributor

We had this issue as well, and the way that always works for us is to image the computer and leave it on wireless for the initial enrollment.

We've had a very high success rate also with USB to ethernet adapters, but a very high failure rate with thunderbolt to ethernet.

bentoms
Release Candidate Programs Tester

I wonder how those of you are seeing this issue have created your base OS.dmg?

Joseph_Morris
Contributor

My ? is going to be how big are your USB 3 drives, and why wouldn't you use them as local repositories and install from there? It takes me 4 minutes to image a brand new MacBook Air out of the box using this method with a Thunderbolt SSD external drive. I have 20 configured as external boot drives with a local repository. My "boot image" is connected to my network via wireless and talks to my JSS when Casper Imaging loads on login. It's pretty slick, and as soon as they reboot they begin running their enrollment scripts.

We were going to use Thunderbolt to Ethernet adapters, but the idea of having removable MAC addresses for computers when they imaged scared us away. We are definitely not giving up the Thunderbolt SSDs anytime soon until we get a solid wireless netboot method. With 802.11ac going in, it's going to definitely be something to try down the road.

znilsson
Contributor II

@bentoms][/url - the base OS on my flash drive is just a fully updated 10.9.3 install. I have also tried it booting from the recovery drive on an rMBP and from the main partition on the drive also, each time with the same results - can't mount an SMB share while using a thunderbolt OR a USB ethernet adapter.

@Joseph.Morris][/url - Because we have a hardware depot about 10 miles away from my office that is supposed to be handling all the imaging, and they are not Mac people and don't know how to do anything Mac related so I built these flash drives so all they have to do is connect new Macs to the network, boot off the flash drives and casper install does the rest. The reason we did it this way is so I can update the thin image when necessary with updated installers or whatever, without needing to update all the individual flash drives here and at the depot 10 miles away.

Putting everything on the flash drives (and there's 5 of them) means I would have to keep them all updated individually which is what I'm trying to avoid because I don't have easy access to them. Wireless isn't really an option either because our secure wireless network requires a personal certificate to get past its eap-tls authentication. Netboot isn't an option because it doesn't work across subnets and I don't have a netboot server over at the depot. I was really counting on being able to use the thunderbolt or usb ethernet adapters for this.

ahambidge
New Contributor II

@znilsson, I (aka my network team) has NetBooting across subnets working. I think they had to set an IP Helper address of some kind on the switches to get it to work. If I can get some documentation from my network guys, would this be useful to you?

znilsson
Contributor II

Sure, I could at least go over it. I have the feeling our network group will not want to make any changes to our network though, and certainly not in any kind of short timeframe. But eventually it would be nice to have the option.

jeffwhiting
New Contributor

We also noticed this issue with 10.9.3 but it appears to be resolved with the 10.9.4 update that was released a few days ago. Successfully imaged a MacBook Air (Mid 2012) with a Thunderbolt Ethernet adapter from a base 10.9.4 image built from the installer that's now available on the App Store.

wmateo
Contributor

Here is a Silly Question or maybe I did not read though all the responses. Have you booted the MAC to the factory OS, & plugged Adapter in, and conenct it to your network? see what the behavior is? This happened to me with a Retina and that is what I did. I made sure I got an IP address, and DNS setttings, then booted again, and it worked fine.

znilsson
Contributor II

@jeffwhiting - I'm going to test this, I really hope that fixes it. Haven't tested yet.

@wmateo - Yeah, we did that. In fact with the adapter, we verified that we had access to the rest of our network, DNS was working correctly, proper IP address, everything. But still couldn't mount SMB shares. I even tried it booting from the recovery partition, and then the main SSD partition, still no dice.

KatieE
Contributor
Contributor

Discovered a potential workaround to this. Add a script at reboot:

networksetup -detectnetworkhardware

This forces the computer to evaluate current network hardware, properly detect the Thunderbolt adapter, and proceed with enrollment after 5 minutes as planned. So far this has alleviated the issue in testing.

were_wulff
Valued Contributor II

@kenglish

Thanks!
I'll get that added to the defect so it's readily available for Support to find as well.

Amanda Wulff
JAMF Software Support

znilsson
Contributor II

@kenglish - Ooh that is a fantastic tip, thank you. I will do that ASAP.

pblake
Contributor III

@kenglish - I think it is best practice to always have that in your "first boot" script. Also make sure you delete the Network Interface.plist and preferences.plist in /Library/Preferences/SystemConfiguration/ on your base image ahead of time.

znilsson
Contributor II

Actually I just thought it through, and it's kind of a chicken or the egg situation. I'm booting something like a Macbook Air or an rMBP off my casper imaging flash drive. The issue is that when I boot from the flash drive and I'm connected to the network using a thunderbolt or usb to ethernet adapter, I can't mount SMB shares, which is what my CasperShare is.

So this actually needs to run when I boot the Macbook from the flash drive, in order to run Casper Imaging in the first place, before it can even get to the point where it needs to reboot. Maybe there's a way to run that script at boot on the flash drive.

edit - also, networksetup -detectnetworkhardware doesn't seem to be a valid parameter. There is one called -detectnewhardware though, and it says it detects new network hardware and creates a default service based on it. Is that what you were referring to, or something else?

clifhirtle
Contributor II

We are 99% MacBook Air environment and have seen a bunch of issues with using Casper Imaging on these machines. I was not aware of this defect and wonder if this why I am seeing failure to mount our DP on NetInstall images created with JAMF NetInstall Creator as well... same symptoms of authentication failing not matter what credentials I enter, while an Air booted from the same external drive I built the NetInstall image from mounts our SMB DP just fine.

FWIW @znilsson I have been using a variant of @drheiner's first boot script (https://jamfnation.jamfsoftware.com/discussion.html?id=5078) and Rich Trouton's excellent first boot scripts (https://github.com/rtrouton/rtrouton_scripts/tree/master/rtrouton_scripts/first_boot) related to a longstanding issue of imaged machines not naming themselves. I believe the issues of network detection may be similar to what you're seeing here with external USB/TB adapters not picking up fast enough on initial boot to register with the system and contact back to the JSS (?)

were_wulff
Valued Contributor II

@clifhirtle

What you’re seeing is likely different, since it’s involving the NetInstall product, which is open source.
The GitHub page for it can be found here: https://github.com/jamf/CasperNetInstallCreator and any bug reports would need to be filed there.

We do know that NetInstall + SMB is an issue, but currently there is no workaround for it, other than to use a share point that is not SMB based.

We have a KB article up on JAMF Nation here: https://jamfnation.jamfsoftware.com/article.html?id=74

If you don't already have a case open and/or have further questions/concerns about the above, please give us a call or contact your Technical Account Manager by sending an e-mail to support@jamfsoftware.com or by using the My Support section of JAMF Nation.

Thanks!

Amanda Wulff
JAMF Software Support

clifhirtle
Contributor II

Wow. @amanda.wulff thanks for the heads up on that KB article! I had avoided bugging my TAM since I was aware of the open sourced nature of JNIC and did not want to venture into grey waters. I was just about to go down the endless road of packet captures so that that acknowledgement saves me a ton of time, thank you.

I know many larger enterprise environments are locked into complex Windows infrastructures so hopefully we will see an update to the product that allows the capability down the line.

Chris_Hafner
Valued Contributor II

Honestly, the machines have been really good since 10.9.4 (Though I've only been imaging 10-12 at the most at the moment). I've taken to deleting the system configuration at least for the network preferences for the past year with mixed results. Thanks for the tip -detectnetworkhardware after!

chlaird
Contributor

Experiencing this issue on 9.7 with MacbookPro12,1 machines fresh from the factory. The factory image is 10.10.2, forked build with support for force-touch trackpad

Our issue a bunch of little things rolled into one, so I have a pretty basic script to resolve them in order.

  • Time is wrong (it sets it to 00:01), so connection to JSS is rejected
  • Timezone is wrong, so updating the time still leaves it wrong
  • Ethernet adapter is not detected, so updating time fails, and connection to JSS fails
  • "Automatic" management/enrollment, and an added quickadd, still fail due to the above issues
  • If we check the "Force Apple Setup Assistant" box during imaging, it won't run the first-run script until AFTER the user creates their account, then it reboots, then it makes them create their account AGAIN, and then it brings them to their desktop, where they'll have two accounts. Buggy buggy buggy.

Fix:

  1. Run a package that places quickadd.pkg in temp directory /var/empty ('Install on boot drive after imaging')
  2. Run this script
#!/bin/sh

# detects the ethernet adapter
/usr/sbin/networksetup -detectnewhardware
sleep 30

# sets NTP to ntp.moravian.edu
/usr/sbin/systemsetup -setnetworktimeserver ntp.moravian.edu
sleep 2

# makes sure time is actually correct from NTP
/usr/sbin/ntpdate -u ntp.moravian.edu
sleep 2

# sets timezone to ET
/usr/sbin/systemsetup  -settimezone America/New_York 
sleep 2

# installs the quickadd
/usr/sbin/installer -pkg /var/empty/QuickAdd.pkg -target /
sleep 30

# deletes the temp directory
/bin/rmdir /var/empty
sleep 2

# manage again to be safe
/usr/sbin/jamf manage
sleep 2

# recon again to be safe
/usr/sbin/jamf recon
sleep 2

# force firstrun on reboot
/bin/rm /var/db/.AppleSetupDone

chlaird
Contributor

Anyone still having this problem, or have a stable fix for it? We imaged 600 macbook pros using thunderbolt ethernet adapters, and my above script. All of them appeared to be working after imaging, showed up in the JSS, etc, but upon reboot between 1/2 and 2/3 of them were broken with 'valid device certificate' errors. Only running the quickadd again fixes them.

So, if we don't run a quickadd during imaging, they never get managed at all
If we do run the quickadd during imaging, they have a high failure rate upon reboot.

Frustrating.

cwaldrip
Valued Contributor

We're just now seeing this I think. We've started (finally) imaging our fleet with Casper, and of course it's machines in NY (We're in Atlanta). Everything seems fine (if slow, it's ONLY an OC192 between here and there), but after First Run starts, it seems to hang. A restart of the machine and it runs through First Run like it should and all is good.

We contacted our Jamf support guy, and a little back and forth and he pointed me to this thread.

So, we're going to try a modified version of @chlaird's script. Just the three sections, we have QuickAdd.app set as an install during imaging.

bentoms
Release Candidate Programs Tester

@cwaldrip can you give some more detail of your imaging workflow?

Mine is detailed here & I'm not seeing those issues.