Netboot Upgrade, fails to netboot

[jwojda]

Rather than redo the work on the netboot, I figured I would just use the 10.9.1 build that i used to create the NBI, run the combo update, and re-run it through SIU. That completes as expected, however when I try to netboot a machine I just get a Circle with a line through it.

I can reboot off the (now 10.9.2) partition just fine though, so it seems like it's failing due to the SIU.

Any ideas?

[Josh_S]

The NetSUS serves the images via both http and nfs. It advertises the http address but, at least today, it allows either connection. As long as Apple doesn't restrict one method or the other as valid protocols in the firmware of machines, either bless command should work. I am not aware of any hardware that has come out that can't use both protocols.

[elislider]

So, I am getting Netboot 10.9.3 image working on a 2012 Macbook Pro 13", but for a 2013 iMac and a 2011/2012 iMac it just locks up at the Option-Boot screen for about 15 seconds, then boots to the internal drive.

edit: correction it is a 2012 macbook pro 13"

edit2: also a 2009 imac does not even show the 10.9 netboot images as options, maybe because mavericks is deemed not to run on those?

[anderb54]

I'm getting "Image Failed" screens when trying to netrestore any 10.9.3 nbi files. 10.9.1 works just fine. Kernalcache file only 15.5MB. NFS and HTTP both fail to restore images via netboot. Last line in installer log says "unable to revert target to bless."

[clifhirtle]

Went back to the drawing board and reviewed Mike Bombich's old NetBoot troubleshooting page:

http://static.afp548.com/mactips/netboot.html

I can confirm that a fresh install of OS X Server (10.9.3) will boot a client, but the exact same freshly-recreate NetBoot image on the NetSUS 3.0.1 running on a fresh VMWare Fusion converted .OVA VM will not.

More specifically, the native NetBoot service (10.102.81.53) responds correctly against a client-side tftp request, while the NetSUS (10.102.81.55) does not (see below):

US-MAC-6147:~ chirtle$ tftp 10.102.81.53
tftp> get NetBoot/NetBootSP0/NB-RestoreDrive-1092.nbi/i386/booter
Received 510024 bytes in 0.5 seconds
tftp> quit
US-MAC-6147:~ chirtle$ tftp 10.102.81.55
tftp> get NetBoot/NetBootSP0/NB-RestoreDrive-1092.nbi/i386/booter
Error code 256: File not found

[Josh_S]

The path is different when using the NetSUS versus Apple's NetBoot, both for the dmg when downloading via http or nfs as well as the booter and kernelcache files. I don't have a NetSUS spun up at this very moment, but I believe the tftp command should be:

get NB-RestoreDrive-1092.nbi/i386/booter

I'll try in a bit and post back if this is wrong.

[clifhirtle]

Thanks @Josh_S][/url. You have a few bless command examples off the thread below, are those still working examples?

https://jamfnation.jamfsoftware.com/discussion.html?id=4801#responseChild30735

[Josh_S]

@clifhirtle

Those ones should still work. I believe for 10.9 that you cannot omit the kernelcache file for the NetInstall anymore.

[elislider]

I had one of my student employees troubleshooting this some more, and he was able to replace the tftp binary on my Mac Mini Server with a linux-compiled TFTP-HPA, which apparently allows for more customization including changing the block size. This appeared to fix the issue with the kernel cache not transferring. however, the clients are still not netbooting due to other issues which we're currently looking into

[clifhirtle]

Thanks for confirming @Josh_S. Problem appears related to the NFS mount, perhaps specifically the syntax of the bless command across subnets.

Using a local subnet VM and/or native Apple NetBoot server I can boot the clients off the exact same NetBoot images by selection within startup disk, option on boot, or basic "bless -server bsdp://ServerIP" command, but get stuck with same vfs_mountroot failed errors I pasted above when using the extended command, whether on local subnet or across subnets.

Server-side I’m showing the following error within a /var/log/syslog tail:

jssadmin@NetSUS:~$ tail /var/log/syslog
Jun 16 12:25:17 NetSUS dhcpd: All rights reserved.
Jun 16 12:25:17 NetSUS dhcpd: For info, please visit https://www.isc.org/software/dhcp/
Jun 16 12:25:17 NetSUS dhcpd: Wrote 0 class decls to leases file.
Jun 16 12:25:17 NetSUS dhcpd: Wrote 0 leases to leases file.
Jun 16 12:25:17 NetSUS dhcpd: Listening on LPF/eth0/00:50:56:af:79:0e/10.110.200.0/24
Jun 16 12:25:17 NetSUS dhcpd: Sending on   LPF/eth0/00:50:56:af:79:0e/10.110.200.0/24
Jun 16 12:25:17 NetSUS dhcpd: Sending on   Socket/fallback/fallback-net
Jun 16 12:26:30 NetSUS in.tftpd[19424]: tftp: client does not accept options
Jun 16 12:26:31 NetSUS in.tftpd[19430]: tftp: client does not accept options
Jun 16 12:27:30 NetSUS mountd[1167]: authenticated mount request from 10.102.81.52:1022 for /srv/NetBoot/NetBootSP0 (/srv/NetBoot/NetBootSP0)

Here's the specific bless command I'm using now to boot to the NetSUS, which matches all examples posted:

[script]bless --verbose --netboot --booter tftp://NetSUSIP/NB-RestoreDrive-1092.nbi/i386/booter --kernelcache tftp://NetSUSIP/NB-RestoreDrive-1092.nbi/i386/x86_64/kernelcache --options 'rp=nfs:NetSUSIP:/srv/NetBoot/NetBootSP0:NB-RestoreDrive-1092.nbi/NetBoot.dmg' --nextonly

[/script]

[elislider]

More testing with wireshark reveals a possible path error with tftp (?) on a 2013 iMac

Wireshark says TFTP is trying to pull this file:

/private/tftpboot/NetBoot/NetBootSP0/Netboot10.9.3.nbi/i386//private/tftpboot/NetBoot/NetBootSP0/Netboot10.9.3.nbi/i386/x86_64/kernelcache

Which appears to obviously be the wrong path.

My student worker was able to get it to work by running bless commands manually, which we have never had to do with Apple's netboot

To make iMacs boot,
sudo bless --verbose --booter
tftp://10.5.128.117//private/tftpboot/NetBoot/NetBootSP0/Netboot10.9.3.nbi/i386/booter
--kernelcache
tftp://10.5.128.117//private/tftpboot/NetBoot/NetBootSP0/Netboot10.9.3.nbi/i386/x86_64/kernelcache
--options
'rp=ncs:10.5.128.117:/Library/NetBoot/NetBootSP0:Netboot10.9.3.nbi/NetBoot.dmg'
--nextonly

Anyone know what's going on here? Is it System Image Utility the root of the problems (buggy/broken)? Or the TFTP version Apple is using in the firmware?

[CasperSally]

Had a 10.9.3 netboot image that was working for all newer hardware, but some 2010 macbooks were unable to boot to it and eventually booting to internal drive (though some same age 2010 macbooks were getting to netboot fine?).

Tried script in this thread and got immediate panic error mentioning 'unable to find driver for this platform "ACPI...."

Back to the drawing board.

[clifhirtle]

Interestingly, I now have NetInstall image working via the same syntax as above, but am still failing out on NetBoot image at the very point where the authenticated mount is listed in the NetSUS' syslog.

Also, I created a new NetBoot image off a machine's existing machine's drive, kernelcache size 36.7MB, and ends up with the GhostBusters symbol on boot. Checking NetSUS syslog shows the following. If I am reading this right, NetSUS is refusing the TFTP request (?). Is there a specific reason the NetSUS would be denying a client's TFTP request? The NBI's NBImageInfo.plist file does show this specific model in its "DisabledSystemIdentifiers" array, but I've seen that across nearly every image I've created as well.

Jun 18 11:19:00 NetSUS dhcpd: DHCPACK to 10.110.200.241 (00:50:56:a0:00:83) via eth0
Jun 18 11:19:21 NetSUS in.tftpd[29869]: tftpd: read(ack): Connection refused

Lastly, FWIW some similar errors reported by a few others and possible link back to sparseimage creation of NetBoot images in thread below:

https://jamfnation.jamfsoftware.com/discussion.html?id=5252

[russeller]

@clifhirtle HAHA. Ghostbusters symbol. awesome.

[clifhirtle]

After scanning more packets than Kevin Mitnick on a double-shot high at Starbucks, I gave up, double-checked with our network folks, and discovered the CO to add the IP helper on our local switch never... quite... made... it... through (though marked as such) 6 mo back. Never been so relieved to hear something did not happen, as I was seriously thinking I was going crazy with all the testing in a back room test lab with no windows and bad A/C in June.

15m + a download of JNIC 4.0 (https://jamfnation.jamfsoftware.com/viewProduct.html?id=13&view=info) and I am up and running with a network Casper Imaging workflow off the universal restore drive I already had booting (10.9.2 + 10.9.3 combo update).

Thanks for all the ideas/support from folks here. Much appreciated.

[dlondon]

Made a new 10.9.4 NetBoot image using System Image Utility on 10.9.4 on Friday and the kernalcache file was 16MB. That should at least simplify things for machines whose firmware still only allows 32MB max TFTP downlad size

[JeremyL]

I had a working NetBoot image with newer models but when I tried it on a 13-inch MacBook Pro (Mid 2012) I got the prohibitory sign. When checking the kernalcache it was >36MB. I decided to try out the recently updated https://jamfnation.jamfsoftware.com/viewProduct.html?id=13&view=info and now have a working NetBoot image with a kernalcache only 13.5MB in size.

Thank you for the long awaited update, JAMF!

[elislider]

Just wanted to re-confirm that after we modified the tftp client on the server, all further netbooting has been successful.

@jlogsdon, thanks for posting that link! I had no idea JAMF updated it. I'll have to try it out soon

[Kumarasinghe]

Same as @dlondon.
Created a OS X 10.9.4 NetBoot image by installing 10.9.4 Combo on top of factory OS and captured using 10.9.4 System Image Utility. The kernelcache file is 16.6 MB.

Thanks

[elislider]

@Kumarasinghe, That's great news! I haven't had a chance to test out the new Image Utility, but it looks promising.
Hopefully this will save me from having to maintain a custom modified version of TFTP on my Mac Mini hosting the Netboot...

[bentoms]

Yep 10.9.4 SIU solves it.

I also employed the reduction method is you're running on 10.9.x - 10.9.3 in AutoCasperNBI. https://jamfnation.jamfsoftware.com/discussion.html?id=11356

[DLiang]

So is it confirmed that System Image Utility in 10.9.4 + Combo Update fixes it?
I've tried running the SIU in 10.9.4 but netrestore fails before I get to the installation progress.

[bentoms]

I'm not sure about NetRestore, was thinking NetBoot.

[DLiang]

Should be the same concept no?

[donmontalvo]

@DLiang wrote:

Should be the same concept no?

Well...

https://jamfnation.jamfsoftware.com/discussion.html?id=9720#responseChild64839

[DLiang]

Right I get that it's for pre-built images but I'm getting a similar error to the above.
I'll just test it out and see if the combo update helps...

[bentoms]

@DLiang is this a NetSUS btw?

[DLiang]

@bentoms][/url it has reposado on it, why? It it bad to use a SUS server with netrestore?

Forgot to mention that the combo update did not make any difference in the netrestore process.

[frank]

FYI for anyone having issues with Sierra netboots on older hardware: https://openradar.appspot.com/28432978

[m_entholzner]

anyone else having a Déjà-vu here? ;)

[bentoms]

@m.entholzner Oh yep.. i spoke to @frank about this on Slack:

Yea. I put a fix in (AutoCasperNBI) for when the issue happened in 10.9, never removed it & lucked out.

[dlondon]

Hi,

I'm seeing the same problem again with an image created with SIU of a 10.12.1 machine.

In verbose NetBoot mode I see "Error loading kernel cache (0x7)" see attached

and the size of the kernelcache file is 68M

A new machine netboots fine but my trusty old test workhorse now gives the above error and fails

Regards,

David

[Lotusshaney]

hahahaha, Apple never learn do they :)

[jhalvorson]

I've reported this issue with building 10.11.4 and higher Netboot images to AppleCare. I have not tried a 10.12 image yet. They are aware and haven't committed to anything.

If I create a NetRestore image using SUI, that creates a properly sized kernel cache. I copy that into my Netboot.nbi and then I am able to netboot older hardware. Another resource for obtaining a properly sized kernel cache has been to take one created by AutoCasperNBI. I suspect this will work until it doesn't. ;)

[bentoms]

AutoCasperNBI actually uses the kernelcache reduction that was needed for 10.9.3... I just left it in & it seems to resolve this issue.

[Lotusshaney]

Yeap autocaspernbi works as does my old scrips for kernelcache reduction.

[dlondon]

Hi,

Thanks for that -

1) I tried replacing the kernelcache file created by SIU for a NetBoot image with that created by SIU for a NetRestore image and that was just plain sad - it does go past the small spinny globe stage but then ends finally in a stage where there is a gear wheel which keeps coming and going.
2) I also tried replacing the kernelcache file created by SIU for a NetBoot image with that created by AutoCasperNBI and whilst that got to the progress bar under the Apple logo stage (so almost all the way) it looks to have stopped at the end of the progress bar.

I'll give AutoCasperNBI it's head and use that next week and see where that gets me using it alone without a frankenNBI

Regards,

David

[bentoms]

@dlondon cool. Report back please.

With ACNBI, I should be able to help further with the boot process if needed.

[bentoms]

@Lotusshaney oh no surprise that your script & ACNBI works, pretty sure the method in ACNBI was from your scripts. :)