Current state of block copy from JDS using Casper Imaging/NetBoot?

[stevehahn]

I've hit a wall with this issue and I'm wondering if anyone has current info--I've found several threads discussing this that are fairly old and none of the solutions work for me.

I have an NBI with Mavericks 10.9.3 and Casper Imaging 9.32. JSS and JDS are also 9.32. OS/Mac model incompatibility is not an issue because I am netbooting the exact iMac that I built the NBI on. When I try to block copy an OS image in Casper Imaging, the Block Copy progress bar goes by in a couple of seconds, then Casper Imaging erases the drive and tries to install the DMG as a package, and then fails.

I have verified that I am disklessly NetBooting. Even so I tried all of the fixes I found related to that and they made no difference.

I've narrowed it down to a problem specifically with block copying an OS image hosted on my JDS. I can block copy the same OS image successfully from an SMB distribution point. When I was building the NBI, I did trust the SSL certs for my JSS and JDS. Anyone have any ideas?

[cstout]

Hello @stevehahn,

I went through several weeks of extensive troubleshooting regarding imaging via JDS using block copy and what I ended up finding resolving it for me was a change in permissions. The Casper Imaging debug log showed me that it was hinting toward a permission-related failure, so I logged into my JDS (Linux based) and navigated to /usr/local/jds/shares/CasperShare/ and checked the permissions on the contents. I found that the particular DMG in question did not have the same permissions as the rest of the files. I ran the command below to make every file the same and tested imaging again and it worked. I hope this helps!

cd /usr/local/jds/shares/CasperShare/
sudo chmod 644 *.* && sudo chown -R root *.* && sudo chgrp -R root *.*

[were_wulff]

Hey @stevehahn

In addition to @cstout ’s info, if that doesn’t work, we can try expanding the NetBoot image size.

To stretch a NetBoot image:

1. On the server, open up a terminal window.
2. Type: sudo hdiutil resize -size 15g /path/to/image/file (Usually something like: /Library/NetBoot/NetBootSP0/ImageName.nbi/System.dmg)

NOTE: 15g is simply an example; if yours is already that large, try 20g or something higher. It can take quite awhile for the process to finish, when it’s done it’ll just drop you back to a terminal prompt.

Let us know if either of those methods takes care of the issue.

Thanks!
Amanda Wulff
JAMF Software Support

[Rosko]

@stevehahn

We had the same issue, but once we did the steps that @amanda.wulff pointed out (as well as the diskless booting guide), everything worked much better.

[bentoms]

Does that actually make the NetBoot.dmg that large, or does it grow to that size on mount?

[cstout]

@bentoms, it actually expands the size of the DMG. My netboot.dmg is now 21GB

[bentoms]

Thanks. Is this needed for JDS then?

I don't do that with our current AFP Casper Shares.

[cstout]

@bentoms This issue does appear to be only when using NetBoot and connecting to a JDS. In my testing I had no issues with File Share distribution points.

[bentoms]

Thanks. May impact a little something I'm doing.

I'll reply here when done, maybe you can test this project for me?

[were_wulff]

@bentoms @cstout

The only time we should need to do it is if we're finding that the JDS, when NetBoot imaging is just flying by the block copy without actually doing the block copy.

It's specific to the JDS being the primary DP when using NetBoot for imaging.

If the JDS is working fine for NetBoot purposes, it shouldn't need to be done.

We had it filed as an open defect for awhile (D-005796), but it was canceled and closed after we found that all we needed to do to make it work was expand the size of the NetBoot image with hdiutil.

[bentoms]

Thanks @amanda.wulff.. That's a shame :(

Oh we'll more to add to my project.

[bentoms]

@amanda.wulff, so when JDS is primary DP & used for imaging?

So like the master DP? Does it affect child JDS?

[were_wulff]

@bentoms

Reading back over the old defect, it looks as though the following two things have to be true to reproduce the behavior:

- The JDS must be designated the master distribution point.
- We must be imaging via NetBoot.

If both of those things are true, we see that block copy fail until we increase the size of the NetBoot image.

It doesn't seem to do the same thing if the master DP is a standard file share and there are child JDSes, as it will mount the actual file share first.

Semi-related to this topic (JDSes vs. Imaging), we do still have a currently open defect (D-006157) in which, if a JDS is designated the default distribution point for a network segment, Imaging (in any form, TMI, running locally, NetBoot) does not respect the segment's default distribution point and will default to the master distribution point which may or may not be the JDS designated.

This can result in a few scenarios that could cause undesired behavior:

1. If a JDS is the master distribution point but the JDS specified in the Network Segment is a different JDS, the default distribution point in Imaging will be the master JDS. This is NOT expected behavior, and is part of the defect.

2. If a File share distribution point is the master, and a JDS is specified in the Network Segment, Imaging will default to the master File share distribution point. This is NOT expected behavior, and is part of the defect.

3. If a JDS is the master and the distribution point specified in the Network Segment is a File share distribution point, the settings in the Network Segment are respected. This is expected behavior.

4. If a File share distribution point is the master but a different File share distribution point is specified in the Network Segment, the settings in the Network Segment are respected. This is expected behavior.

[bentoms]

Right, so only resize if master DP is JDS?

Meh, JDS looks like a mess especially for us with macs globally deployed with local DP's & non-Casper literate techs doing the imaging.

[were_wulff]

@bentoms

Right.

If yours is working as is with the file share as the master DP, there's no need to change anything with it.

If you do start testing with a JDS as the primary, test it out with your NetBoot image as-is first and see if it works; if it does, no need to tweak anything. If it doesn't, we can see if the steps in this thread take care of it.

[cstout]

@bentoms, I was hooked on the potential and automation of the JDS and ran into issue after issue. It's in a rough place right now, but as soon as these bugs get hammered out I really do think it will be a superb addition to the Casper Suite. In the meantime, I've kept my two JDS's up and running and switched back to distribution shares while waiting for future releases.

[bentoms]

@amanda.wulff, thanks.

FWIW, I'm working on something called: AutoCasperNBI.

I was going to add an option to import the JSS CA if using a JDS, I think I'll grey it out until the defects are resolved.

Right, I'm off to tinker some more.

[bentoms]

@cstout, yea I think we'll keep waiting.

Especially as we're multi-national, bandwidth is at a premium.

So can't have people imaging from sources across the MPLS nor can we use the JDS without being able to schedule replication. (I've filed an FR for that one).

[cstout]

@bentoms, is your FR for scheduling replication the one titled, "Staging failover & replication of distribution points."?

[bentoms]

@cstout, it's not that one, but another FR of mine: https://jamfnation.jamfsoftware.com/featureRequest.html?id=2177

[cstout]

@bentoms, thanks! There's my vote. Instant replication for small files isn't a huge deal for our WAN link here, but I'd love the ability to schedule replication when I've updated a base image or some large software suite.

[stevehahn]

Hi @cstout and @amanda.wulff, thanks for your suggestions. My JDS is on a Mac and so the owner and group are _www:staff rather than root:root, but I modified the command accordingly--unfortunately no help there. I also had already tried growing the dmg while on a call with my support rep, also to no avail. I suppose it's possible I messed something up while building my NBI; I might just try rebuilding it from scratch and see what I get.

[bstrube]

We are having the same problem with our JDSes. We have tried to grow our images without any luck. We even downloaded the JSS Built-IN CA to our NetBoot computer and that did not help either. It was already there, but we replaced it anyway. It did not change anything. Our rep thinks it might have to do with the JDS and possibly certificates. Our permissions on our JDS CasperShare are the same as yours. He looked at the certificates we had on our JDS and we only had the self-signed root certificate. The server we was using at JAMF had different certificates. Not sure if this is the problem. Waiting to hear back from JAMF.

[were_wulff]

@bstrube

He looked at the certificates we had on our JDS and we only had the self-signed root certificate. The server we was using at JAMF had different certificates. Not sure if this is the problem.

It may not be the only issue, but it certainly could cause problems if the JSS is using a third party SSL certificate. Typically, we do need to have the root and intermediate CAs imported to the JDS as well as the CA from the JSS (either its built-in or a third party, whichever your environment is using). If your JSS is using a third party certificate, that will also need to be imported into the JDS' certificate store, though those not being there typically won't cause 'unable to connect' errors; when it's a certificates issue we usually see errors that specifically mention problems with a certificate.

One thing that comes to mind, off the top of my head: Do you have Enable Certificate Based Communication checked on your Master JDS (or any of your JDSes, for that matter)?

We can check by going to Computer Management >> JDS Instances >> Select the instance we want to check >> Distribution.

If that box is checked, it's possible you're running into D-006853 or D-007005. In both cases, the workaround is to uncheck “Enable Certificate Based Communication” for the JDS. The symptoms we see with those defects are Casper Admin being unable to mount the JDS or packages being unable to be downloaded from the server with an error similar to "Error: Could not connect to the HTTP server to download $PackageName."

Amanda Wulff
JAMF Software Support

[bstrube]

We do not have "Enable Certificate Based Communication" checked on any of our JDSes. We are using the JSS Built-In CA on the JSS and it is in the System Keychain of the machine we are NetBooting from. It is like it is cannot mount the CasperShareDAV on the JDS.

[were_wulff]

@bstrube][/url

Well, that rules out the 'easy fix' option there, I guess!

Outside of a NetBoot environment, are your JDSes working as expected?

I see that you mentioned that you did try growing the image file with sudo hdiutil resize; how large did we expand it?

When I redid one of mine, I had to kick it up to 25GB (sudo hdiutil resize -size 25g /path/to/image/file) to make it work with my JDS. The original size had been 10GB, and I'd tried it at 15 and 20 with no luck.

Amanda Wulff
JAMF Software Support

[bstrube]

We tried it with 15GB. I can try goring it to 20 or 25. I will try that and report back. Replication between our JDSes is working correctly so I believe everything is set up correctly.

[bstrube]

Latest Update:

Some of my JDSes work and some do not. I expanded the image to 20GB and tried it on our Root JDS. It worked. I was feeling pretty good, but we have 22 JDSes so I thought I would try it on another one. It did not work. I compared the two JDSes and they appeared the same. However when I put their HTTPS address in a web browser, one came up with JSS Built-In CA and the other came up with the server self-signed certificate.

Also, on the one that worked thee are two entries under Websites in the server.app for SSL. The one that didn't only had one SSL entry. I looked at my servers and found that only a few were coming up with the JSS Built-In CA. I remembered that the ones that were working, I went through the "Uninstalling a JDS Instance" document but did not take it out of the JSS. I then ran the JDS Installer. Those servers have two SSL entries under websites and when I click on the lock in the web address, they are reporting they have the JSS Built-In CA.

I am trying it out on a few more servers. It seems to be working. The only problem with this is that it has to pull down all the packages again. I can live with that if the JDSes work. Not an optimal solution but it seems to working for me.

[were_wulff]

@bstrube

At least we've got some progress on it! I'll update the case you've got open with this post so the Support Specialist you're working with is in the loop.

Thanks for your patience!

Amanda Wulff
JAMF Software Support

[were_wulff]

@stevehahn @bentoms @bstrube @cstout @Roskos

I think I pinged everyone in this thread, apologies if I missed anyone.

At any rate, I figured I’d give you guys an update.

There was an older defect that was closed out with the conclusion of increasing the NetBoot image size with “sudo hdiutil resize -size 15g /path/to/image/file” (or 20g, or 25g if 15g doesn’t work; 15g is just an example size); that defect ID was D-005796, just for reference.

Development has been convinced now that this is, in fact, defect behavior; they opened up a defect for it yesterday evening. D-007061 is the new defect ID, for reference, and it’s essentially all of what we’ve been discussing here, but the stripped down version is: “Netboot/Netinstall doesn’t work if the Master Distribution Point is a JDS.”

The workarounds remain the same as the older defect:

1) We can try resizing the NetBoot image with hidutil until we hit a size that lets it work. When I did testing with it a couple of months back, I had to take mine to 25g before it would work, so if 15 or 20 doesn’t do it for you, give 25 a shot.

2) We can use AFP/SMB file shares for the primary distribution point for imaging. If we don't have an AFP/SMB share already set up, we'll need to create one.

If you do have further questions/concerns on this, I’d recommend contacting your Technical Account Manager so we can get things documented and taken care of.
If you’d like to reference the defect behavior, please use the new defect ID of D-007061.
If you’ve already got a case going, it may be a good idea to reference D-007061 with them so they can get your case attached to the defect for tracking purposes.

TL;DR: You’re not doing it wrong, and you’re not losing your mind; it’s on our end, the JDS really just isn’t working quite right with NetBoot and Netinstall right now.

Apologies for that, we do know this is a pretty big, pretty inconvenient one, especially for environments that use primarily or exclusively JDSes as their distribution points.

Thanks for your patience while we get that ironed out!

Amanda Wulff
JAMF Software Support

[stevehahn]

Glad to hear the bug is re-opened and hopefully a fix will be forthcoming soon. Unfortunately resizing our NBI up to 25GB did not help.

[were_wulff]

@stevehahn @bentoms @bstrube @cstout @Roskos

Just to give you all a quick update for workaround options, if your JDS is currently on a Mac, the following KB & included Python script may be helpful in getting it up and running like we'd expect.

Since the article succinctly explains what the script does, I'll just paste it in here as I can't think of a clearer way to reword it: "The python script will create an AFP share, a distribution point in the JSS, and a package monitor that will create links in the AFP share for any package replicated to the JDS instance."

Right now, it's an OS X only solution, and we'll hopefully see one for Linux JDSes soon (unfortunately, I don't have a time table on that, though).

Thanks!

Amanda Wulff
JAMF Software Support

[jonathan_spiva]

@amanda.wulff, any updates on this?

[stevehahn]

@amanda.wulff can we get an update?

[were_wulff]

@jonny74bit @stevehahn

I had a quick chat with the QA engineer assigned to this particular issue, and it is being actively worked on and tested.

I don’t have any concrete release versions or dates to pass along, but I do know this is one of our higher priority issues to get fixed as we know it impacts a pretty big group of customers, so it’s certainly up there on the list to be ticked off and wrapped up soon.

Amanda Wulff
JAMF Software Support

[jenn_dibella]

Syncing to File Share Distribution Points is a manual process that can only occur when using Casper Admin on the Mac. This needs ot be fixed.

[bentoms]

@amanda.wulff, can you find out the location that the JDS downloads files too before then installing on the target Mac when imaging?

[nigelrenton]

I'm very interested to see how this develops. I came along the block copy /JDS issue last year after being sold the JDS concept with much enthusiasm at a JumpStart. Lost a lot of time and patience in the process. Will be sticking to smb shares until I hear some good news!

[wmateo]

same here!!!! I have JDS setup and we don't even use it

[winningham_2]

I would expect, after at least 16 months since I reported it, that JAMF is not able to do anything constructive on the issue. This issue alone has been enough that I have recommended not renewing Casper to put a dollar amount on the problem. I've been a long time fan of JAMF but that was before v.9 (which on the surface, would have fixed all my woes here at OSU). I seriously would love to work for JAMF just to fix this issue, along with the limits on Sites administration/use, to restore the success they had with v.8 (and v.7). I love the company. I have met some great people there and I have a passion to help make them better. But this is one of those things that gives me poor spirits (or the motivation to apply).

So perhaps a feature request is in order? I searched for a feature request on this and didn't find one but I admit, it was only a half-arsed look. Maybe JAMF needs to see a vote tally to make progress on this matter.

The worst part for me is, with this being such a pain and the limits my overlords have placed on us, I have had to research the open-source solutions and that means my supervisors know I could easily transition us out of the JAMF's sphere of influence and save us much dinero. That would be a pity as I do believe Casper is best-of-breed and, again, I have had a great history with their product/company, but its been over 16 months and two of my major complaints have gone unanswered. That is a ¼ of the time I've been running their product here at OSU and makes it nearly impossible to be the Casper Suite product-evangelist I used to be.

I hope the community or someone like @amanda.wulff can push up the importance of this issue and either just do away with JDS in it's current guise, or make their products work as advertised. Fingers crossed.