Hello junki! Casper patching done right!

[loceee]

Finally! munki for jamf, a.k.a. junki... borrowing a bunch of ideas from munki, and implementing them Casper.

http://www.rehrehreh.com/files/298bd40665ab46319574bafecd411d30-0.html

Take a look at the overview video here ...https://www.youtube.com/watch?v=aeOOPHH3-NY

Get on board and help make it better!

http://munkiforjamf.github.io/junki/

[Russell_kenny]

Nice work Lachy, Looks awesome!

[hcodfrie]

This looks very good think i will dig into junki this weekend thx for posting and your effort

[davidhiggs]

looks interesting, will check it out. good work!

[loceee]

It's been brought to my attention that Office updaters are combos now (thanks @davidhiggs !), but for this example let's just pretend they aren't.

[alex030cc]

Great - Thanks.

[donmontalvo]

Start with a developed/supported solution (Casper Suite), extend its functionality. Nice! :D

[nimoyjohnson]

Nice work loceee! Made my weekend. Really awesome job.

[btaitt]

This looks great, can't wait to dive in! Thanks for your work!

[ImAMacGuy]

I've spent the last 3 or so hours diving in. good stuff so far. I'm a little lost as i'm not sure what smart group criteria is for "all junki clients" as referenced in the docs - https://github.com/munkiforjamf/junki/blob/master/docs/setup_core_policies.md

Further that page in particular could use some more distinction between what is needed for advanced mode in addition to the basic functionality. The docs in the beginning were good distinguishing Advanced from basic, but as they go on they get less clear. You also show in the screen shot at the bottom a Reset defer counter policy that's disabled... do we need that? I hadn't seen it mentioned thus far in the docs.

[loceee]

Hey @jwojda !

I totally agree, I know the docs got a little iffy as they went on. I really wanted to get the thing out, and my deadline was set or it wasn't ever going to see the light of day. Great feedback so far, I will address these issues.

I think the preupdate trigger / policy as it's currently named will become mandatory. The plan being that all of the scoping is done on this policy. Then we can get rid of the confusion around "all Junki Clients".

Also that "Reset Defer Counter" is a policy that you can use to set the defer counter. It just runs

defaults write /Library/Application Support/junki/com.github.munkiforjamf.junki DeferCount -int x

Uses for this could be that you want to deploy an update (we used it when moving all clients to 10.8) by a certain date. Send an email to all clients advising of the impending update, then set your counter so that users will be forced to deploy by Friday / etc. Works really well, users can deploy earlier if they like.

[loceee]

Thanks heaps for all the kind words! I am glad people like it.

I've had feedback and confusion about munki's involvement in junki (ie. none - https://twitter.com/ChipPearson/status/467331897522012160 ) and I don't want to tread on Greg and co.'s shoes.

As of today, junki (which was always a working title) is becoming....

"Patchoo!" - (the sound that a raygun makes).

I am going to update docs, repos, and URLs to reflect the name change... Will post once it's all on the way.

[loceee]

It's all moved, excuse any references to junki in screenshots, and please advise if there are problems with my frantic search and replace.

http://www.rehrehreh.com/files/junki-is-now-patchoo.html

http://patchoo.github.io/patchoo

[ImAMacGuy]

Locee - not sure if you want to use JAMF as the forum for issues or if you'd rather someplace else. But in your doc on https://github.com/patchoo/patchoo/blob/master/docs/install_triggers.md - the link to the triggers is not functioning - leads to page not found. Not a critical issue, since they are included in the zip/tarball and we can make our own pkg's

[dpertschi]

As of today, junki (which was always a working title) is becoming.... "Patchoo!" - (the sound that a raygun makes).

I love it. I suppose then we shouldn't pronounce the letters as a word, but to 'speak it' like a sound effect...

[scottb]

Looks pretty cool! Why not call it "Baconator"? :) Some of you guys amaze me. Perfect timing to as we are discussing options for updating our client's Macs finally.

[ImAMacGuy]

when I run a recon with the EA's, I just get an n/a as an entry, though I have at least 1 pending update according to the MAS. I think it's because I don't have the prefs path - /Library/Application Support/patchoo/com.github.patchoo
Did I miss a step for creating that or how it gets created?

[loceee]

Hey @jwodja,

https://github.com/patchoo/patchoo/blob/master/0patchoo.sh#L195 creates the application support folder ... BUT you've just pointed out a bug. The preferences are trying to be written BEFORE this folder is created on the first run.

Subsequent runs will fix itself, but I will correct that now and push the change. Thanks! In the mean time, if you trigger another 'update' run, it should write the preferences, then do a recon manually, that should bring it in order and the ext attribute should be sorted.

https://github.com/patchoo/patchoo/issues/5

[loceee]

Please pull or download the latest 0patchoo.sh ... let me know how it goes.

[jaharmi]

Both this project (which is “use at your own risk!”) and Munki seem to be equally unsupported — or that is to say, community supported. Is the primary technical advantage of this effort, then, that it doesn’t need the standard Web server (HTTP/HTTPS) that would be required by Munki?

In any case, a lot of effort clearly went into this (by LOC, if by no other measure), so thank you for contributing it. I like to see a vibrant field of ideas on how to manage OS X systems, especially where software deployment is concerned.

[loceee]

Hey @jaharmi,

Patchoo utilises existing Casper methodologies and infrastructure (your CDPs, pkgs, computer groups, policies) which should be familiar to existing Casper admins, and all infrastructure should already exist within your install. The packages you use in Casper Imaging to deploy your Macs, will be the same packages you use in your Patchoo update policy.

Originally, junki did start as a project to integrate munki and Casper... but I couldn't find a cohesive way to make the pieces and methodologies fit together... leveraging all the greatness of munki (I am a massive fan) would have made life easier in some ways, but generated more confusion around which tool is responsible for what. Do you drop using Casper Imaging and computer configurations entirely? Not all of our admins are as command line savvy, do we have another management portal on top of JSS? How do you allocate the munki client identifiers, and how would I allocate different catalogs to clients and keep this control within Casper?

I couldn't come up with something that was cohesive enough to bring to my fellow admins. One main consideration was I didn't want to be the only guy that knew how to drive it. :)

Then it dawned on me, Casper already has really great (and supported) components that we've already paid for .. why not just fill in the gaps?

Using Casper Imaging to deploy the software (without an erase and image) made sense, and it was a supported and easy way for people to deploy their software initially. Self Service is a super great way for users to deploy optional software (if we were using munki, would we be presenting optional installs there too?).

BUT... we all knew that Casper's patch and software installations POST deployment (a.k.a. in the wild) could be greatly improved.

So the design considerations around junki became don't re-invent the bits that work well, but rather fill the gaps and integrate nicely with existing frameworks. I tried to wring munki-esque functionality out of Casper's frameworks. Tying software deployment groups dev/beta/production to Casper's existing computer groups, then tying those deployment policies to custom triggers.. and even dynamically writing your CatalogURL based on this to point at at reposado fork, (I think) is a clever way to accomplish something munki users have taken for granted. AND existing Casper peeps can understand the methodology.

I don't re-invent pkg caching, casper does that fine. I did however build a UI and workflow around end-user notification, and installation.. since we all tend to know Casper is lacking there.

eg. By scoping- policy: "Install New Software" - patchoo.sh --installselfservice to smart group: "updatePatchooInstallsWaiting" A nice little icon pops up in Self Service if you have software installations waiting.

What I REALLY hope I have demonstrated is a workable way to accomplish all of this within Casper, and now JAMF can go and make us all a shiny new "cache Appel Sotware Updates" policy and "Install All Cached" policy for Casper 9.5...

JAMF - my PayPal account is .......... :)

[loceee]

Vote on renaming the "preupdate" trigger.

I never really like that name... my feelings are it should be called something else:

startpatchoo
patchoostart
patchoogo
gogopatchoo

or even just

jamf policy -trigger patchoo

then that trigger calls the patchooPolicy, which fires the update-xxx, update triggers to drive the rest of the process?

Thoughts from anyone that's playing with it and reading the docs @jwojda.

Also please feel free to raise issues and enhancements directly on https://github.com/patchoo/patchoo/issues .. it's much better place to track stuff than in a forum thread.

[ImAMacGuy]

patchoo or startpatchoo is fine.. or patchoopatchoo :) in all seriousness, it doesn't matter to me.

[ahambidge]

@loceee, this is completely awesome. If nobody else has said it, there should definitely be a round or two of the beverage of your choice provided to you at the next social gathering at JNUC. :)

Amazing work!

[donmontalvo]

+1...thanks for the effort, 1st/2nd beer at JNUC2014 on me!!!

That's right, we're going to get you smashed! :D

Word of the day:

Luddite
http://www.merriam-webster.com/dictionary/luddite

[loceee]

If I can convince my employer to spot flights to JNUC I be happy take you all up on that.

But alas, it's probably not going to happen when I am on the other side of the world. *sad panda*

[Kumarasinghe]

what about having JNUC in Australia? :)
An idea .......

[bentoms]

@loceee, if you don't ask.. You don't get.

So give it a go! Personally I'll be flying in from the UK as do quite a few non-US.

Maybe one of the AUS frequenting JAMFS can help? @kitzy

[ImAMacGuy]

so, is patchoo kind of back on hold while some of these changes get worked through? I haven't really been able to get a proof of concept going thus far - which is kind of disappointing.

I've found the patchooStartup policy logged the following:

Executing Policy zzz-patchooStartup... Running script 0patchoo.sh... Script exit code: 1 Script result: patchoo 0.982 --startup - Wed May 21 07:17:51

SetSoftwareUpdateServer policy did set the server

Triggers installed on my test machines

PatchooPromptandInstallAtLogout

Executing Policy zzz-patchooPromptAndInstallAtLogout... Running script 0patchoo.sh... Script exit code: 1 Script result: 2014-05-21 07:15:36.452 defaults[8892:507] The domain/default pair of (/Library/Application Support/patchoo/com.github.patchoo, InstallsAvail) does not exist 2014-05-21 07:15:36.464 defaults[8894:507] The domain/default pair of (/Library/Application Support/patchoo/com.github.patchoo, DeferThreshold) does not exist 2014-05-21 07:15:36.480 defaults[8896:507] The domain/default pair of (/Library/Application Support/patchoo/com.github.patchoo, DeferCount) does not exist 2014-05-21 07:15:36.493 defaults[8898:507] The domain/default pair of (/Library/Application Support/patchoo/com.github.patchoo, BlockingAppThreshold) does not exist 2014-05-21 07:15:36.509 defaults[8900:507] The domain/default pair of (/Library/Application Support/patchoo/com.github.patchoo, BlockingAppCount) does not exist patchoo 0.982 --logout - Wed May 21 07:15:36 patchoo: no software to install, nuffin' to do.

I think part of it is from the bug that's now fixed with creating the Library/Application Support/Patchoo folder and contents. I think in my troubleshooting I manually created them which caused permissions/owners to be different. I removed them and rebooted. The folders were recreated as expected, though its still showing "No" in patchoo installs.

[loceee]

Hey @jwojda, thanks heaps for trying it out and getting your hands dirty!

Most of this stuff is expected (needs to be cleaned up though - thanks for flagging) I've added an issue and fixed some of it on the dev branch. https://github.com/patchoo/patchoo/issues/11 .I will bounce over these fixes and the improved --cache mechanism to the testing branch as soon as I've documented how it works. https://github.com/patchoo/patchoo/issues/4

As for your issues, patchoo only can install pkgs that it knows about. It's not terribly sophisticated in that regard (munki is).

What you are seeing would be expected on first runs and if there is NO metadata in the /Library/Application Support/patchoo/pkgdata folder. Pop in and take a look in there and take a look.

What happens if you run manually call the update trigger?

If you've configured the check apple software update policy, that should check and download any outstanding updates on the the mac (and write metadata to pkdata folder), then the prompttoinstall policy should fire and present a dialog.

If you choose to defer, a recon will be run and the mac will fall into/out of the the correct smart groups.

[loceee]

If you are super brave you can switch the dev branch, which is somewhat complete to 0.99. But the docs aren't quite there. The script parameters have changed .. docs not completely updated to reflect these changes... but we can now eliminate passing the pkg name and pkg description.

NOW just cache your pkg, and run 0patchoo.sh --cache AFTER. junki will pull the info metadata direct from the JSS. Cool! Also I have a fudged progress bar for JAMF installs.. double cool!

https://github.com/patchoo/patchoo/tree/dev

I hope to finish updating the docs tomorrow and pull the dev changes into the testing branch for the almost as brave.

[ImAMacGuy]

ah ha!

I think I found the issue.

When I ran it with the reset SWU Server pointing to our internal server I got the following:

$ sudo jamf policy -trigger update Checking for policies triggered by "update"... Executing Policy 001-PatchooCheckASU... Running script 0patchoo.sh... Script exit code: 1 Script result: patchoo 0.982 --checkasu - Thu May 22 07:29:42 patchoo: setting asu CatalogURL to http://<server>/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1_prod.sucatalog patchoo: checking for apple software updates... Can't load data from the Software Update server (<server>). patchoo: no updates found. Submitting log to https://<JSSServer>:8443/ Executing Policy zzz-patchooPromptToInstall... Running script 0patchoo.sh... Script exit code: 1 Script result: patchoo 0.982 --promptinstall - Thu May 22 07:29:42 patchoo: nothing to install Submitting log to https://<JSSServer>:8443/

when I did a jamf removeSWUSettings and reran it...

$ sudo jamf policy -trigger update Checking for policies triggered by "update"... Executing Policy 001-PatchooCheckASU... Running script 0patchoo.sh... Script exit code: 1 Script result: patchoo 0.982 --checkasu - Thu May 22 07:32:46 2014-05-22 07:32:46.745 defaults[5829:507] The domain/default pair of (/Library/Preferences/com.apple.SoftwareUpdate, CatalogURL) does not exist patchoo: no asu server set, using apple's... patchoo: checking for apple software updates... patchoo: softwareupdate is downloading Safari7.0.4Mavericks-7.0.4 Software Update Tool Copyright 2002-2012 Apple Inc. Finding available software Downloading Safari Downloaded Safari Done. patchoo: USERNOTIFY:: Downloaded, Safari 7.0.4 patchoo: softwareupdate is downloading OSXUpd10.9.3-10.9.3 Software Update Tool Copyright 2002-2012 Apple Inc. Finding available software Downloaded OS X Update Done. May 22 07:33:30 ushofml00990f softwareupdate CLI[5904] <Error>: /usr/sbin/softwareupdate: XPC error in __60-[SUCommandLineTool _runSessionWithUpdates:skippingInstall:]_block_invoke (Error Domain=NSCocoaErrorDomain Code=4099 "Couldn’t communicate with a helper application." (The connection was invalidated from this process.) UserInfo=0x7fc48ba001f0 {NSDebugDescription=The connection was invalidated from this process.}) patchoo: USERNOTIFY:: Downloaded, OS X Update 10.9.3 Submitting log to https://srshofmacdply01.kih.kmart.com:8443/ Executing Policy zzz-patchooPromptToInstall... Running script 0patchoo.sh...

[ImAMacGuy]

it seems to be okay with the dev branch (patchoo.sh and updated trigger) as well. However, I had it running all week on the previous branch (minor tweaks aside) and it never did anything until I manually ran a trigger on my box - and none of my other dev boxes have done anything... In my head I've been meaning to ask that question, how does it get initially launched as everything seems to be tied to a trigger, there's no policy setup to call it to start the checks. The triggers in System/Library/LaunchDeamons don't seem to be responding...

Okay, now the every120 trigger just kicked off ;)

[donmontalvo]

FWIW, valid criticism from our beloved @gregneagle over at the IRC channel. I agree with him, I'd like to see patchoo pick up steam and fill in that gap:

http://osx.michaellynn.org/freenode-osx-server/freenode-osx-server_2014-05-22.html

external image link

Direct link to the above image:

http://donmontalvo.com/jamf/jamfnation/IRC/valid-constructive-criticism.png

And the URL @gregneagle is referring to:

https://jamfnation.jamfsoftware.com/discussion.html?id=10705

[mm2270]

I think its almost a unanimous agreement that software patching is a serious weak area for Casper and we would all like to see this kind of process built into the product. This really should be coming from JAMF. Building some adjunct tools is fine and reasonable. but having to develop an entire workflow to get around this limitation is not IMHO.

I don't have any specific criticisms of junki/patchoo, etc, but my primary concern with it is the same reason I have some sleepless nights about processes that Ive also developed where I work. It relies on cocoaDialog, and as anyone that has paid attention to it will know, that product has seen 0 development in the last 2 years now, with no indication in sight on whether it will ever be further developed. Hardly anyone out there loves cocoaDialog more than me, but I seriously worry that the "next" OS release will break it completely and that will be that. And we'll all be crying into our soup. I'm very hesitant to throw all my eggs in a basket that relies so heavily on cD as its primary method of interaction for this reason. There's no support on cD and the developer has essentially fallen off the grid as far as keeping it up to date.

None of what I've said above negates the amount of work and effort that Lachlan has put into this though. Its outstanding work. I just pray that it isn't brought to its knees by a future OS X release. :(

[donmontalvo]

@mm2270 A couple years ago JAMF Software had a session with some of their bigger clients, where input was requested on what features/enhancements should be considered for v9. Our biggest concerns were (1) exclusions and (2) segregation...we got both.

Today, I think @gregneagle's concern is shared across the JAMF Nation community. I hope there will be a similar session at JNUC2014, so this kind of functionality is burned in to the next version of JSS.

[loceee]

@jwojda - to answer your question on how the update sessions are triggered, they should be fired by the periodic trigger which runs daily at 12:00.

The PreUpdate/patchooStart (in dev branch) is responsible for then firing the necessary triggers based on group membership eg. Beta groups... fires update-beta... and policies that cache the beta software. (this check isn't run unless utilising patchooSWReleaseMode)

Then the start policy triggers "update" this is the main trigger that runs the rest of the update process.

BUT if the patchoo trigger isn't running at midday that will be your problem.

The every120 trigger will fire the reminder bubbles, and if a prompt is missed (timeout, screensaver etc) it will bring up a full prompt again. But it should only present a full prompt once per day.

https://github.com/patchoo/patchoo/blob/master/docs/overview.md

From your logs,

Script result: patchoo 0.982 --checkasu - Thu May 22 07:29:42
patchoo: setting asu CatalogURL to http://<server>/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1_prod.sucatalog

patchoo has re-written your catalogURL to point at axxxxx_prod URL. Unless you have a branch named this in NetSUS / Reposado it won't hit it.

If you aren't using reposado branches, or are using an Apple Software Update server, you need to set ```
patchooasureleasemode=false
```

https://github.com/patchoo/patchoo/blob/master/docs/setup_patchooasureleasemode.md

By using https://github.com/patchoo/patchoo/blob/master/extras/asucatalogset.sh you can replicate the old Casper 8 functionality around setting ASU server based on the network segment data in the JSS. Again, you can set this to write reposado full client URLs, or Apple Software update server URLs that can re-write on based on client OS info.

[loceee]

I agree with @gregneagle 100%.

From a workflow point of view I feel that software deployment and delivery shouldn't be handled by policies and smart groups. There needs to be a specific software deployment section. The patchoo methodology extends and improves existing workflow, but I would like to see it completely shaken up.

munki could be leveraged, and the way I think it should be handled is by extending and exposing the "Computer Configurations" via the API. If we had access to the configurations via the api, some parallels to munki's manifests could be hacked together. Computer configurations, in the same way that manifests let you nest infinitely need to be extended. Perhaps make something called a "pkg group" and have it separate from a computer configuration.

But... deploying a package update should require nothing more than ingesting a pkg into your repo (hopefully automatically) and then dragging it into a group / configuration, jamf installer on the client (as munki does) should be responsible for checking receipts, dependancies, checking for running apps and prompting the user, etc. Lots of scope here... but I think one thing for sure...

...software installation and deployment via policy and scoped smart groups is the wrong approach moving forward.

[donmontalvo]

@loceee +1

...software installation and deployment via policy and scoped smart groups is the wrong approach moving forward.

@gregneagle's 2014 schedule is booked...but wouldn't it be great if he could make it to JNUC2014 to present his perspective on patch management?

Wouldn't it be great if JAMF could break the mold and make JSS 10 what it deserves to be? ;)

Don

[loceee]

@mm2270 - I don't think the reliance on cocoadialog is THAT much of a concern. It wouldn't be unpossible to switch most GUI over to built-in applescript and/or jamfHelper. I do agree that Casper patching needs to be improved massively OOTB though! I consider patchoo a good stopgap, and a potential workflow. Although if you read my post above, I agree that the current workflow is way too admin heavy!

THIS! https://jamfnation.jamfsoftware.com/featureRequest.html?id=2261