Announcement - JSS add-on for autopkg hits beta, ready for testers

[Banks]

Hey folks, I'm announcing the release of my add-on to the autopkg project which integrates it with the JSS. Please see my post over on the 318Inc blog, http://www.318.com/2014/01/introducing-jssimporter-for-autopkg/, for more information, and let me know what you think if you get a chance to test it. Thanks, Allister

[clifhirtle]

Between AutoDMG, AutoPKG, Automater, and (of course) Autobots, we can only expect this to come shortly...

"Effective 2014, PSUMacAdmins will now be known as PSUAutoAdmins!"

Looks good. Will test. Already pulling custom CrashPlan EA reports via JSS API service account so this should be pretty auto-drop and go.

[donmontalvo]

http://osx.michaellynn.org/freenode-osx-server/freenode-osx-server_2014-01-06.html

Direct link to image: http://donmontalvo.com/jamf/jamfnation/auto-dmg-announcement.png
external image link

@gregneagle][/url, I'm gonna guess that folks will. ;)

Good work, mister @Banks][/url!

Don

[denmoff]

@Banks][/url Great work with this. I just tested it with three recipes and it worked like a charm. :-)

First impression:
1. Since it's a Self Service policy, would be nice to have an icon associated with it, instead of the default open box icon.

1. Would the smart group be more useful if it was used as an "Exclusion" scope? So, it would exclude all computers that already have that application and version. And leave the the "Target" scope empty so that it doesn't "go live" until an admin adds the "All Users" group or whatever

2. I don't love that it creates a category based on the name of the app. I guess that can fixed with a Recipe-Override.

I guess that's all i have for now. Thanks again!

[Banks]

Thanks for the feedback! Here's some initial responses:
1. I would be surprised if the icon art you're asking to set is able to be uploaded, even if we were able to derive it automatically from the installer bundle. If you know how to set an icon without uploading it, that would probably go in the XML we're POSTing, here: https://github.com/arubdesu/jss-autopkg-addon/blob/master/JSSImporter.py#L347
I don't think I have an easy or short-term solution for that, perhaps JAMF peeps can comment regarding how feasible they perceive that to be

1. Oh, you mean like make the smart group work, in comparison to what I'm doing now? :)
   I'm really hoping there can be some consensus on how to best fit the common case, and make it overridable. I've started the conversation here: https://github.com/arubdesu/jssRecipes/issues/1

2. You could, except I didn't code it in an overridable state as of yet - you'd always have to set a category wether you wanted one or not, even though the code can handle if that is missing in the recipe. Just for everyones benefit, you can mass-generate your own recipes and leave that key/value pair out with this script that dumps a fresh set of recipes in /tmp, you can modify the templated recipe (somewhat) easily by removing line 28 and 29:
   https://gist.github.com/arubdesu/7f9630f9a19b27e0db8c#file-jssrecipegenerator-py-L28

[clifhirtle]

Getting the following errors in my testing:

Terminal:

Error in com.github.autopkg.jss.Thunderbird: Processor: JSSImporter: Error: Error parsing XML from checkItem results.

Logs:

1/8/14 1:11:14.000 PM kernel[0]: hfs: summary table not allowed on FS with block size of 2048
1/8/14 1:11:14.000 PM kernel[0]: hfs: could not initialize summary table for Thunderbird

Setup: Test 8.7.3 JSS, SMB DP, full R/W API access account, AutoPKG/JSSImporter setup via steps posted.

Confirmed standard pkg/DMG/download recipes are processing to local cache folder as expected.

Still wet behind the ears with AutoPKG, anything obvious to check here?

EDIT: newbie ?: is JSSImporter assuming Casper 9.x API?

[Banks]

Hey @clifhirtle, pardon me I wasn't as explicit about this as I should have been, Version 8.x of the JSS API does not work to create smart groups. That's not the error being raised, and obviously my error-checking isn't as helpful as it should be. For your own edifimication, you could re-run the processor with the -v or -vv flags to tell which step it's having the issue on. (That output is a bit noisy if you do, I can certainly look into it if you wanted to post that to gist.github.com or pastebin.)
The only two instances in which that code runs is checking the criteria used in a smart group(specifically the pkg id) or policy(specifically both the associated group and pkg), so when the smart group isn't created correctly that very well can fail. If you manually created the smart group you should be able to keep that association on 8.x.

If you instead want to target pre-existing groups(static or otherwise) for your policy, you'd need a recipe that takes advantage of that. Currently it's commented out in all recipes, but you can 1. expand and customize lines 34 and 53 and 2. regenerate the whole lot of recipes with this script:
https://gist.github.com/arubdesu/7f9630f9a19b27e0db8c#file-jssrecipegenerator-py-L34

Let me know how you want to address this. Thanks, Allister

[clifhirtle]

Thanks @Banks! I'll rebake the recipes @ 375 on static groups and see what tastey pastries pop out of the Casper oven. In meantime, here's the double-v output, though not much here more than what we had before so suspect the trip up's with policy creation as you mention:

http://pastebin.com/uC7ptWAh

It may be worth mentioning that many Casper admins I've spoken with have not yet updated to v9.x due to major DB schema changes that prevent proper DB updates/imports, not to mention some features (like central logging) that were stripped from 9.x and many of us rely on. Unclear if JAMF's released migration stats, but for some of us it may be a while still till we move up to 9.x (and the newer API calls).

[Banks]

SadTrombone.com, that looks like the python on the system in question can't convert the value returned from the API when you ask for all categories into a string... which potentially could be a local problem, but could also be an issue with either your JSS or the API user(unlikely, but I'm not ruling anything out yet.) What happens when you run this: ```
curl -kv -u svc_apiwrite:MyPass https://jss.myorg.com:8443/JSSResource/categories -X GET > /tmp/validreturngoesHerePlease.xml
```

[clifhirtle]

Installed APKG-JSSI (more letters please) this AM on a fresh system and got similar logs kicked back.

Through an unrelated JAMF support call we upgraded same JSS to 9.2.2. I can revert/retest back to 8.7.3 in a couple days, but this VMs gone la vida 9.x through Friday. Re-ran tests. Same JSS (now upgraded) + same local APKG-JSSI instance appears to be working now.

Here's the results from CURL query above this AM (JSS 8.7.3):
http://pastebin.com/EdxWrjGj

Here's results from CURL query above this PM (JSS 9.2.2)
http://pastebin.com/v0ZSEWNL

JSS 9.2.2 Thunderbird VV Bake Off Log (new):
http://pastebin.com/FbeaF8iG

JSS 8.7.3 Thunderbird VV Bake Off Log (from last post):
http://pastebin.com/uC7ptWAh

[clifhirtle]

Update: working now on 8.7.3, via tweaking API write user access privs.

One thing: VV output suggests the importer is adding a smart group, while actual action taken is adding a static group.

{'Output': {'jss_category_added': False,
            'jss_policy_added': True,
            'jss_policy_updated': False,
            'jss_repo_changed': True,
            'jss_smartgroup_added': True,
            'jss_smartgroup_updated': False,
            'jss_staticgroup_added': False,
            'jss_staticgroup_updated': False}}

Overall incredible work. This tool is amazing to see in action.

[Banks]

Awesome, I'm glad it wasn't something I've overlooked on the 8.x end, but please do realize, even if it makes the smart group, I am pretty sure it wouldn't properly set the criteria. Also, I'm not using Appname-dot-app as the conditional "Application Title -> is -> Thunderbird"(for example,) as reported here:
https://github.com/arubdesu/jssRecipes/issues/1
so that may require tweaking as well. Thanks for following up!

[ImAMacGuy]

How is the pkg files being uploaded to the JSS/JDS? I got a ton of errors

This package is a PKG or an MPKG, and the index.bom file is not found. Attempting to open the package as a flat package...

[Banks]

@jwojda we're only dealing with flat pkgs - no dmgs, nothing bundle-style, and it's being copied to an older-style locally mounted distribution point which gets served to clients via AFP/SMB. I'm not sure what you're getting errors from in specific, is that Casper admin? We're hoping to avoid interaction with Composer and CA altogether.

[RaulSantos]

is any one actively working on JSS related Recipes?

[tkimpton]

just been trying it out and im very impressed :) One thing missing is the Firefox ESR jss recipe.

Ive tried looking on how to make a recipe but i must be missing something because i can find instructions.

[rtrouton]

@tkimpton][/url,

You should be able to create Firefox ESRs using the existing Firefox recipe and specifying the release you want. I have a post showing how to do this with the Firefox.pkg AutoPKG recipe:

http://derflounder.wordpress.com/2013/11/10/using-autopkg-to-download-and-create-installers-for-fire...

[Banks]

Hey tkimpton, thanks for giving it a whirl! Here's the official guidance on recipe writing in the meantime:
https://github.com/autopkg/autopkg/wiki/Recipe-Writing-Guidelines
I'll post something JSS-add-on-specific soon.
Your note about the Firefox ESR support being missing is what overrides accomplish, as described here:
https://github.com/autopkg/autopkg/wiki/Recipe-Overrides
The Firefox.jss recipe is the one that should be overridden to produce that result, I just gave it a shot to make sure it works.

FYI, If you've already imported Firefox and want both, you may want to give the ESR one a distinct category/group and/or policy name(s), although just deleting the non-ESR version's group/policy/pkg may cut the mustard as well(otherwise the add-on will think there's a more recent version and malfunction when calling the API.)

[tkimpton]

Thanks guys its working now.

I was reading this too literally and thats why it wasn't working "For example, we can change RELEASE to latest-esr" here https://github.com/autopkg/autopkg/wiki/Recipe-Overrides instead of under the RELEASE key change it from latest to latest-esr.

[agirardi]

Just got this setup, and it works great. We will be upgrading our production JSS to 9 on Friday. Looking forward to using this with JSS 9

[agirardi]

Works great with Smart Groups in JSS 9. Want to start contributing to recipes, but have not had time to get back to this.

Just wanted to bump this to see if anyone else is still utilizing this.

[denmoff]

Just wanted to bump this discussion. I really want to get autopkg going on JSS to handle some of those more frequently updated apps like Google Chrome, Firefox, Flash, Java, etc.

Thank you @Banks][/url for taking this on. So I'll just list some of my issues with the current implementation of JSS recipes.

• Override recipes do not work. I want to be able to change the default Category, Smart Group, and Policy Name
• Smart groups are somewhat broken. For instance, when the smart group for Firefox is created, it sets the Application title to "is" Firefox. Since 'is' is used instead of 'has', it fails to find Firefox because it should be Firefox.app. Also, we might need to set the 'Site' for the smart group as well. By default it is set to 'None'.
• I would personally prefer a smart group that setup a list of users that DOES HAVE the current version. And then have the SS policy setup to use that smart group as an exclusion group. I'd then have the option to go in and set the target scope to "All Computers" or some other scope.

[Banks]

@denmoff Hey Mr. Moffett, I updated the code and bumped the release version (download here:https://github.com/arubdesu/jss-autopkg-addon/releases/tag/0.0.2) which includes the proper way to omit things you don't want to set through the official override process(instead of hard-coded in a recipe.) If you're still having issues with overrides please let me see as much of the debug output as you can safely share through whatever channel you'd like to reach out to me to do so. I also fixed the logic error you pointed out with the missing '.app', which is unfortunately hard-coded into one of the template xml's being shoved into the JSS via API. I'll ask the experts how to best abstract those templates out into supporting files for future reference so it doesn't balloon the code and you can modify it as you see fit for things like what scoping is used. Thanks to everyone else expressing interest, it keeps me going!

[charliwest]

This looks great, this combined with patchoo could save us many man hours here!

Any way to get it working with JDS?

[clifhirtle]

Great work @Banks. I'm currently reconstructing our test environment so I should be able to give the newer version a fresh look soon. @dwest confused, is Patchoo TheScriptFormallyKnownAsJunkie?

[charliwest]

@clifhirtle yeah thats it

[loceee]

I haven't said it here... so I will. Love it! When I have some time I will (attempt to) hack something to make it pop out patchoo ready smart groups and policies.

[loceee]

TheScriptFormallyKnownAsJunkie

Like that too... next version may just be called a symbol :)

[calumhunter]

@loceee][/url
So I've been playing with Patchoo! and the JSSImporter for AutoPkg recently and they are both awesome!

What would be really awesome is to get the JSSImporter to work with Patchoo!
I've had a bit of a play around with editing the JSSImporter.py file.
What i've got so far is up on my GitHub here
https://github.com/hunty1/jss-autopkg-addon/blob/master/JSSImporter.py

Essentially this now creates the correct smart group criteria and required policy for Patchoo to do its thing.
Its not setup very nice and was just a quick way draft to see how hard it would be to do. Im no python scripter by any means but Im going to try to do a bit more work to make it play nice with dev and production categories so that automatically downloaded packages don't go straight out to production and instead go to a dev group and then have to be enabled before they actually go out to them.

[loceee]

@calumhunter !

Excited to try it out! I just haven't had any capacity to look at this, so thanks heaps! Having the new pkgs land in dev will be totally bad ass!

[calumhunter]

Boom bad ass status achieved. Was a lot easier than I thought it would be.

https://github.com/hunty1/jss-autopkg-addon/blob/master/JSSImporter.py

[calumhunter]

So I just setup Jenkins with this now as well. Oh my gosh.

EDIT

Ahhh.. Ok slight issue.
Running the recipe (Firefox.jss) once works great, creates everything as it should. But running it a second time, it fails when it tries to check if the smart group is created in the JSS.
It seems to be to do with the name of the smart group in the recipe now not matching what I have changed it to in the JSSImporter.

The recipe is saying to use a smart group name of LessThanMostRecent_%NAME%
but the JSSImporter is creating update%PROD_NAME%-%version%

I've created an override for the recipe, and changed the smart group name to update%NAME%-31.0 this runs fine.

So how would I go about getting the version variable into the recipe? is this possible?

[calumhunter]

For those following along at home.
The version variable is defined by a processor which runs after the recipe, so can't be evaluated in the recipe.
Solution - lets not try to create the name of the smart group in the recipe. instead lets create the smart group name variable in the JSSImporter which is able to evaluate the version variable.

I'm just not sure how to do that yet