Jamf Pro 10.15.1 FAQ

[Garrett_Denney]

Hi Jamf Nation,

We recently became aware of a security vulnerability that impacts versions of Jamf Pro 9.4 and later. To mitigate the issue, we are making Jamf Pro 10.15.1 available today.

This vulnerability does not pose a risk to private data or managed devices. It does have the potential to impact the integrity and availability of your web server.

Cloud customers will be automatically upgraded during the upgrade window outlined below. Premium and Custom customers can contact their Customer Success representative to schedule an upgrade. On-premise customers can download the installer via the My Assets page on Jamf Nation.

Details we are able to provide at this time are below. If you have additional questions, please contact your Jamf representative or leave a comment below.

Thank you,
Garrett

---

Update #1: Sept 29, 2019
Jamf Pro 10.13.1 - Now Available

Yesterday we disclosed a critical security vulnerability that impacts all Jamf Pro instances from 9.4.0 through 10.15.0 and made Jamf Pro 10.15.1 available to mitigate the issue. Today we are making an additional build available for customers that are unable to upgrade directly to the latest Jamf Pro release.

We recognize that some customers might have specific constraints that prevent them from immediately upgrading to 10.15.1. To give those customers an immediate path to mitigation, we’re making 10.13.1 generally available today.

Because all standard cloud customers are already upgraded to 10.15.1 (and protected from the known vulnerability), 10.13.1 is only available to customers that control their instance version, such as On-Premise and Premium Cloud.

To upgrade to Jamf Pro 10.13.1, please contact our Customer Success team at success@jamf.com. We have the capacity this weekend should you want to upgrade immediately.

---

Frequently Asked Questions

What is the issue?
We recently became aware of a critical security vulnerability that could potentially impact any Jamf Pro instance. Jamf Pro 10.15.1 mitigates this issue. This issue does not impact any other Jamf products.


Why is this important?
We take security very seriously and want to move quickly to give you every option to upgrade and stay secure. This vulnerability does not pose a risk to private data or managed devices. It does have the potential to impact the integrity and availability of your web server.

Is my instance impacted?
All Jamf Pro instances running version 9.4 or later are impacted and should be upgraded to 10.15.1 as soon as possible.

When will my standard cloud instance be upgraded?
Cloud upgrades began during a global cloud maintenance window today (Sept 28) at 1700 UTC and will continue through 0500 UTC on Sept 29.

When will my Premium Cloud instance be upgraded?
Please contact success@jamf.com to schedule an upgrade for your environment.

How can I secure my on-premise instance?
An installer is available now in the My Assets page on Jamf Nation.

[afarnsworth]

Is this currently being exploited to your knowledge? Do you have a CVSS score for this vulnerability?

[Aaron_Kiemele]

Thank you for your question. I am Jamf’s Chief Information Security Officer and will be answering questions where possible. Because this is an ongoing security issue, we are not able to share much detail at this time but we will make every effort to provide as much information as possible. The security of your environment is our top priority which is why we are being intentional about how we communicate about this issue.

We are not currently aware of any attempts to actively exploit the vulnerability. We consider the issue serious, and recommend all customers patch. More information about the specific issue will be communicated directly to customers as soon as reasonable.

[ericjboyd]

If you are upgrading from a version older than 10.14.x, please check this article for Incremental upgrade instructions:
https://www.jamf.com/jamf-nation/articles/647/incremental-upgrade-scenarios-for-jamf-pro-10-0-0-or-l...

[davidhiggs]

Hi @Aaron.Kiemele, I'd ask you to consider this feature request:

Email Options: JAMF Security alerts and vulnerabilities

Thanks!

[rtrouton]

Is there a CVE and severity rating for this issue? The reason I’m asking is that having a CVE and severity rating will help with emergency change requests in certain organizations. Otherwise, upgrades at those organizations may have to go through the normal (and potentially much more lengthy) change request process.

[Aaron_Kiemele]

We understand that there may be a desire for more information and intend to share additional details as we’re able. In order to best protect all customers, we are intentionally limiting public information at this time as upgrades continue.

We continue to recommend all customers upgrade to Jamf Pro 10.15.1 immediately to mitigate this issue.

[anverhousseini]

Will there be a 10.14.x release? We have a Jamf Pro instance which is still running 10.14.1 because iOS 9.x was removed from minimum required compatibility with 10.15.0.

[hunter990]

I have to agree with Rich Trouten here. You don’t want to give out any information. You need to find a way to then list it as a critical update if that is the issue. If I can’t get a CVE I can at least use that to get an emergency window to update. Without it I have no recourse and will be required to use the next maintenance window which right now is a couple of weeks away. All I need is a severity rating with an official document or a URL. A Jamf Nation article isn’t going to cut it.

[tlarkin]

@Aaron.Kiemele

Can jamf at least disclose this type of vuln? Is it a RCE, DoS, etc? Many times when going through changes, especially during an end of quarter change freeze, Orgs might need to know more info to weigh the risks.

Thanks
Tom

[bradtchapman]

Seconding what @hunter990 and others have said. We need a CVE number and a description of the vulnerability to escalate an emergency change control.

I had been planning to upgrade to 10.15 next week, but if I can do it sooner then that would be super.

[mschroder]

As we have seen our user facing server server misbehave very badly just before I got the email about the update I really would like to know a bit more about the vulnerability and its potential impact. When I read "potential to impact the integrity and availability of your web server" I want to know in how far the integrity of the server may be impacted, and what are the signs of an impact I may have to look for.

To me "impact the integrity" means someone can gain privileged access to the server, install back-doors, extract and even change information. For servers that configure plenty of clients this is a nightmare.

So I would appreciate more assurance that it is not required to re-install the servers and to restore a backup of the database matching a date before my server started to act up massively.

[mschroder]

What I forgot to say: I appreciate a lot the fact that Jamf made a fixed 10.13.1 version available. I spent the day preparing and testing the upgrade from 10.13 to 10.15.1, and the change of the java version required a lot of fiddling with the config management.Would I have had to change the MySQL version as well I would have been stuck...

[diradmin]

@Aaron.Kiemele if Jamf wants to be a “big boy” software company, they need to abide by the same rules and standards as others. Organizations need a CVE to justify and coordinate an emergency maintenance window to patch this vulnerability. You’re not helping your customers by not empowering them with this information.

[UbiquitousChris]

Echoing @rtrouton, we are also in need of a CVE to get this pushed through our quarter-end change moratorium. I understand the need to keep the “how” under wraps, but I also need more than a “because Jamf says so.”

[Aaron_Kiemele]

We are intentionally limiting how much we disclose about this issue on this public forum. Any Jamf Pro customers that require additional information to properly handle an unplanned upgrade should contact their account team, or Security@Jamf.com. We will do our best to provide more granular data one-on-one.

The CVE ID has been requested and is being processed. We will send an update as soon as we receive it.

[phil998877]

Can we safely assume this is Remote Code Execution on the Jamf server e.g. via a mechanism such deserialisation? If the bug can impact the integrity and availability of the Jamf web server, what stops the exploit taking control of the Jamf fleet?

[kitzy]

Forgive my ignorance here, but would it not be possible to disclose the potential impact/outcome of this exploit without disclosing the how? Surely that would be enough data for most organizations to determine how severe the impact is to them?

[daniel_jacobson]

Withholding information from your customers is not a way to earn trust. Transparency and honesty is the only way to handle situations like this. We deserve to know what is going on and what our potential impact is.

This does not look good.

I need to understand if I'm at risk for this or if I can wait to upgrade my entire environment to a version of software that came out hours ago.

[jhuls]

I think it's pretty obvious that this is pretty serious and that everyone can be affected. Jamf has their reasons for withholding and it certainly benefits us from a security standpoint but I do agree it would be nice to know at least something about it(severity rating, etc.). While I as a tech person might understand this situation, it doesn't always mean the people I report to will see this with the same urgency as there are processes in place to deal with different situations. Jamf needs to realize that many of their customers are in situations beyond their control where they don't have someone sitting in front of a computer and reacting on the fly to patch immediately. The world has become full of business processes.

[kitzy]

@Aaron.Kiemele

I reached out to our account rep per your instructions, and I was referred back to this thread.

Any Jamf Pro customers that require additional information to properly handle an unplanned upgrade should contact their account team, or Security@Jamf.com. We will do our best to provide more granular data one-on-one.

So at this point I can only assume that the issue is a recursion loop.

[Aaron_Kiemele]

We are working to be transparent in a responsible way, by limiting public disclosure for a window of time to provide opportunity for organizations to go through the change process.

Any Jamf Pro customers that require additional information to properly handle an unplanned upgrade should contact their account team or Security@Jamf.com. We will do our best to provide more granular data one-on-one.

@kitzy Please email Security@Jamf.com and we will respond quickly with additional details. Apologies for the miscommunication.

[bvrooman]

This, unfortunately, seems to happen virtually any time Jamf announces anything security-related. It's not an appropriate response from a mature enterprise software developer.

I get an email from Apple on the same day they release security patches listing the specific CVE and a summary (for example, "A remote attacker may be able to cause unexpected application termination or arbitrary code execution"). From that, I can pull up a severity rating and know if this is a "patch it next week once everyone has been notified and approves" or a "put in an emergency change and have some downtime today" situation. There's nothing in there telling me how to exploit it, and security is not (further) compromised by the disclosure.

If Jamf Pro's vulnerability is something so egregious that it can be exploited trivially, then put the actual disclosure in My Assets or similar. The key, as numerous others have mentioned, is that information about the vulnerability — complete enough to be reviewed and acted upon by security and application teams — needs to be made available. A vague statement of "you should patch because this one's pretty bad you guys" is not acceptable, especially if that's what is being delivered to people who have already escalated to their account team.

[kitzy]

+1 on everything @bvrooman said.

[kjen]

I echo @bvrooman post. The example they provided is exactly what I expect and receive from other software vendor like Microsoft, Adobe, Google, etc...

The decision point for my organization right now with the information provided is do we treat this as if its the worst case scenario and assume its a vulnerability that could lead to unauthenticated remote command execution or do we hope its a denial of service vulnerability and wait for more information to come from JAMF before determining how and when we update.

[dgreening]

Echoing others concerns here. We have the 10.13 -> 10.15 upgrade in the pipeline, and really need to know the severity of this so that we can either patch to 10.13.1 in the immediate timeframe or wait for the scheduled 10.15.1 upgrade. Enterprise Change Management requires the CVE details which have been requested again and again.

[kitzy]

I'd encourage you all to email security@jamf.com. They eventually did respond to me with enough detail that I was able to move things forward in my organization.

I tried posting the email here because I believe every Jamf customer has the right to the information without spending half a day chasing it down, but my post was deleted almost immediately. I'm not surprised, but I am disappointed.

[bpavlov]

@kitzy While i can understand the frustration, I don't think posting that information here when Jamf doesn't feel ready is the right move either. You've posted it elsewhere and well now it probably screws it up for other potential situations in the future where Jamf may now have to re-think how they go about communicating issues like this. That is, they may have to get NDAs and what not signed before agreeing to communicate any further on security issues. This essentially makes it even tougher for admins who work at organizations where taking action on a security issue like this may already be tough without the proper "paperwork".

[gachowski]

As far as I am considered almost everyone who posted here owes Jamf and Aaron an apology.

I am more than happy with the way Jamf handles these types of issue. I think this is only the 2 or 3 one that I can reminder in the past 11+ years something this happened. I like this approach and I want Jamf to continue it without any changes.

Here, I have a CVE code for you "CVE-because Jamf said so" if you don't trust your vendors then that is on you!!

"Your" internal BS process shouldn’t have any impact on how Jamf address this, as admins we are creating bureaucracy loops and it's everyone's fault. Keep it up and we will be in “Metropolis”. As this thread proves we have started the downward spiral.

Cutting and pasting my post from two years ago..

Posted: 3/3/17 at 1:30 PM by gachowski I think you guys might be looking at this the wrong way... While I agree transparency is important. You guys are trying get Jamf to disclose something that they feel they shouldn't. Do your change control recommend the change and say that the vendor recommended the upgrade ASAP for not made public security reasons. If your change control approval group denies the change then that is on them. C PS The more I think about it, it's not really very cool for anyone to push Jamf to release details about the vulnerability.

[tlarkin]

This isn't the first time jamf has been under scrutiny for this. The API exploit that allowed an attacker unauthorized access to your scripts was also not disclosed at all. We have two choices here, one assume all notifications like this are really really bad, or two just wait it out.

I am in the crowd, where when I see jamf post stuff like this and basing it off of past experiences, this means that it is bad. Also, jamf saying they don't think anyone is using it in the wild is not really an excuse. This also makes jamf look a bit inept, as they cannot build a proper process to release this information and instead want us to use jamf nation. Why would a jamf nation post be the best place to have this discussion or disclose this info? Now I have to go to jamf nation and search for this article and sift through all the comments to get a glimpse of the info we need?

I propose these topic points as discussion points for both jamf and the community:

• An actual vulnerability process be documented and implemented on jamf's end
• A method and process for customers or third parties to disclose security issues to jamf
• a SLA built upon delivery of said fix and disclosure of what the vulnerabilities were
• A centralized place to find security patches and updates (I don't think searching jamf nation discussions is a good UX)
• Allow for additional disclosures like: email, phone call, etc. If people have to sign NDAs that is fine too

What I am seeing here, is yet jamf dropping the ball again, on what every modern tech company should have. This isn't the first time this has happened, and it seems we haven't learned or grown from the previous times this has happened.

At this point jamf is really telling us to just reverse engineer the product and diff the installers to find the bugs ourselves since it won't be disclosed.

Just like my opinions man.

[mhrono]

100% agreed with @tlarkin. Jamf's response to this incident has been abysmal, and their efforts to prevent the admin community from helping each other only makes it worse.

We should expect better from our vendors. We deserve better from our vendors. Jamf isn't the only MDM out there, and events like this make each renewal decision just a bit more difficult.

Dear Jamf: less gobbling up talent in the admin community, more fixing PIs and learning to communicate effectively with your users.

Edit: A proper response would have been something like: "Hey Jamf users, we found a really bad vulnerability involving DoS/RCE/bbq/etc and we have a patch available. Please update as soon as possible. We will be making a full disclosure on x date."

[RobertHammen]

Going to third @tlarkin's post. In order for tightly change-controlled organizations to update out-of-band, particularly when they may be in an end-of-quarter change freeze, they're going to need a LOT more than "Vendor says it's bad and that we should update immediately". CVE score and ID (if one has been issued) are the minimum baseline. Also, what's the risk? Compromised server? Unauthorized access to client data? Unauthorized access to endpoints?

It would seem to me that this information could be disclosed without detailing the vulnerability, but I'm not a dedicated security professional, do not play one on TV, and did not stay at a Holiday Inn Express last night, so...

[tlarkin]

So, I have seen the note on this exploit and I won't disclose anything about it, because that is the ethical thing to do. I can tell you, it is indeed bad enough to warrant any emergency changes you may have to go through to get it done immediately. I will say, I am disappointed that the info I have seen on this makes either jamf look inept, or it assumes the customer base is dumb, which is to say the least sort of insulting. Also, reading the wording of this note, it is completely misleading and you can tell it was written so for damage control from a marketing perspective.

In the end, if a vuln has the ability for RCE, then just state that. How you respond to vulns speaks more volumes than this cloak and dagger stuff. Also, the vague wording in the note I saw, did not really instill confidence in the spirit of transparency to paying customers.

In the end, this is really just disappointing, and if I really want to know the risks I am put in a spot where I have to start diffing installers and RE the product since the vendor won't disclose it.

[bvrooman]

The question is not whether I trust Jamf or not. I agree that this is extremely rare for a Jamf product, and if they weren't operating within verticals where audit and compliance is a focus, I can see where this would look like a responsible way to go about handling this sort of situation. The issue is that no one else really cares who I trust, and they shouldn't. My change control team (and the governance teams auditing them, and the bank examiners/industry auditors reviewing them, etc.) work in a vast sea of documentation, and if I'm not contributing what they need to make the step above them happy, then I'm not getting approval to change anything.

I'm a little confused by the accusation that by participating in reasonable change control, I'm somehow the creator of bureaucracy. I'm certainly not apologizing for working somewhere with audit and security requirements, because that's insane.

I didn't get any sort of reply from security@jamf.com, so it seems that the "responsible disclosure be damned" side has won the day, and perhaps this whole conversation is moot.
In the interest of fairness, I did receive a reply about the same time that I posted, with a CVSS score and sufficient information to submit an emergency change request. What I received is not enough to provide any insight on how to exploit anything (nor should it be), and I'm honestly surprised that what I received is being hidden from customers. It's completely within the norm for a responsible vulnerability disclosure, since a patch is already available.

[blackholemac]

@tlarkin <<sighs>> in terms of dealing with change control, I think I can get this pushed through on iOS 13 compatibility grounds alone though I hate not being able to fully brief the bosses. If those were the only grounds I was going to hold off for a couple more weeks. Is there anyone at Jamf I can talk to on NDA basis that will reveal what you saw?

That being said, I now shift to the technical side...When I perform the manual upgrade to our clustered environment, does said vulnerability target anything related to the Tomcat version itself? Does it target any specific build of MySQL or is it strictly an issue with the web application that Jamf provides from the download portal? Should I be tightening up my server.xml beyond their current advice?

Guessing you can’t answer any of that due to ethics grounds...Jamf support says I should be fine doing a “vanilla” upgrade to 10.15.1... That being said I find myself wanting to make sure that what I do truly patches whatever this issue is.

[tlarkin]

@blackholemac

The thing is, many Orgs have their own Compliance and Governance, Security Engineering, Blue/Red Teams, Change Approval Board (CAB), and they have a process they have defined internally to accomplish these goals. All the major vendors I deal with disclose Vulns, and we actively pull data from Vuln Scanners to correlate them with data from our data collection tools. jamf is one of those data collection tools for macOS and iOS endpoints. It is up to those teams at each Org to accept the risks, or to approve and push forward patches.

jamf's role in this is simply information. There is a vuln, jamf is aware of it, there is an existing patch for it, but what are the risks? How do we communicate to the people for changes, and how do we go through our defined processes. I agree, I won't apologize or make excuses about change boards and compliance and governance policies, as well as security patching SLAs.

See here is the thing, I have a 14 day SLA to patch anything critical and is available via network access for any RCE. This is a policy set by my company and it doesn't matter if I agree or disagree with it, it is a policy that was put in place by people above my pay grade. So, I must abide by it. This means I have to patch everything with in 14 days, unless I can otherwise prove an exception for. Of course we have emergency patching that is different than the standard SLA.

I cannot raise an exception if I don't know what it is. I cannot weigh the risks if I don't know what it is. God forbid we get audited, but if we did we cannot explain to our auditors anything about our endpoint management solution with this vuln because the vendor will not disclose the info.

I get it, security disclosures are difficult, and jamf wants to protect all those customers who just don't patch their environments. However, jamf needs to build a policy and process for disclosures. I am not going to tell jamf how to do that, because jamf needs to do it so it benefits them and all customers, not just me. It is very clear, at this point in time, none of this exists.

So, jamf needs to engage with the community and build a proper process for disclosing vulns to customers, so customers can leverage their internal processes and do what is best for them. I work for a data company, so we get audited on a regular basis and I am used to it at this point and I am very aggressive with all my patching (OS, third party apps, configs, infra, etc) and the thing we cannot control is our SaaS, which we are jamf cloud. So, for me to not have this disclosure I simply go back to what I stated previously. Which is, I guess if I really want to know the severity of this vuln I will go back to running diffs against the ROOT.war files and start to RE the code myself. Which is something I should never have to do to get a security disclosure.

Also, to address this:

As far as I am considered almost everyone who posted here owes Jamf and Aaron an apology. I am more than happy with the way Jamf handles these types of issue. I think this is only the 2 or 3 one that I can reminder in the past 11+ years something this happened. I like this approach and I want Jamf to continue it without any changes. Here, I have a CVE code for you "CVE-because Jamf said so" if you don't trust your vendors then that is on you!! "Your" internal BS process shouldn’t have any impact on how Jamf address this, as admins we are creating bureaucracy loops and it's everyone's fault. Keep it up and we will be in “Metropolis”. As this thread proves we have started the downward spiral.

I will not apologize for having expectations that every modern tech company follows or should follow. I will not apologize for my Org being security conscious and putting SLAs on everything. We are well with in our abilities from a technology standpoint to actively patch and update our tech, so we should strive to do so. Also, to accept any risk, one must understand what that risk is. If you cannot understand the risk, you just have to assume the worst. Which is not a great way to do business.

Also, if you haven't ever worked for an Org that has standard processes for these things, and you want to call them, "BS," I think you should do a bit of light research. A lot of Orgs right now are probably on end of Q3 change freezes right now, and to emergency patch jamf they might have to get an exception. This is very common for EOQ freezes to allow finance to close the books out on everything and not have any chance of anything external impacting them. Of course, there is always an exception to every process, and jamf is not disclosing enough to make that exception really.

[blackholemac]

@tlarkin first off I’m sorry if I came off as ANYTHING negative to you. I wrote that post mainly looking up to you.

I do have a basic understanding of what major corporate change control boards are like...Via horror stories.

I agree that Jamf needs to find a way to responsibly disclose this sort of thing to their customers in a way that we can take it to whomever the powers that be are. In my case, I’m fortunate enough that my change control board is a small meeting with 6-7 colleagues once a week where we discuss and schedule any infrastructure changes in my org. Like you I hate going in with minimal info because it then looks like I didn’t prepare enough.

Before I knew of this vulnerability, I was already going to put a 10.15 upgrade on the docket but was going to defer a week or two due to personal scheduling issues. Alas given this vulnerability and given my nature as a custodian to my orgs equipment, I’m gonna have to get it on this weeks docket for 10.15.1.

I do fully support your views on finding a way for Jamf to disclose this sort of thing.

Thank you for your time.

[tlarkin]

@blackholemac all good homie I did not take it personally at all, and some red tape at Orgs is ridiculous, but the process exists for reasons.

:-)

[diradmin]

@gachowski

[gachowski]

: )