Jamfds segfault after RHEL update

[rstasel]

Hi All,

I have a ticket in with Jamf about this, but thought I'd ask here. Anyone seeing jamfds segfaulting after and update of glibc? I'm trying to reinstall the jds, but it can't contact the primary since it seems to segfault during setup. =/

It looks like it might be related to ld-2.12.

Thanks!

[were_wulff]

@staze

Did you recently update your NSS dependencies?

If so that may be the problem, which we have filed under PI-003727.

The workaround is to downgrade them again.

If this does turn out to be the case, support will be able to provide you with the command to downgrade the dependencies, at which point the JDS should start working again.

While I do realize that downgrading the NSS dependencies is not ideal it is, unfortunately, the only option we have to let the JDS start working again if it turns out that's the cause of why it stopped.

Thanks!
Amanda Wulff
Jamf Support

[rstasel]

Hi Amanda,

Looks like it.

(127/206): nss-3.27.1-13.el6.x86_64.rpm                                                                                                                                                                                                    | 873 kB     00:00     
(128/206): nss-sysinit-3.27.1-13.el6.x86_64.rpm                                                                                                                                                                                            |  50 kB     00:00     
(129/206): nss-tools-3.27.1-13.el6.x86_64.rpm                                                                                                                                                                                              | 443 kB     00:00     
(130/206): nss-util-3.27.1-3.el6.x86_64.rpm                                                                                                                                                                                                |  68 kB     00:00

If you could let me know how to downgrade those back, that would be great. Otherwise, we'll wait for Jamf buddy to respond.

[rstasel]

Figured it out. Having to wait to try to re-join the jds to the jss so I can confirm or deny.

yum downgrade nss nss-utils nss-sysinit

yum downgrade nss nss-tools nss-utils nss-sysinit
Loaded plugins: product-id, rhnplugin, search-disabled-repos, security
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Downgrade Process
No package nss-utils available.
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.21.3-2.el6_8 will be a downgrade
---> Package nss.x86_64 0:3.27.1-13.el6 will be erased
---> Package nss-sysinit.x86_64 0:3.21.3-2.el6_8 will be a downgrade
---> Package nss-sysinit.x86_64 0:3.27.1-13.el6 will be erased
---> Package nss-tools.x86_64 0:3.21.3-2.el6_8 will be a downgrade
---> Package nss-tools.x86_64 0:3.27.1-13.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

Package Arch Version Repository Size

Downgrading: nss x86_64 3.21.3-2.el6_8 rhel-x86_64-server-6 859 k nss-sysinit x86_64 3.21.3-2.el6_8 rhel-x86_64-server-6 47 k nss-tools x86_64 3.21.3-2.el6_8 rhel-x86_64-server-6 437 k

Transaction Summary

Downgrade 3 Package(s)

Total download size: 1.3 M
Is this ok [y/N]: y
Downloading Packages:
(1/3): nss-3.21.3-2.el6_8.x86_64.rpm | 859 kB 00:00 (2/3): nss-sysinit-3.21.3-2.el6_8.x86_64.rpm | 47 kB 00:00

(3/3): nss-tools-3.21.3-2.el6_8.x86_64.rpm | 437 kB 00:00

Total 5.0 MB/s | 1.3 MB 00:00 Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction Installing : nss-sysinit-3.21.3-2.el6_8.x86_64 1/6 Installing : nss-3.21.3-2.el6_8.x86_64 2/6 Installing : nss-tools-3.21.3-2.el6_8.x86_64 3/6 Cleanup : nss-tools-3.27.1-13.el6.x86_64 4/6 Cleanup : nss-3.27.1-13.el6.x86_64 5/6 Cleanup : nss-sysinit-3.27.1-13.el6.x86_64 6/6 Verifying : nss-3.21.3-2.el6_8.x86_64 1/6 Verifying : nss-sysinit-3.21.3-2.el6_8.x86_64 2/6 Verifying : nss-tools-3.21.3-2.el6_8.x86_64 3/6 Verifying : nss-sysinit-3.27.1-13.el6.x86_64 4/6 Verifying : nss-3.27.1-13.el6.x86_64 5/6 Verifying : nss-tools-3.27.1-13.el6.x86_64 6/6

Removed: nss.x86_64 0:3.27.1-13.el6 nss-sysinit.x86_64 0:3.27.1-13.el6 nss-tools.x86_64 0:3.27.1-13.el6

Installed: nss.x86_64 0:3.21.3-2.el6_8 nss-sysinit.x86_64 0:3.21.3-2.el6_8 nss-tools.x86_64 0:3.21.3-2.el6_8

Complete!

[rstasel]

correction, in my case, I also had to downgrade nss-tools.

[rstasel]

worked! Thanks!

Now I need to figure out how to prevent yum from updating those.

[rstasel]

Okay, you just add a line to /etc/yum.conf of:

exclude=nss*

Is there going to be an update to the jds package that will work with these newer binaries? I note this: RHEA-2017:0671 as the reason it's been updated.

[were_wulff]

@staze

We don't have any ETA on a fix for that particular PI, but once it's taken care of it'll be listed in our release notes.

Thanks!
Amanda Wulff
Jamf Support

[rstasel]

or, install versionlock "yum install yum-plugin-versionlock" then "yum versionlock nss-*"

Thanks!

[jkeller13]

Downgrading the nss-tools and nss-sysinit packages to 3.21.3-2.el7_3 fixed the problem we were encountering. We also added an exclusion for nss in yum.conf. Thank you VERY much for this post @staze !!

[jyergatian]

Can confirm a downgrade resolved a similar issue.

Issue found within /usr/local/jds/logs/jamf.log

2017-05-08 03:00:02,594 WARNING Lock file exists, but the program is not running. The program may have exited abnormally.
2017-05-08 03:00:02,596 INFO Checking for policies...

Resolution

yum downgrade nss-3.21.3-2.el7_3 nss-tools-3.21.3-2.el7_3 nss-sysinit-3.21.3-2.el7_3

[rstasel]

Downgrade did fix the issue. I then version locked the files so they won't update on that machine until jamf resolves the NSS dependancy.

Thanks!

[rdwhitt]

@amanda.wulff Are there any concerns regarding the critical vulnerability in NSS and NSS-UTIL that RedHat disclosed in April? Until this issue is fixed, we're unable to patch the NSS packages.

https://www.redhat.com/archives/rhsa-announce/2017-April/msg00042.html
https://access.redhat.com/security/cve/cve-2017-5461

[were_wulff]

@rdwhitt

I can’t make that call for you; it’s up to your organization to decide how critical they feel the NSS updates are to your environment.

Unfortunately, upgrading the NSS dependencies will break the JDS so you may be in a spot where you’ll have to choose between using the JDS or updating the NSS dependencies.

There is nothing else we can do as that product issue is still open.

Personally, I’d recommend migrating away from using the JDS entirely and switching to either an HTTP/HTTPS file share, a cloud based share, a non-Windows based SMB share, or an AFP share.

If you have any other questions regarding PI-003727, please get in contact with Support either by giving them a call, sending an e-mail to support@jamf.com, or by using the My Support section of Jamf Nation.

Thanks!
Amanda Wulff
Jamf Support

[rdwhitt]

@amanda.wulff Thanks for the info, I did contact support and they directed me to post in this thread. I wasn't really asking you to decide anything about my environment, I was just curious if the critical vulnerability was a concern for Jamf or in any way would help expedite a fix. I've been considering moving away from the JDS for a while now, so this is probably the time to finally do it. It's just sad since the JDS had so much promise, just not a lot of development or improvements since 9 was released.

Thanks again!

[were_wulff]

@rdwhitt

I wish I had a better answer for you on that one than the answer I'm about to give, but I don't; this particular issue has been open since 2014 and it's in our development team's backlog, but I can't say for certain when it'll be fixed.

We generally aren't given ETAs on things like that in case something comes up or it proves more difficult than initially thought and the time frame has to be pushed back; I understand why that is, as we don't want to be in a position where we tell customers, "It'll be done by X date" then have to come back later and say, "We're not going to make X date, it'll be Y now", but it does make it tricky to answer these types of questions. :)

Security issues are definitely something that tend to get a closer look but, as with most things, it's all taken on a case by case basis.

The only thing I do know for certain is that it will not be fixed in 9.99.0.

Amanda Wulff
Jamf Support

[jamf_tx]

Does JAMF realize this NSS update fixes a CVE with a CVSS score of 9.8/10 (critical)?

It's very hard to believe JAMF takes security seriously when they recommend customers leave themselves vulnerable to a remotely exploitable critical bug so they can use a JAMF product. We just purchased a license, and now I've learned I can't use JAMF JDS without knowingly leaving systems exploitable (with no ETA for a fix). That's not a good first impression.

[soonblue]

I just received a response from support directing me to this thread as well. I've got to agree with the poster above. JAMF telling people that the workaround (which apparently has now been a workaround for three years??), is to downgrade packages that fix a major security hole is inexcusable and worse, irresponsible for Linux users who might not understand the implications of downgrading these packages. That this has been an issue since 2014 and it still has not been patched frankly astounds me and leaves me with a very poor impression of JAMF from a security standpoint.

[richardt]

This was the workaround suggested to me as well in a recent support tickets. Unfortunately, we utilize two separate JAMF Pro environments at my company any heavily use RHEL servers as the platform for JSS and JDS instances. Downgrading nss packages is not a viable workaround or solution if it opens up critical vulnerabilities for your customer. The mere fact that there are forum posts dating back 3 years on this same topic are rather concerning, as others have mentioned. A security hole like this should be a very high priority on JAMF's list to get corrected, and as soon as possible.

I know it has been mentioned before that there is no planned date to fix this, but I will ask anyway as my upper management and security director is tracking resolution to this. When will we see a permanent fix for this problem released to your customers?

[analog_kid]

We'd also like to see this addressed as soon as possible.

[jamf_tx]

This wasn't fixed in 9.99.0 - will it be fixed in the next release?

This puts me in a very awkward situation. I advocated strongly for the JAMF product. Now that we're trying to deploy it, on CentOS, I can't install JDS at all unless I want to expose my company to a critical, remotely exploitable CVE. On top of that, JAMF hasn't committed to fixing this in any release, which leads me to believe time hasn't been allocated for this fix, and it's not even in progress. So I have no timeline to report to my bosses.

In summary:

• This issue blocks customers from installing JDS
• This issue blocks customers from keeping their systems up to date
• The workaround exposes customers to critical security vulnerabilities

Yet, JAMF has not commented on when (or if) this will be fixed

I'm not sure how JAMF prioritizes cases, but I would think any of the above situations would cause the issue to rocket to the top of the list. This makes me very uncomfortable. I'm not seeing that here. As my first exposure to the JAMF environment, it makes me wonder if I made a mistake pushing so hard for this product. There are other alternatives, maybe not as robust, that seem much more responsive to security problems.

Device management software should be secure by default and I'm not seeing that here.

[dustin_liddick]

I just wanted to post my 2 cents as well. I appreciate the others that have gone through support and supplied their answers as I have yet to do that. With that being said, I will not downgrade in order to leverage the JDS. I have moved to SMB (yes I know) as well has HTTPS downloads. Id rather lockdown my SMB server with SELinux and IPTables than to make my company vulnerable to outside threats. Thanks to all who have posted!!

[Kedgar]

Well, I now have my 3 JDS instances that are unusable even after a downgrade of the nss packages... I will open a ticket with JAMF shortly. I love this product, but I'm wondering if they are looking to phase out the JDS? If that is the case, it is quite sad, and we NEED a replacement that provides us with:
- Ease of deployment
- Simplicity
- Security
- Bandwidth throttling
- Automatic and granular synchronization to other JDS instances

Nice to have:
- Support Microsoft DFS
- Get rid of caching package uploads to the mysql database

Simply managing synchronization manually through Casper Admin, or by us admins having to script it is not an ideal solution when this is a paid product.

[Kedgar]

The NSS packages I had to downgrade to seem to be the following:
nss.x86_64 0:3.21.3-2.el7_3
nss-sysinit.x86_64 0:3.21.3-2.el7_3
nss-tools.x86_64 0:3.21.3-2.el7_3

[rstasel]

Hi All (jamf specifically),

Any update on this?

[were_wulff]

@staze

There is no update.

When we have an update on JDS support, @michael.devins has stated that an update will be posted.

Thanks!
Were Wulff
Jamf Support