JSS web performance

[mjhersh]

Somewhere around the 9.96 upgrade, we saw a major decline in performance. We've been living with it for a long time now, and it's a drag. We've have had many calls with Jamf over the years, tweaked this and that, and never seen major improvements.

I recently spoke with some techs at another company to compare notes. They had similar performance issues, despite their being in the cloud and our being on-prem.

So I'm wondering what everyone's experience is with the JSS web performance and I'd like to do a quick poll:

1. How many policies do you have, and how long does your Policies page take to load?
2. Ditto for Computers (e.g. a inventory search to show every Computer in all sites)
3. Ditto for Smart Groups.
4. How many sites do you have?
5. Are you in the cloud or on-prem?
6. Which version of the JSS are you using?

For our environment:

1. 2100 policies, and about 15 seconds (give or take).
2. 3000 computers, and about 5 seconds.
3. 1200 smart groups, and about 5 seconds.
4. 60 sites.
5. On-prem.
6. 10.7.1

I understand we have a LOT of policies and smart groups, and we are actively trying to prune them. However, with so many sites there's no way to avoid a certain degree of duplication.

[lkrasno]

have you looked at Spruce

are you gathering App Usage?

you probably already up-voted ? https://www.jamf.com/jamf-nation/feature-requests/1874/show-where-a-smart-group-is-used

additional scripts for reporting in that thread

[sam_g]

2100 policies? Wow! I think you've clearly summarized what the root cause of your issue is. We have 300 policies and I feel like that's too much! I know it's easy to say, but you need to drastically reduce the number of policies and smart groups. Is it at all possible to combine some of those sites? Or move them to separate servers? Or change them to departments/buildings instead of sites? I'm not sure what your reasoning is for all the separate sites.

Another approach would be to address all duplicate policies by creating a master policy at the "root" level of the server and then scope to specific sites as needed? That way instead of 30 policies on 30 sites doing the same thing, you just have one policy that does that one thing on all 30 sites. I know you currently can't scope to sites in jamf, but you could have a smart group where the client self identifies what site they're on (for example use an ARD field where you put the name of the mac client's site in there that gets written back in recon to jamf).

Regardless of what you do, I don't think polling other jamf users will reveal too much. I feel like you're an outlier in terms of policy/site/smart group numbers, but also everyone's server hardware may be different, which will skew how well someone's jamf instance performs.

[m_donovan]

Our environment:
On Prem: Linux
JamfPro Version: 10.8
Sites: 75
7 Webapp servers (6 under load balancer)
2 Memcached servers
Computers 26,252: 66 smart groups
Mobile Devices 21,561: 32 smart groups
Policies 781: Policy page load during school can be a minute. However, at 4pm it is 14 secs.

We have had times of huge latency and complete outages fairly regularly. The worst of which seems to be under control at this point. We have made a lot of changes to try and help this. We converted to InnoDB and added two Memcached servers a year and a half or more ago. That helped some but we still had issues from time to time especially if a campus would push an update inventory command to 300 devices at once it would bring down the webapps. We started using a web based monitoring service called Datadog to help figure out what was going on. At the time we were all on Windows servers. Throughout this time we tweaked and adjusted MaxConnections, MaxThreads, Thread pool size, MySQL memory settings and a bunch of other settings. With limited benefit. We eventually made the decision to move to Linux servers and that helped even more. We would still have the odd hangup or very slow response times occasionally but full outages were very rare. Recently we increased the page limit and tweaked Java garbage collection settings and at least for now (about two weeks we have not had a single hiccup). Ultimately managing an on prem Jamf instance is a full time job and requires being a jack of all trades and a master of none. Every environment is unique and monitoring is the only way to find what works best for your environment. I would HIGHLY recommend hosting on Linux if you are not already, setting up monitoring and setting up garbage collection logs and evaluating how your webapps are handling Java memory. Here are some links about Java memory that I found helpful.

https://www.datadoghq.com
https://confluence.atlassian.com/kb/how-to-define-xmx-based-on-gc-logs-960142303.html
https://gceasy.io
https://dzone.com/articles/enabling-and-analysing-the-garbage-collection-log
https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/g1_gc_tuning.html#g1_gc_tuning
https://www.youtube.com/watch?v=sSAHvuA0B40&feature=youtu.be
https://www.youtube.com/watch?v=Gee7QfoY8ys

[ega]

We see the same kind of performance issues as @mjhersh with the stats below. @float0n there is a new reality and you are the outlier. As Jamf Pro is adopted the usage cases change and only now is Jamf understanding and adapting to these cases. For the record, @float0n is correct for the current versions on the ground, performance can be made better by the clean up suggested but that is not what is needed or demanded by our use models.

For our environment:

4,076 policies, and about ~32 seconds
13,462 computers, and about ~19 seconds.
2,476 smart groups, and about ~34 seconds.
64 sites
All Cloud including JCDS master
10.8.0