Sluggish JSS web interface after upgrade to 9.93

Ditto... but we aren't using any Apple Ed stuff...

Same here. Been happening for a few weeks. Are you hosting locally or in the cloud? We are hosted in the cloud and it seems to get dreadfully slow at times...even more so after the upgrade.

Mixed bag here. What I have found is that it plays nice in Safari, but not Firefox.
And AES is not enabled.

We are hosted locally and it doesn't seem to make a difference which browser is used if it's a current version or older version..

Just curious, any common web browsers in play? What versions? Tried other browsers to see if it changes?

I'm seeing it in Safari 9.1.2 especially badly, but Chrome 52 seems OK.

Ditto

With Firefox, if you have more than on pane open, and you time out, FF just hangs and fails on a script. Well at least for me it does ;-)

I noticed that as well - after increasing my Tomcat memory allocation it improved, will continue to monitor it though

@KarmaIT if you don't mind sharing what is your current allocation?

I changed it to 2GB - It definitely isn't as responsive as before the upgrade but has improved - I notice the slow down when viewing opening polices and displaying the JSS dashboard - regardless of the browser. The JSS is hosted locally on a Windows server - don't know if that is consistent with everyone

I need to check mine to make sure it didn't revert back to the default settings for some odd reason. Typically we run at either 16 or 24GB on a 32GB RAM server for Tomcat.

I'll post a follow up after checking this back at the office.

I had the same issues, I contacted support to get it resolved. I suggest contacting support, I got a quick resolution.

I took my Tomcat minimum RAM from 256MB to 1GB. I also had some MySQL issues, resolved with editing the my.conf with the suggestions from support.

I was getting an error when launching the JSS Database Utility and I was not getting any auto backups either. I'd recommend checking to make sure you are getting backups just in case it is the same.

I saw script errors on Safari, FF and Chrome until I bounced the entire in house VM (CentOS 6) post upgrade.

I've been using Safari 9.1.1 (OS X 10.10.5) to access my hosted test instance of 9.93 as I use Chrome to access my hosted production instance. Several times now JSS 9.93 has become unresponsive while Safari has a process that runs away and grabs a lot of CPU and RAM.

This issue persists with 9.96; apparently the JSS web interface has gotten very sensitive to network latency.

If your latency to the JSS web interface is around 10 ms, everything is okay (regardless of browser). If latency rises to 30 ms, the user interfaces gets very sluggish (simply test with ping, or use the Safari debug menu).

Also, on systems where the system language is different from the language set for the JSS account, the tab titles are first shown in the system language, and after 10 seconds are replaced by their customized names, and the sidebar is finally shown).

I noticed this on our cloud hosted instance as well. Sometimes the url changes but the content doesn't load.

We went through a few days of JAVA hitting 300% on the CPU and MySQL in the 270% range consistently after 9.93 update especially when our few thousand devices started to check in. (JSS and MySQL on different servers.) Users had little to no access to Self Service; actions that would take seconds were now 15-30 minutes, maybe. Just short of a nightmare scenario, besides a complete shutdown.

Though we played with pool sizes and topping off the ram, the problem always went away for a spell after a manual backup. So the database repair that was being done with manual backup seem to stem the problem for a few hours. Ah ha!

All in all, serious check of JSS logs pointed us in the direction of SQL database repair and rebuild. After clearing an exhaustive list of failed commands and rebuilding VPP database twice, we were back in business. The latest JSS update is to allow greater control over failed commands, looking forward to it.

We also saw an issue with Safari locking up on the JSS even when the JSS was in working order between slowdowns. Switching to FireFox solved the issue. It became prevalent when trying to alter Profiles or Configurations, and it was not the lag one sees when leaving results to "All" most of the time.

A big must "say" is that what made this solution successful was the tireless effort of engineer Nathan at JAMF. Life saver extraordinaire.

If you hit the wall, call support and do some deep digging in your setup for MySQL and Tomcat, you want the fastest lanes possible for your setup. An update to the JAVA won't hurt, but check with JAMF on what version you should run on your server.

Of the things I learned, the biggest take away was how Smart groups are designed and what not to do. Stay away from nesting groups (smart groups within smart groups), remove all unnecessary smart groups that are returning 0 results. Streamline your process, keep it simple.

Has anyone here found a real resolution to this issue? We operate globally so we have support staff using the JSS in multiple regions across the globe with some latency. We've found that when introducing any latency to 9.93 or 9.96 in a our production or a basic vanilla dev environment the product is still incredibly sluggish and somewhat unusable.

I've made a video demonstrating the problem via support but am still a bit unsure if the issue is related to our specific environment or an engineer fault / bug in the product. Still a bit curious what the community thinks of this, surely we are not the only concerned JAMF citizens.

https://bulldrive.redbull.com/dl/X6jiPfJoXw

Any thoughts or outside the box tactics / recommendations?

Do you happen to have Patch Reporting enabled?

If so, what do you see when you execute the following MySQL command?

show full processlist;

Don't have the passwords to our dev sql db but will look at that tomorrow. We've had issues with our quality environment so we've stood up a dev vanilla build. There are no patch software reporting titles listed. only one or two clients, no policies or smart groups. The issue is doesn't appear to be related to our Casper DB. Maybe something java or apache related that is causing issues. A colleague thinks there is a javascript issue relating to the multilingual regional settings that are reloading on every page click that may be causing a problem. Something java related, it just hangs at a javascript.

Cheers,

Pascal

mysql> show full processlist;
--------------------------------------------------------------------------------------------------------------------------
| Id | User | Host | db | Command | Time | State | Info |
--------------------------------------------------------------------------------------------------------------------------
| 104882 | jamfsoftware | RBEURJSS01-D.rbdom-d.rbroot-d.net:60002 | jamfsoftware | Sleep | 11 | | NULL |
| 104890 | root | localhost:60009 | NULL | Query | 0 | init | show full processlist |
| 104898 | jamfsoftware | RBEURJSS01-D.rbdom-d.rbroot-d.net:60015 | jamfsoftware | Sleep | 60 | | NULL |
| 104899 | jamfsoftware | RBEURJSS02-D:58060 | jamfsoftware | Sleep | 38 | | NULL |
| 104900 | jamfsoftware | RBEURJSS02-D:58061 | jamfsoftware | Sleep | 76 | | NULL |
| 104901 | jamfsoftware | RBEURJSS01-D.rbdom-d.rbroot-d.net:60021 | jamfsoftware | Sleep | 60 | | NULL |
| 104902 | jamfsoftware | RBEURJSS01-D.rbdom-d.rbroot-d.net:60022 | jamfsoftware | Sleep | 73 | | NULL |
| 104903 | jamfsoftware | RBEURJSS01-D.rbdom-d.rbroot-d.net:60023 | jamfsoftware | Sleep | 73 | | NULL |
| 104904 | jamfsoftware | RBEURJSS02-D:58063 | jamfsoftware | Sleep | 46 | | NULL |
| 104905 | jamfsoftware | RBEURJSS02-D:58064 | jamfsoftware | Sleep | 46 | | NULL |
| 104906 | jamfsoftware | RBEURJSS02-D:58065 | jamfsoftware | Sleep | 16 | | NULL |
---------------------------------------------------------------------------------------------------------------------
11 rows in set (0.02 sec)

Which versions of Tomcat and Java are you using?

see this other thread as well

Using the current Java 1.8.0_101 which works fine with 9.81. We haven't brought 9.96 into production because of the issues in our dev and quality environment. Quality has no devices, policies, clients, smart groups so the DB has basically nothing in it and is still deadly slow when you present some latency which has never been the case in version prior to 9.93. Going to try to roll Java back to 1.8.0_92 in our quality environment to eliminate that as a root cause. Apache Tomcat 8.0.36 running in a Windows 2008 VM server.

I've actually got a really good picture to show what is happening. Every page click is transferring 2-4mb which if you were on the local network is fine but when you have some latency and use this product from another site is crippling. 9.81 was only transferring a bit of data on every click and was never a problem.