Issues upgrading JSS to 9.81 on Windows 2008 R2 Server

[CarrieNZ]

Hi, I've logged a support ticket about this but thought it worth mentioning here in case anyone else has had a similar issue or was checking for potential issues before proceeding with their upgrade.

I upgraded my JSS to v9.81 this morning and all seemed okay until the final step of the install process when the Tomcat service starts, the installer seemed to crash at this point. I checked the install.log and it showed that the update to 9.81 had been successful so I manually started the Tomcat service and went through my post upgrade checks, all seemed well at this point.

Just over an hour later I noticed that the Tomcat service had crashed and failed to restart automatically. I restarted the host server but the service once again failed to start automatically or manually and was crashing with the event logs showing "service specific error code 4".

I've rolled back to 9.8 and am waiting to hear from JAMF Support before attempting the upgrade again.

[talkingmoose]

JAMF Support helped us troubleshoot and fix a similar issue this morning. After updating from 9.8 to 9.81, Tomcat would not start on our Windows Server 2012r2 system. We made several changes, but I believe what ultimately fixed our issues was:

• Updating the path to the Java JDK in the Windows environmental paths list
• Making sure we entered the correct password for the jamfsoftware database (originally changed from the default "jamfsw03")
• Reinstalling our SSL certificate

This was definitely something I'd never done before and wouldn't have known to do without their assistance.

[CarrieNZ]

Thanks talkingmoose, I've been advised by support to make some changes to the Permgen and max database connections in the Tomcat and MySQL settings and have another crack at it. Unfortunately I forgot to copy the logs off before reverting to the pre-install checkpoint so I didn't give them much to go on.

My suspicion is that it's a Java environment issue so your fix of updating the path to the Java JDK may be what I need.

[talkingmoose]

Just saw this yesterday on Twitter:

What's the most viewed Knowledge Base article on #JAMFNation this week?

Configuring the JAVA_HOME and JRE_HOME Environment Variables on a Windows Server
https://jamfnation.jamfsoftware.com/article.html?id=348

[Aziz]

Is Tomcat using the correct version of Java? I updated Java and uninstalled the old version, for some reason Tomcat was pointed to the previous version of Java and would not start.

Find out what version of Java Tomcat is using. I'm using Java 8, Update 60.

Open

C:Program FilesJSSTomcatin	omcat7w

Then go to the "Java" tab and see if the "Java Virtual Machine" is correct.

Mine says

C:Program FilesJavajre1.8.0_60inserverjvm.dll

[CarrieNZ]

BINGO! I just checked the environment variables and JAVA_HOME wasn't set, thanks for sharing your solution, hopefully saved me another rollback.

[CarrieNZ]

Hmmm... well the installation appeared to complete successfully this time (as opposed to crashing upon starting the Tomcat service like first time), but I thought I'd best restart the server just to check and Tomcat service failed to start again.

After the restart the same issue occurred, but with a different error this time, so I suspect I resolved the first issue only to hit another (potentially an issue with the older java version I'm running on). At least this time I remembered to copy the log files out before rolling back, hopefully that gives support something more to go on.

[Emmert]

What's the error message? I ran across an issue where Tomcat uninstalled itself and I had to manually reinstall the jar files on reboot.

[jtrappey]

Carrie,

That's interesting that this fixed your issue. We had similar issues with v9.65 and added that to our environment variables. That seemed to really stabilized the JSS. I went back to upgrade to 9.81 this weekend from v9.72 and noticed that it was missing (again). I'm wondering if perhaps Microsoft patches could be blowing this away. Odd.

[tanderson]

We updated today from 9.72 > 9.81. The installer finished okay but I've had two instances since where Tomcat hung/crashed and required a reboot to get going again. I'm working with support again tomorrow and will post some details if we get it figured out.

[JPDyson]

Upgraded from 9.73 > 9.81 on a Win 2008 R2 system, can't get Tomcat to start (nor will the JSS uninstaller complete - throws an error about missing a script). All environment variables are good, Tomcat pointed to correct JRE - service just won't start.

Edit: Don't do what I did, because if you break your junk, it's on you, but... I forced the installer to run again (msiexec /fa) and everything is cool now. Checked the tomcat config and I don't see any differences. Maybe tomcat just didn't lay down right the first time I ran the installer.

[Kaltsas]

I had tomcat crash 3 times after upgrading to 9.81 from 9.61 but I tweaked the tomcat memory/permgen settings and it has been running smoothly for the past week.

[tanderson]

@Kaltsas Just got off a support session and that's what we did (tomcat memory/permgen). Too soon to call but hopefully on the road to a more stable 9.81 JSS.

[CarrieNZ · Accepted solution]

UPDATE:

After adjusting the permgen/memory settings and having another try at installing 9.81 my java vm was still unable to start. At that point I talked to support and decided a clean java install would be the best solution, I decided to take the opportunity to upgrade MySQL as well.

I upgraded MySQL to 5.67 by installing to a new path and restoring the database to it, I then had to update the "path" variables to get backup working correctly (it was referencing the old MySQL version and giving the error: "Encountered an error during database back up: mysqldump: Couldn't execute 'SET OPTION SQL_QUOTE_SHOW_CREATE=1': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_QUOTE_SHOW_CREATE=1' at line 1 (1064)". I've seen a few posts on here referencing the error but not the fix so thought I'd share).

Once that was sorted I uninstalled all traces of Java 7 then installed Java 8 Update 65 and JDK/JCE, checked my JSS was running then ran the 9.81 installer successfully.

JAMF Support have suggested I run manual upgrades until the next major release.