tomcat freeze prevents login

[marcusransom]

Hi everyone,

we are having a strange issue here and I am hoping someone might be able to point us in the right direction to resolve this.

Casper 8.3.1 hosted on Windows server 2008 R2 (not sure if updating to 8.5.2 will resolve but it is scheduled in to be done in a couple of weeks).

Tomcat locks up a couple of times a week and needs to be re-started. The service shows as still running but the JSS web page won't load and policies won't run. Restarting tomcat using the tomcat7w.exe resolves the issue until it freezes again.

up until recently this was only a 'minor' inconvenience as the users could still work without policies until I could restart Tomcat. Several weeks ago it began to manifest itself in a considerably more frustrating manner. If users tried to log in when Tomcat had frozen, their machines would stall at the login screen when it tried to check for casper login policies. Sometimes machines would complete login after 30 minutes (not ideal) sometimes they were unable to login. This would happen offsite as well as onsite if they were connected to the internet. Once again, restarting Tomcat fixes immediately (even machine stalling at login complete login the second tomcat is stopped).

any suggestions would be greatly appreciated!

Marcus Ransom

Senior Support Analyst
Carey Baptist Grammar School

[AndyBeaver]

We had a similar issue. Verify that the JSS is not flushing the logs and attempting to create a nightly back up close to the same time. I was restarting tomcat every morning.

[andyinindy]

We are also seeing periodic freezing of Tomcat on Windows 2008 server.

How can I check on the log flush schedule on the JSS to make sure that the backup is not occurring at the same time? Is this a configurable value on the JSS, or should I just reschedule the database backup?

--Andy

[andyinindy]

I rescheduled the database backup to see if that helps...

[Lincoln]

This was one of the things I checked into with my recent Tomcat troubles.

Flush database logs schedule can be set from the Settings tab in the JSS.

Database backup time is set from the JSS Database utilty in:

Program FilesJSSin it is called JSSDatabaseUtil.jar

The recommendation is that the flush is not done within two hours of the backup. I have my flush set up for 9pm and backup for midnight.

This was not my issue so I can't speak for how it may affect yours.

Lincoln

[andyinindy]

FYI, so far moving the database backup forward a few hours seems to have resolved our issue. Just wanted to provide an update.

[andyinindy]

Well, I spoke too soon. We are still seeing tomcat crashes, but only on weekends at around 2 AM. Interestingly, we had initially rescheduled our backup job for 2 AM, so this would suggest a correlation. However, this does not explain why the crashes only occur on weekends. I have now rescheduled the backup for 4 AM; we'll see if this makes a difference.

[andyinindy]

Another FYI; not sure if I was able to remedy Tomcat crashing via rescheduling the backup, but the service has not crashed/hung for any appreciable amount of time ever since I changed the recovery behavior to "Restart the Service" for all failures. Since the fix was always to simply restart the service manually anyway, setting this to happen automatically in Server Manager-->Services has effectively resolved the issue for me. It does not explain why the service crashes (which would be nice to know), but I suppose that I don't care as long as it is able to restart cleanly.

[techras]

Hi Guys,

Have you all considered increasing the Xms and Xmx options of the JVM used by Tomcat?
I have experienced freeze in tomcat in the past due to frequent GC activities in the JVM of tomcat server.
Set the Xms and Xmx to higher values (ideally double the current value) and restart the tomcat server.

Cheers,
Techras