Using cURL as a way to monitor Tomcat / restart if needed

[hkim]

Has anyone out there used something simple like cURL to monitor if JSS goes down, and to restart Tomcat automatically if it gets no response?

[russeller]

I have the JSS running on Ubuntu servers. My less eloquent solution is using crontab to schedule a script that stops and starts tomcat at 3am daily. https://help.ubuntu.com/community/CronHowto

[hkim]

launchd in OS X would do the same thing, but we need our JSS around the clock. We've also seem to be running into a database corruption issue when we restart tomcat "at the wrong time".

[jarednichols]

What's your server platform? Is this something you're looking to run locally on the server or remotely from something else? Do you have any existing enterprise monitoring solutions?

[hkim]

It's currently OS X.

To be honest it just be nice to have something that'll auto restart Tomcat when it goes down. Doing it at a scheduled time is not the answer and doesn't necessarily fix the problem of JSS crashing.

As for monitoring, I'd think something simple like cURL could just do the job, I could ask if other teams here have something like Nagios or Zenoss to do the heavy lifting. I'd hate to set that up just for JSS and nothing else.

[jarednichols]

Give me a bit and I'll get you in the right direction with launchd

[bjohnson]

One thing that you may want to take into consideration is what is causing your JSS to not being accessible. JAMF had a monitor services feature in the past that would also give you the option to restart those services. I'm guessing that it was taken out because more damage was caused by just restarting those services and never fixing why they are going down. Basically making the problem worse by never fixing it. The snowball down the hill analogy. Just something to think about.

[hkim]

I agree 100% and JAMF Support formally is looking into my environment to try to figure out what's going on. But until then, I still got angry mobs with pitchforks that need to be held at bay when JSS is unavailable.

[rtrouton · Accepted solution]

I'm using netstat on my Linux box to monitor port 8080 to see if Tomcat is up and running on my JSS. If it's not running, it compiles and sends an email, then restarts Tomcat.

I've got a post on what I did:

http://derflounder.wordpress.com/2011/12/14/monitoring-the-casper-jss-tomcat-on-red-hat-linux/

[hkim]

As usual Rich comes to the rescue!

[jarednichols]

Damn Rich, that's purdy.

[barnesaw]

Rich comes through in the clutch. Easy enough to modify to suit just about any environment.

[donmontalvo]

We. Must. Clone. Rich.

[gregp]

I've been using curl since last fall from another box monitoring the JSS on 9006. Also use another script with curl on our JSS box that will restart Tomcat automatically if its non-responsive. Both will notify me when it doesn't work. Wanted to make sure we have another machine monitoring as if our jSS server totally breaks, there's another machine to tell me it went down. If our JSS server dies, it can't tell me its dead (or if the network goes away, etc).

We were having some problems last fall with Tomcat and set this up to tell us when it was going down. That problem has since been fixed, but left the monitoring in place as Tomcat/Java will consume tons of memory until it becomes unresponsive. The script to auto-restart it catches this & restarts Tomcat pretty quickly. This only happens about once every 4-8 weeks and depending on what the system has been up to.

Watching netstat is interesting, but it wouldn't have worked with our problem from the fall. Tomcat had the port open and was barely active (it would accept the connection and that's it). curl is able to check to see if its alive.