Help

Tomcat logs ballooning...

adroitboy
New Contributor III

Posted on ‎04-09-2012 11:37 AM

Something seemed a little off in the last few days. The database checks out fine, and everything appears healthy, however there are a few things that are strange.

  1. I keep getting some "failed" messages when running policies. This happens on several machines. Here's an example with a jamf recon.
Submitting data to https://jss.company.com:8443//... There was an error submitting the computer to the JSS. Here is what the JSS reported back: <html><head><title>Apache Tomcat/6.0.24 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:1 java.net.URLDecoder.decode(URLDecoder.java:173) com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.createParameterValues(Unknown Source) com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.&lt;init&gt;(Unknown Source) com.jamfsoftware.remote.RemoteController.processRequest(Unknown Source) com.jamfsoftware.remote.RemoteController.doPost(Unknown Source) javax.servlet.http.HttpServlet.service(HttpServlet.java:637) javax.servlet.http.HttpServlet.service(HttpServlet.java:717) </pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/6.0.24 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.24</h3></body></html>

So looking at the tomcat logs in /Library/Logs/Tomcat I noticed some large logs.

-rw-r--r-- 1 root _appserveradm 1.5M Mar 29 00:02 localhost.2012-03-28.log -rw-r--r-- 1 root _appserveradm 1.3M Mar 30 00:04 localhost.2012-03-29.log -rw-r--r-- 1 root _appserveradm 22M Mar 31 00:00 localhost.2012-03-30.log -rw-r--r-- 1 root _appserveradm 198M Apr 1 00:00 localhost.2012-03-31.log -rw-r--r-- 1 root _appserveradm 474M Apr 2 00:00 localhost.2012-04-01.log -rw-r--r-- 1 root _appserveradm 710M Apr 3 00:00 localhost.2012-04-02.log -rw-r--r-- 1 root _appserveradm 1.1G Apr 4 00:00 localhost.2012-04-03.log -rw-r--r-- 1 root _appserveradm 1.6G Apr 5 00:00 localhost.2012-04-04.log -rw-r--r-- 1 root _appserveradm 2.4G Apr 6 00:00 localhost.2012-04-05.log -rw-r--r-- 1 root _appserveradm 3.1G Apr 7 00:00 localhost.2012-04-06.log -rw-r--r-- 1 root _appserveradm 3.5G Apr 8 00:00 localhost.2012-04-07.log -rw-r--r-- 1 root _appserveradm 3.9G Apr 9 00:00 localhost.2012-04-08.log -rw-r--r-- 1 root _appserveradm 1.9G Apr 9 11:27 localhost.2012-04-09.log

Those logs are being filled with this kind of stuff:

java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: ") " at java.net.URLDecoder.decode(URLDecoder.java:173) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.createParameterValues(Unknown Source) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.<init>(Unknown Source) at com.jamfsoftware.remote.RemoteController.processRequest(Unknown Source) at com.jamfsoftware.remote.RemoteController.doPost(Unknown Source) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680) Apr 9, 2012 11:36:06 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet remoteController threw exception java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: ") " at java.net.URLDecoder.decode(URLDecoder.java:173) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.createParameterValues(Unknown Source) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.<init>(Unknown Source) at com.jamfsoftware.remote.RemoteController.processRequest(Unknown Source) at com.jamfsoftware.remote.RemoteController.doPost(Unknown Source) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680) Apr 9, 2012 11:36:06 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet remoteController threw exception java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: ") " at java.net.URLDecoder.decode(URLDecoder.java:173) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.createParameterValues(Unknown Source) at com.jamfsoftware.remote.JAMFHttpServletRequestWrapper.<init>(Unknown Source) at com.jamfsoftware.remote.RemoteController.processRequest(Unknown Source) at com.jamfsoftware.remote.RemoteController.doPost(Unknown Source) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680)

Additionally java seems to be using up quite a but more processing power on the old xserve he are hosting the machine on. Anything that I should be looking for?

0 Kudos
5 REPLIES 5

rhysforrester
New Contributor

Posted on ‎04-10-2012 06:01 PM

We updated to 8.43 about 2 weeks ago and since the update our logs have been ballooning also. Prior to the update they were ~150kb in size and they have steadily grown to ~3Gb.

I'm guessing (nil prior experience with TomCat) that there must be a verbose logging option that was switched off previously that has now been switched on since updating.

0 Kudos

rhysforrester
New Contributor

Posted on ‎04-10-2012 11:05 PM

SEVERE: Servlet.service() for servlet remoteController threw exception
java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: " T"

The only similarity is the space for the input string - guessing blindly - there's an option somewhere that shouldn't contain a space. Where said option is...? Any folk about to help?

0 Kudos

zach
New Contributor III

Posted on ‎04-11-2012 06:45 AM

Hey everyone!

This issue was addressed with the 8.5 release. Here is the snippet from the release notes:

[D-002447] Fixed an issue that caused the JSS to write errors to Tomcat’s localhost.log file when certain malformed requests came in.

Hope that helps!
z

0 Kudos

rhysforrester
New Contributor

Posted on ‎04-11-2012 05:08 PM

Thanks for the heads up Zach.

0 Kudos

svenschott
New Contributor

Posted on ‎04-15-2012 04:34 PM

Hi

I was having the same problem (just upgraded to 8.43) except that we traced it to the recon screwing up (and more importantly the QuickAdd packages when they do their initial recon) because of some data it didn't like in our extension attributes (they had URLs and HTML with forward slashes). We deleted the EAs and the error went away and more importantly recon and quickadd now worked (previously, data had not been updated on the JSS).

Was this a known problem? Can we get more info if it was? We're now having to plan another upgrade because we needed those EAs.

Sven

0 Kudos