Tomcat – How to Run Strace or Ltrace on Catalina

rhel5stracetomcat

Running ltrace isn't trivial. This RHEL 5.3 system has based on a Tomcat Catalina (servlet container) which uses text scripts to tie everything together. When I tried to find an executable here's the rabbit hole I went down:

/etc/init.d/pki-ca9 calls dtomcat5-pki-ca9

#Path to the tomcat launch script (direct don't use wrapper)
TOMCAT_SCRIPT=/usr/bin/dtomcat5-pki-ca9

/usr/bin/dtomcat5-pki-ca9 calls a watchdog program

/usr/bin/nuxwdog -f $FNAME

I replaced nuxwdog with a wrapper

[root@qantas]# cat /usr/bin/nuxwdog
#!/bin/bash
ltrace -e open -o /tmp/ltrace.$(date +%s) /usr/bin/nuxwdog.bak $@

[root@qantas]# service pki-ca9 start
Starting pki-ca9:              [  OK  ]

[root@qantas]# cat /tmp/ltrace.1295036985
+++ exited (status 1) +++

This is ugly. How do I run strace or ltrace in tomcat?

UPDATE

Here is the tomcat "process"

[root@qantas]# ps -ef | grep tomcat
pkiuser  21767 21766  0 10:10 ?        00:00:09 /usr/lib/jvm/jre/bin/java
 -Djava.endorsed.dirs=/usr/share/tomcat5/common/endorsed
 -classpath :/usr/lib/jvm/jre/lib/rt.jar:/usr/share/java/commons-
collections.jar:/usr/share/tomcat5/bin/bootstrap.jar:/usr/share/tomcat5/bin/commons-
logging-api.jar:/usr/share/java/mx4j/mx4j-impl.jar:/usr/share/java/mx4j/mx4j-
jmx.jar:/usr/share/tomcat5/common/lib/nuxwdog.jar -Dcatalina.base=/var/lib/pki-ca11
 -Dcatalina.home=/usr/share/tomcat5 -Djava.io.tmpdir=/usr/share/tomcat5/temp
 org.apache.catalina.startup.Bootstrap start

Here are all the children

[root@qantas]# pstree -A -p 21767
java(21767)-+-{java}(21768)
            |-{java}(21769)
            |-{java}(21770)
            <..snip..>
            `-{java}(22104)

This is what happens when I try to start ltrace on the text script

[root@qantas]# ltrace /usr/bin/dtomcat5-pki-ca11
ltrace: Can't open ELF file "/usr/bin/dtomcat5-pki-ca11"

When I attach ltrace to the parent and let it run for a day there is no output

[root@qantas]# ltrace -e open -o /tmp/ltrace.1295465058 -p 21767

...24 hours later...

[root@qantas]# ls -l /tmp/ltrace.1295465058
-rw-r--r-- 1 root root 0 Jan 19 11:24 /tmp/ltrace.1295465058

Best Answer

Unless you need to trace something in the startup process, strace and ltrace both have a -p parameter which attaches to an existing process and begins tracing it. Once tomcat is running, you would get the process id from ps, and run

strace -p 1234 -e open -o outputfile

or

ltrace -p 1234 -e open -o outputfile

where 1234 is the process ID.

The other option, if these "text files" are shell scripts, you should be able to

strace -f -e whatever -o whatever start-tomcat.sh

strace will begin tracing the shell executed to run the script, the -f will tell it to follow as it forks and executes each command. You'll need to filter through the output to figure out which process is which program (using -ff instead of -f will help).