Java – Performance issue on solaris 10 zone. Multiple pollsyscalls

dns-zonejavaperformancesolaris

We have slow running JAVA program on a Solaris 10 Zone. I ran a truss on the pid and noticed following messages getting generated even when program is not doing anything:

pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/29:    pollsys(0x00000000, 0, 0xFFFFFFFE242FF970, 0x00000000) = 0^M
/29:            timeout: 0.050000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/29:    pollsys(0x00000000, 0, 0xFFFFFFFE242FF970, 0x00000000) = 0^M
/29:            timeout: 0.050000000 sec^M
/22:    lwp_cond_wait(0x101585840, 0x101585828, 0xFFFFFFFE250FE960, 0) Err#62 ETIME^M
/63:    pollsys(0xFFFFFFFE135FE458, 1, 0xFFFFFFFE135FE380, 0x00000000) = 0^M
/63:            fd=46 ev=POLLIN rev=0^M
/63:            timeout: 0.100000000 sec^M

It is also preceeded/followed by:

lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M
/62:    pollsys(0xFFFFFFFE137FE4D8, 1, 0xFFFFFFFE137FE400, 0x00000000) = 0^M
/62:            fd=32 ev=POLLIN rev=0^M
/62:            timeout: 0.100000000 sec^M
/63:    lwp_mutex_wakeup(0xFFFFFFFF7EAF2AE0, 0)         = 0^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/37:    lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M
/37:    lwp_mutex_wakeup(0xFFFFFFFF7EAF2AE0, 0)         = 0^M
/61:    lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M

Would we know what this might mean? Anyway to figure more on it? I read we can run dtrace, but is there any other utility we can run to get where the slowness is happening. The app takes a lot of time to run basic queries, which works much smoother on a windows server.

Best Answer

Your best bet is to use the Java debugger and see what the the JVM threads are doing. It's really hard to tell from a system call trace what's going on.