Monit Failed Protocol Test [HTTP] at [localhost]:8080 – Fix Guide

centos7monittomcat8

I get the error "monit failed protocol test [HTTP] at [localhost]:8080" every so often. The application, tomcat, has not stopped and is still running just fine. I know port 8080 is open because I can get to it in the browser on my personal PC ( so NOT localhost ). What else can I check to see what is happening?

Some logs in response to questions from someone else:

It happened again this morning at 3:26 or 3:27 am this morning, so here are, first of all, the monit log for the time in question:

[EDT May 19 03:26:01] info     : Reinitializing monit daemon
[EDT May 19 03:26:01] info     : 'newapp.turnsmith.com' Monit reloaded
[EDT May 19 03:26:01] error    : 'tomcat' failed protocol test [HTTP] at [localhost]:8080 [TCP/IP] -- HTTP error: Server returned status 404
[EDT May 19 03:26:01] info     : 'tomcat' start: '/etc/init.d/tomcat start'
[EDT May 19 03:26:01] error    : 'tomcat' failed protocol test [HTTP] at [localhost]:8080 [TCP/IP] -- HTTP error: Server returned status 404
[EDT May 19 03:27:01] error    : 'tomcat' failed protocol test [HTTP] at [localhost]:8080 [TCP/IP] -- HTTP error: Server returned status 404
[EDT May 19 03:27:01] info     : 'tomcat' start: '/etc/init.d/tomcat start'

And here is the log from http, error_log:

[Sun May 19 03:26:01.573204 2019] [auth_digest:notice] [pid 3625] AH01757: generating secret for digest authentication ...
[Sun May 19 03:26:01.573817 2019] [lbmethod_heartbeat:notice] [pid 3625] AH02282: No slotmem from mod_heartmonitor
[Sun May 19 03:26:01.573856 2019] [mpm_prefork:notice] [pid 3625] AH00163: Apache/2.4.6 (CentOS) configured -- resuming normal operations
[Sun May 19 03:26:01.573859 2019] [core:notice] [pid 3625] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

The error actually happens more, but we only get the "alert" email that one time.

Also, I don't know if this will help, but here is monit config for tomcat ( /etc/monit.d/tomcat ):

check host tomcat with address localhost
 stop program = "/etc/init.d/tomcat stop"
 start program = "/etc/init.d/tomcat start"
 if failed port 8080 and protocol http then alert
 if failed port 8080 and protocol http then start

Update 5/20/2019:
Based on @asktyagi advice, I changed my config to the below, seeing if it works:

check host tomcat with address localhost
 stop program = "/etc/init.d/tomcat stop"
 start program = "/etc/init.d/tomcat start"
 if failed port 8080 and protocol http retry 5 then alert
 if failed port 8080 and protocol http retry 5 then start

Best Answer

Can you check monit and http logs at the time of alert? some possibilities are: http port exhausted or http under pressure/busy. If so try to use RETRY. Edit: Also you can try to remove protocol and use like below if failed port {port number} then restart