Solr 4.4 with tomcat 6 throws 503 on tomcat manager

503-errorsolrtomcat6

System Details:
OS: Cent OS 5.5 x86_64
java – 1.7 (1.7.0_25)
tomcat : 6.0.37
solr 4.4 (configured for multi-core)

I had solr 4.4 multicore running perfectly on jetty (although running just one core at the moment).

I need to run the same on tomcat 6 to see if there it can perform better than jetty.

Here is what I did:
1. downloaded tomcat 6
from : http://mirrors.sonic.net/apache/tomcat/tomcat-6/v6.0.37/bin/apache-tomcat-6.0.37.tar.gz

2. extracted apache-tomcat-6.0.37.tar.gz to /usr/local/tomcat/
created start up and shutdown scripts and the process runs through user:group tomcat:tomcat
check the start up scripts and found tomcat to run fine. I modified the port from 8080 to 8400

3. created a solr.xml under tomcat for the solr app:
file: /usr/local/tomcat/conf/Catalina/localhost/solr.xml

<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/var/www/projects/solr-oncars/example/solr/solr-4.4.0.war" debug="0" privileged="true" allowLinking="true" crossContext="true">
    <Environment name="solr/home" type="java.lang.String" value="/var/www/projects/solr-oncars/example/solr" override="true" />
</Context>

4. copy solr.war to core's home-path

cp /var/www/projects/solr-oncars/example/webapps/solr.war  /var/www/projects/solr-oncars/example/solr/oncars/

even tried with solr-4.4.war

cp /var/www/projects/solr-oncars/dist/solr-4.4.0.war /var/www/projects/solr-oncars/example/solr/

5. Edit solrconfig.xml
5.a : I have tried editing the datadir in solrconfig.xml
from : 

<dataDir>${solr.core0.data.dir:}</dataDir>

to: 

<dataDir>${solr.core0.data.dir:/var/www/projects/solr-oncars/example/solr/oncars/data}</dataDir>

5.b I have also tried after reverting the above setting

6. copy slf4j jars to tomcat:
Encountered lots of sl4j errors in catalina logs when I tried to start solr app from tomcat manager. This was resolved by copying sl4j libraries from solr to tomcat. As given below:

cp -v /var/www/projects/solr-oncars/example/lib/ext/slf4j-*  /usr/local/tomcat/lib/
cp -v /var/www/projects/solr-oncars/example/lib/ext/log4j-1.2.16.jar  /usr/local/tomcat/lib/

from 1 to 6 were done to execute existing solr from tomcat. Under Tomcat Manager, '/solr' shows as started. Stop and Start of the same does not throw errors.

I can see that solr folder is created under webapps in tomcat:

ll webapps/solr/

output:

-rw-r--r-- 1 tomcat tomcat 5642 Jul 10 11:11 admin.html
drwxr-xr-x 3 tomcat tomcat 4096 Sep 19 18:53 css
-rw-r--r-- 1 tomcat tomcat 1146 Jul 10 11:11 favicon.ico
drwxr-xr-x 4 tomcat tomcat 4096 Sep 19 18:53 img
drwxr-xr-x 4 tomcat tomcat 4096 Sep 19 18:53 js
drwxr-xr-x 2 tomcat tomcat 4096 Sep 19 18:53 META-INF
drwxr-xr-x 2 tomcat tomcat 4096 Sep 19 18:53 tpl
drwxr-xr-x 3 tomcat tomcat 4096 Sep 19 18:53 WEB-INF

When clicked on /solr from tomcat manager i.e. URL: http://localdomain.com:8400/solr/
The following page shows up:

HTTP Status 503 - Server is shutting down

type Status report

message Server is shutting down

description The requested service is not currently available.
Apache Tomcat/6.0.37

SOLR CONFIG:
/var/www/projects/solr-oncars/example/solr/solr.xml :

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
  <cores adminPath="/admin/cores">
    <core schema="schema.xml" name="oncars" instanceDir="oncars" conf="solrconfig.xml"/>
  </cores>
</solr>

I have tried looking around the web for help, but I couldn't find any that worked. Even resources from serverfault

Best Answer

The issue was probably with solr, since we had modified solrconfig initially and later even solr.xml (of solr) in the process of trial and error.

So, I did the whole thing again from ground zero. This time I referred another, rather relevant How-To

In sort, the mistake I made was to make changes to solr. Don't do that!
Solr Tomcat Wiki suggests: "Solr4.3 requires completely different deployment. These instructions are not current and are for an indeterminate version of Solr."
With wiki you tend to click on the index and miss fine print! I had referred it, therefore messed with working solr (lucky I had taken a back-up).

From the How-To I referred, I learned following things:

1 Don't make changes to a working solr config (i.e. run solr from jetty to check if it starts up fine)

2 Get tomcat up and running prior to adding solr in to it. (run both independently. Stop both the service if you are satisfied that both are fine)

3 Let's say solr path is /usr/local/solr-4.x, then copy solr-4.x.war from /usr/local/solr-4.x/dist/solr-4.x.x.war to /usr/local/solr-4.x/example/solr/solr.war . That's all you need to do in the solr directory. Rest, you need to take few files from solr directory in to tomcat directory (pt. 6)

4 Modify tomcat's server.xml and make sure you have URIEncoding in it:

<Connector port="8080" protocol="HTTP/1.1"
    connectionTimeout="20000"
    redirectPort="8443"
    URIEncoding="UTF-8" />

5 Create solr.xml in tomcat's conf : vim /usr/local/tomcat/conf/Catalina/localhost/solr.xml add following lines:

<Context path="/solr" docBase="/usr/local/solr/example/solr/solr.war" debug="0" crossContext="true">
        <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr/example/solr" override="true"/>
</Context>

note: solr cores are under the path: /usr/local/solr/example/solr/core{0,1,..n}

6 Copy Solr extensions files and log4j configuration file to tomcat:

cp -v /usr/local/solr/example/lib/ext/*  /usr/local/tomcat/lib/
cp -v /usr/local/solr-oncars/example/resources/log4j.properties /usr/local/tomcat/conf/

For the rest, cross-check with the 'How-To'

In the end, grant tomcat ownership to solr and tomcat directories (if tomcat run as user/group 'tomcat'):

chown -R tomcat:tomcat /usr/local/tomcat
chown -R tomcat:tomcat /usr/local/solr

Start up tomcat and access the solr-url, it worked!

Related Topic