Loading field information in Graylog 2.1.x failed with status 500 (Internal Server Error)

elasticsearchgraylog

I'm working on setting up a Graylog and I'm getting the following error:

Loading field information failed with status: cannot GET http://192.168.2.108:12900/system/fields (500)

I am on this page:
http://docs.graylog.org/en/2.1/pages/getting_started/check_messages.html

I don't see the Histogram or Search result. However, I am getting message in from rsyslog from the many linux servers streaming to this stream.

This is my tailf:

tailf /var/log/graylog-server/server.log
2016-10-04T22:43:24.753-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:43:54.755-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:44:24.757-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:44:54.759-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:45:24.762-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:45:54.764-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:46:24.767-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:46:47.865-04:00 INFO  [IndexRetentionThread] Elasticsearch cluster not available, skipping index retention checks.
2016-10-04T22:46:54.770-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:47:24.772-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:47:54.774-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:48:24.777-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:48:54.779-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:48:58.490-04:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.<init>(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
        at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.<init>(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
        at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
        at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
        at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
        at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
        at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
        at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
        at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
        at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
        at org.graylog2.indexer.searches.Searches.search(Searches.java:244) ~[graylog.jar:?]
        at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
        at sun.reflect.GeneratedMethodAccessor190.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2016-10-04T22:49:24.783-04:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2016-10-04T22:49:28.481-04:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.discovery.MasterNotDiscoveredException
        at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:226) ~[graylog.jar:?]
        at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236) ~[graylog.jar:?]
        at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804) ~[graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

Ideas as to why I am not getting the same results as the documentation?

Best Answer

The log file tells your Elastic Search service is not up and running. You need to find out the reason why it is so by looking at its log and configuration.