Opscenter can not connect to the nodes

cassandraopscenter

Our opscenter can not connect to the nodes, in the logs the next error appears:

2015-08-17 08:56:37+0200 [SNPaware_Cluster] INFO: Nodes with agents that appear to be no longer running: 172.31.7.245, 172.31.7.244, 172.31.7.247, 172.31.7.246, 172.31.7.243
2015-08-17 08:56:37+0200 [SNPaware_Cluster] INFO: Nodes with agents that appear to be no longer running: 172.31.7.245, 172.31.7.244, 172.31.7.247, 172.31.7.246, 172.31.7.243
2015-08-17 08:56:38+0200 [] INFO: Stopping factory

We have disabled the firewall in all our nodes and opscenter (in fact we have another cluster configured which is actually working). The datastasx agents (version 2.0.5) are running in all the nodes but opscenter gets a timeout, the Cassandra version we are using is 2.0.16

Have anyone a hint on this?

Best Answer

At the end this is what it works for me.

Remove all the datastax agents in each node.
Delete the opscenter keyspace from the cluster.
Reinstall the datastax agents in each node.
Add the cluster to the opscenter througth the webinterface.
Run a repair on each node on the opscenter keyspace.

Related Solutions

OpsCenter 5.0: No Data on Dashboard Graphs

Figured it out! Found a bunch of the following in the agent logs:

WARN [Thread-1] 2014-08-08 15:07:06,527 Thrift operation queue is full, discarding thrift operation
 WARN [Thread-1] 2014-08-08 15:07:06,527 239710 operations dropped so far.
ERROR [thrift-processor-1] 2014-08-08 15:07:07,832 Error when proccessing thrift callme.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:You have not logged in)

This is kind of weird since all the other communication was fine, and the graphs initially worked. To fix it, I just added the following to the address.yaml file on all the agents and restarted the agent service:

thrift_user: <username>
thrift_pass: <password>

OpsCenter shows a datacenter as unknown even though the agents are connected

The datacenter is now properly displayed. What seems to have been a problem is the lack of Keyspaces replicated to the new datacenter.

During our first attempt to join the datacenter to the cluster, the OpsCenter Keyspace was still using SimpleStrategy and was replicated to the new nodes, but we changed it to NetworkTopologyStrategy before the second attempt.

So when we altered our business Keyspaces to be replicated on the new datacenter, the new DC was displayed in OpsCenter right away.

Best Answer

Related Solutions

OpsCenter 5.0: No Data on Dashboard Graphs

OpsCenter shows a datacenter as unknown even though the agents are connected

Related Topic