Summary
Checking network connectivity issues between OpsCenter and cluster node.
Applies to
All versions of OpsCenter.
Symptoms
There may or may not be any errors. In one case specifically, the error was:
Error saving cluster: Unable to connect to cluster.
Unable to connect to any seed nodes, tried [u'10.133.72.72']
And at the top of the OpsCenter screen:
Error initializing cluster data: Unable to load /AZUSC/cluster status: 404
Cause
Needs to be identified. Could be network connectivity issues, which is what we'll check here.
Solution
First, verify that all the ports are open on the nodes where the agents are running. You can check with this command, look for port 61621:
$ netstat -ntl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:9042 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8983 0.0.0.0:* LISTEN
tcp 0 0 192.168.56.30:8984 0.0.0.0:* LISTEN
tcp 0 0 192.168.56.30:7000 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:47802 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:7199 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:9160 0.0.0.0:* LISTEN
tcp 0 0 :::61621 :::* LISTEN
tcp 0 0 :::22 :::* LISTEN
tcp 0 0 ::1:25 :::* LISTEN
Then on the OpsCenter machine check the same for port 61620:
[datastax@OpsCenter ~]$ netstat -ntl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.1:61619 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:61620 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 :::22 :::* LISTEN
tcp 0 0 ::1:25 :::* LISTEN
Once you have established the ports are open, stop OpsCenter and the agents on one of the nodes, and use the "nc" tool to make sure we can pass a simple message across TCP from one machine to the other. The idea here is to see if there's any network issue preventing a TCP connection between the two hosts:
On the agent machine (substitute 192.168.56.30 for the local IP where the agent will listen):
$ nc -l 192.168.56.30 61621
On the OpsCenter machine (again substitute the IP):
$ nc 192.168.56.30 61621
Hello
(HIt enter after the message)
You should see the message appear on the agent machine:
$ nc -l 192.168.56.30 61621
Hello
You should also try this in the reverse direction too, its best to check both directions since OpsCenter listens on a different port.
Comments
0 comments
Please sign in to leave a comment.