Summary
When bootstrapping a node, the bootstrap fails even though communications from the new node into the cluster appears to be ok
Symptoms
When bootstrapping the new node the following error is seen in the /var/log/cassandra/system.log
ERROR [main] 2016-03-03 19:19:44,911 CassandraDaemon.java:579 - Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
Checking the communications from the new node it is observed that the seed nodes can be reached ok (i.e. using ping). Even when checking using nc or telnet the default gossip port of 7000 (or 7001 if using SSL) the new node can reach the existing seed nodes.
Cause
Gossip communications must be bi-directional. The new node could not be reached from the existing seed nodes or any other nodes for that matter, thus preventing the bootstrap.
Testing
With the DSE process on the new node shutdown a simple test using nc is as follows:
On the new node run nc to listen on port 7000
nc -l 10.1.2.3 7000
On the existing seed node(s) run the following nc command
nc -v -w2 10.1.2.3 7000
A good connection:
Ncat: Connected to 10.1.2.3:7000.
A refused connection:
Ncat: Connection refused.
Solution
Investigation of the network connectivity between the existing nodes and the new node will need to be carried out. The bootstrap will only work correctly once the communications can be achieved on both directions.
Further reading
The following docs outline the ports used by Apache Cassandra:
https://docs.datastax.com/en/cassandra/2.1/cassandra/security/secureFireWall_r.html