DataStax Help Center

Node bootup hung, cannot get past "Initializing system.local"


A node which has been re-provisioned or re-purposed, i.e. previously decommissioned from a running cluster, is unable to startup with the new cluster settings.

Some customers decommission some nodes in a cluster and decide to either use them to build another cluster or add them to an existing cluster. The nodes are reconfigured but hang when DSE is started.


DSE appears to go through the normal startup process but eventually hangs when it gets to the line Initializing system.local as in the log extract below:

INFO [main] 2014-11-03 18:21:35,351 (line 249) Initializing system.hints 
INFO [main] 2014-11-03 18:21:35,355 (line 249) Initializing system.schema_keyspaces
INFO [main] 2014-11-03 18:21:35,359 (line 249) Initializing system.range_xfers
INFO [main] 2014-11-03 18:21:35,364 (line 249) Initializing system.schema_columnfamilies
INFO [main] 2014-11-03 18:21:35,368 (line 249) Initializing system.NodeIdInfo
INFO [main] 2014-11-03 18:21:35,372 (line 249) Initializing system.paxos
INFO [main] 2014-11-03 18:21:35,379 (line 249) Initializing system.schema_columns
INFO [main] 2014-11-03 18:21:35,383 (line 249) Initializing system.IndexInfo
INFO [main] 2014-11-03 18:21:35,671 (line 249) Initializing system.peers
INFO [main] 2014-11-03 18:21:35,674 (line 249) Initializing system.local


This issue has occurred because the node has not been prepared correctly for the new cluster after it has been decommissioned.

Some of the known causes are:

  • data directory has not been cleared of its old contents
  • similarly, commitlog and saved_caches directories have not been emptied
  • one of the directories, e.g. commitlog, has been recreated with incorrect permissions and Cassandra is unable to write to it


Follow these steps to resolve the issue:

Step 1 - Shutdown DSE.

Step 2 - Delete the following directories. This will ensure that all their contents are deleted completely.

  • data
  • commitlog
  • saved_caches

Step 3 - Recreate the directories above and make sure Cassandra has full permissions.

Step 4 - Start DSE.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Powered by Zendesk