Full ERROR Message Example
INFO 11:46:05,414 StorageService.java:1337 - JOINING: waiting for schema information to complete
ERROR 11:46:06,414 MigrationManager.java:171 - Migration task failed to complete
ERROR 11:46:07,414 MigrationManager.java:171 - Migration task failed to complete
ERROR 11:46:08,414 MigrationManager.java:171 - Migration task failed to complete
What does this error mean?
The “migration tasks” basically means getting the schemas from other nodes. The node won't bootstrap unless the migration tasks complete first.
Why does this error occur?
During bootstrap, the node requests schema information from other nodes. This introduced a wait on the completion of in-flight migration tasks. This error is logged if the completion latch wait times out. The timeout is controlled by the cassandra.migration_task_wait_in_seconds system property and the default is 1.
How do you fix this error?
Generally this error is harmless and goes away as soon as all the migrations tasks are completed. Below info will confirm the migration tasks are completed and good to proceed:
INFO [main] 2020-04-07 11:50:28,420 MigrationManager.java:222 - Ready to bootstrap (no more in-flight migration tasks).
However, in case the bootstrapping node does not hear back from other nodes (for n number of reasons), it continues to throw the error and then finally prints below message which means one or more migration tasks have failed and as a result the bootstrap will fail as well:
Fatal schema migration error; unable to start server. See log for stacktrace.ERROR [main] 2020-09-10 11:19:57,836 CassandraDaemon.java:829 - Fatal schema migration error
org.apache.cassandra.exceptions.MigrationException: Can't bootstrap (all migration tasks failed)
To find the root cause of the issue, please follow below steps:
Look for the error in log file to figure out why the node did not hear back from other nodes. There could be plenty of reasons, like:
- The other nodes are busy in some activity and due to GC pauses, it is not replying to the bootstrapping node.
- The node is down which result in no communication between the nodes
- There is some networking issue in between both the nodes.
To fix the issue, please follow below steps:
- Fix the root cause because of which nodes could not communicate with each other.
- Check nodetool status output and make sure the node did not join the cluster.
- Shut the node down , remove all the data and re-bootstrap the node to the cluster.