Full ERROR Message Example
ERROR [DSE main thread] 2020-08-21 19:00:25,233 CommitLogReplayer.java:449 - Ignoring commit log replay error
Typically followed by the following exception that provides more details about why the error occurred.
ERROR [DSE main thread] 2020-08-21 19:00:25,233 CommitLogReplayer.java:449 - Ignoring commit log replay error
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: Could not read commit log descriptor in file /var/lib/cassandra/commitlog/CommitLog-680-1598035981105.log
at org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:203)
at org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:137)
at org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:157)
at org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:214)
at org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:195)
What does this error mean?
The error means that the commit log CommitLog-680-1598035981105.log is corrupt and cannot be read, and since the C* startup parameter is configured to ignore commitlog replay errors using: -Dcassandra.commitlog.ignorereplayerrors=true, the C* process is ignoring the issue with the commitlog and will continue starting the C* process.
Why does this error occur?
This error occurs because of the following possible issues causing commitlog corruption:
- There was a hardware failure such as bad sectors on the commit log disks.
- The server crashed while the filesystem was in the middle of syncing the file to disk
- The Cassandra process got interrupted mid-transaction, like by stopping C* by killing the C* process with kill -9 without nodetool flush and drain.
However, since the startup parameter is configured to ignore commitlog replay errors using: -Dcassandra.commitlog.ignorereplayerrors=true, the commitlog errors are being ignored, and the C* process is started.
How do you fix this error?
This error is an informative message that indicates the C* process found commitlog is corrupt and cannot be read. Your next steps should include:
- Determine why the commitlog is getting corrupted or cannot be read and fix it.
- Once C* is up and running, run repairs on the node to repair the missing mutations which did not get replayed as a result of the commitlog issues.