Full ERROR Message Example

ERROR [CompactionExecutor:141824] 2019-03-16 22:37:37,869 StorageService.java:399 - Stopping native transportWhat does this ERROR message mean?

The error message indicates that the storage engine has encountered a critical issue and is shutting down the Cassandra process.

Why does this ERROR occur?

This error occurs if one of the storage subsystems encounters a fatal failure such as a filesystem error or a commitlog segment is not able to be read/written to. This relates to both DefaultFSErrorHandler.java class and CommitLog.java class whereby a number of potential issues could lead to system instability causing the storage engine to shutdown.

How do you fix this ERROR?

This is dependent on the specific error noted in the entire stacktrace provided in the startup.log, the system.log or debug.log.

Insufficient disk space available:

ERROR CompactionExecutor:141824 2019-03-16 22:37:35,790 CassandraDaemon.java:205 - Exception in thread ThreadCompactionExecutor:141824,1,main
 org.apache.cassandra.io.FSWriteError: java.io.IOException: No space left on device

You will need to confirm your current disk space usage for Cassandra. The default location of your Cassandra data related files is located under /var/lib/cassandra , check your cassandra.yaml for the location of your Cassandra data files in case a custom location has been specified for data, commitlogs, saved_caches, hints.

Run a df -h or a similar command to get the current disk space utilization on your node(s) and confirm there is sufficient disk space available for the Cassandra process to run.

This is also related to your current compaction strategies currently in use. See here for additional detail about compaction specific disk requirements and capacity planning

https://docs.datastax.com/en/dse-planning/doc/planning/capacityPlanning.html#Table2.Diskspacerequirementspercompactionstrategy

Commitlog errors:

ERROR [COMMIT-LOG-ALLOCATOR] 2015-11-20 00:30:07,259 StorageService.java:463 - Stopping native transport

In most cases the problem is a disk issue such as a hardware or permissions issues, or again a lack of disk space for the commitlogs which causes the native transport to stop.

Confirm that there is adequate disk space available, and check the permissions on the commitlog directory and make sure that the cassandra user (If using a package install) or the user starting the cassandra process (if using a tarball install) has the correct permissions to access the commitlog directory.

Check the syslog under /var/log/syslog if there is any noted errors about the disk that is being used as hardware errors or file system errors which are usually noted in the syslog and may help shed light on a disk related problem.

If you are unable to confirm the exact cause or experience different issues outside of what has been discussed please contact Datastax support.