DataStax Help Center

Opscenter is unable to delete commitlog when commitlog backup is enabled

Summary

In some circumstances it has been observed that datastax-agent is unable to delete the commitlog after the specified retention time.

Symptoms

This error is logged in the agent.log:


ERROR [clus] 2016-07-04 13:03:41,060 Unhandled route Exception: java.io.IOException: Couldn't delete /usr/share/datastax-agent/archive_commitlog.sh

 This error shows that the agent tried to delete the commtilog files from "/usr/share/datastax-agent/" directory, which is the datastax-agent installation directory. 

Another symptom of the problem is that "/usr/share/datastax-agent/" directory is empty of any regular file. 

Cause

The cause of this issue is in the missing directive in the <cluster_name>.log file:

[agents]
backup_staging_dir = 

If this directive is empty, the agent will try to delete the commitlog from its own installation directory, deleting all files in "/usr/share/datastax-agent/".

Datastax documentation explicitly says to configure this directive before enabling the commitlog backup with the retention time. However it could happen that a user forgets to do it or other third parties Software Configuration Management systems replace the file with an empty or old one.

Solution

Set the directive "backup_staging_dir" into the <cluster_name>.conf file before enabling commitlog backup and removal.

The internal jira OPSC-9393 has been opened in relation to this problem to improve the logging and the error message or warn the user for missing "backup_staging_dir" directive.

 

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk