Summary

Using a custom Cassandra system log file name leads to diagnostic tarball failure to download

Applies to

DataStax Enterprise 6.0 (and previous versions)
OpsCenter 6.5.3

Symptoms

Set a custom name for the system log in /etc/dse/cassandra/logback.xml. ie:

<file>${cassandra.logdir}/system-9161.log</file>

Go to OpsCenter UI and generate a diagnostic file via Help > Diagnostics
A pop up to provide the system.log file location /path/to/system.log appears
Following error shows in the agent.log:

ERROR [qtp1412832486-86] 2018-11-07 17:42:57,441 Can't access file system.log, locations checked: ["/var/log/cassandra/system.log"]. Errors: {"/var/log/cassandra/system.log" "File or directory denoted by path /var/log/cassandra/system.log does not exist"}

Based on the setup above, provide the below path and click “Save”:
/var/log/cassandra/system-9161.log
Effect in log:

INFO [async-dispatch-4] 2018-11-07 17:45:39,864 Configuration change for component class opsagent.environment.dynamic.DynamicEnvironmentComponent: before: {:location-conf {:cassandra_log_location "/var/log/cassandra"}}, after: {:location-conf {:cassandra_log_location "/var/log/cassandra/system-9161.log"}}

The agent restart
The “Download” button stays grayed out, so press the “Cancel” link
Attempt to generate the diag file.
The pop-up re-appear and following error shows up in the agent.log:

ERROR [qtp1412832486-87] 2018-11-07 17:49:29,439 Can't access file system.log, locations checked: ["/var/log/cassandra/system-9161.log/system.log" "/var/log/cassandra/system.log"]. Errors: {"/var/log/cassandra/system-9161.log" "Expected /var/log/cassandra/system-9161.log to have the following permissions: #{:read :execute}, but got #{:read :write}", "/var/log/cassandra/system.log" "File or directory denoted by path /var/log/cassandra/system.log does not exist"}

Cause

Custom system log filename isn't currently handled by OpsCenter

Solution

This issue is referenced as OPSC-15403 and is under review by development. Please click “Follow” on the top of the KBA to receive updates on this issue.

At the time of writing this article, the only viable solution is to use the default name "system.log" for diagnostics to be generated successfully. You can workaround the issue by creating a "fake" system.log name in the folder defined by the "[cassandra] log_location" option in the cluster_name.conf to generate the diagnostic file successfully. However, you will need to manually download the logs. Please also notify support when uploading the diagnostic tar ball.