DataStax Help Center

Nodetool clearsnapshot fails on OpsCenter generated snapshots


When a backup is run from OpsCenter it creates a snapshot of the specified keyspaces.  If nodetool clearsnapshot is used to clear snapshots it will fail to delete some snapshot files and reports a failure to clear the snapshots with an error message similar to below.

Requested clearing snapshot for: all keyspaces 

Exception in thread "main" FSWriteError in /raid0/cassandra/data/system/batchlog/snapshots/opscenter_adhoc_2013-09-05-08-25-58-UTC/cf-info.txt 
at org.apache.cassandra.db.Directories.clearSnapshot( 
at org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot( 
at org.apache.cassandra.db.Table.clearSnapshot( 
at org.apache.cassandra.service.StorageService.clearSnapshot( 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke( 
at sun.reflect.DelegatingMethodAccessorImpl.invoke( 
at java.lang.reflect.Method.invoke( 
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2( 
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2( 
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM( 
at com.sun.jmx.mbeanserver.PerInterface.invoke( 
at com.sun.jmx.mbeanserver.MBeanSupport.invoke( 
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke( 
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke( 
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke( 
at java.lang.reflect.Method.invoke( 
at sun.rmi.server.UnicastServerRef.dispatch( 
at sun.rmi.transport.Transport$ 
at Method) 
at sun.rmi.transport.Transport.serviceCall( 
at sun.rmi.transport.tcp.TCPTransport.handleMessages( 
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0( 
at sun.rmi.transport.tcp.TCPTransport$ 
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask( 
at java.util.concurrent.ThreadPoolExecutor$ 
Caused by: Failed to delete /raid0/cassandra/data/system/batchlog/snapshots/opscenter_adhoc_2013-09-05-08-25-58-UTC/cf-info.txt 



When the snaphshots are created by OpsCenter the files "schema.json and cf-info.txt" are owned by the opscenter-agent user and group.  The Cassandra user won't have write access to these files as they are not owned by Cassandra and Cassandra is not in the opscenter-agent group.


To get around this issue you would add the 'cassandra' user to the 'opscenter-agent' group.  This will give write access to the files for the 'cassandra' user and allow the snapshot to be cleared.  After adding the 'cassandra' user to the  'opscenter-agent' group you may need to restart DSE so that the new permissions are applied for the DSE process.


*Note - This solution assumes that the DSE process is owned by the 'cassandra' user and that the the datastax agent process is run by the 'opscenter-agent' user belonging to the 'opscenter-agent' group.  If you have a alternative setup of users then you would need to apply privileges as required by your custom setup. 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Powered by Zendesk