How to evict tombstones on STCS without a major compaction

We want to reiterate why running a major compaction is not advisable. The main downside is that we will end up with one very large sstable that will almost never have a compaction partner in the future. For example if a table on a node has a total size of 500GB, running a major compaction on it means that we end up with a single 500GB sstable.

For SizeTieredCompactionStrategy, We would recommend temporarily reconfiguring the following table sub-compaction properties using the ALTER TABLE command:
- min_threshold - set to 2 so that only a minimum of 2 similar-sized sstables are required to trigger a minor compaction instead of the default 4
- tombstone_threshold - set to 0.1 so that if at least 10% of an sstable are tombstones, Cassandra will compact the table alone instead of waiting for the higher default ratio of 0.2
- unchecked_tombstone_compaction - set to true to allow Cassandra to run tombstone compaction without having to check if an sstable is eligible for compaction

We can make these settings more aggressive by temporarily lowering gc_grace_seconds to below the default 10 days (e.g. 259200 for 3 days). Here is an example command on how to use ALTER TABLE:

cqlsh> ALTER TABLE music.album_by_tracks WITH compaction = {'class': 'SizeTieredCompactionStrategy', 'min_threshold': 2}; 

NOTE - Create a backup of the schema before making any changes using the following command:

$ cqlsh -e 'DESCRIBE FULL SCHEMA' > schema.cql 

You should save a copy of the output schema.cql for future reference.

Please be aware that this is just a temporary measure and you should revert the configuration once the desired outcome has been achieved. After monitoring the compactions and you are satisfied that the tombstones have been compacted out, you should reset the sub-compaction properties for the relevant tables. For more information, see Compaction subproperties.