AssertionError thrown by Materialized View update of a null base row

Summary

This article discusses a scenario where updates to a materialized view fails with an assertion error.

Applies to

• DataStax Enterprise 5.0.0 and 5.0.1

Symptoms

In this example, a DSE 5.0.1 node fails to startup as a result of failures to replay the commit logs. Here is a sample entry from the system.log:

ERROR [main] 2018-06-25 12:34:56,789 CassandraDaemon.java:698 - Exception encountered during startup 
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError: We shouldn't have got there is the base row had no associated entry 
    at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:50) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:372) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.replayMutation(CommitLogReplayer.java:630) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:519) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:414) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:148) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    ...
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError: We shouldn't have got there is the base row had no associated entry 
    ...
Caused by: java.lang.AssertionError: We shouldn't have got there is the base row had no associated entry 
    at org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:485) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:394) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    at org.apache.cassandra.db.commitlog.CommitLogReplayer$1.runMayThrow(CommitLogReplayer.java:616) ~[cassandra-all-3.0.7.1159.jar:3.0.7.1159] 
    ...

Cause

The issue occurs when updating the materialized view (MV) when the row in the base table contains a null value, i.e. the clustering column has been deleted (CASSANDRA-12247). Instead of updating the existing row in the materialized view, no update should take place on the MV.

Workaround

In the case where the replay of the commit logs is preventing DSE startup:

Step 1 - Move the contents of the commitlog directory to another directory.

Step 2 - Start DSE.

Step 3 - Repair the node to synchronise with other replicas.

NOTE - This will only allow DSE to be started on the node. The underlying issue with the MV updates will continue in the background and get repeatedly reported in the logs.

Solution

The fix for CASSANDRA-12247 is included in DSE 5.0.2. Upgrade to the latest version of DSE 5.0 to take advantage of all the available fixes.