DataStax Help Center

Read failures with "java.lang.IllegalArgumentException: Comparison method violates its general contract!"

Summary

This article relates to failures in Cassandra read operations as a result of a defect with the AbstractEndpointSnitch class in early versions of Cassandra.

Symptoms

Some read operations fail resulting in an IllegalArgumentException. Here is a sample error reported in the system.log:

ERROR [Native-Transport-Requests:44] 2015-12-14 10:59:00,641 ErrorMessage.java (line 210) Unexpected exception during request
java.lang.IllegalArgumentException: Comparison method violates its general contract!
at java.util.TimSort.mergeHi(TimSort.java:868)
at java.util.TimSort.mergeAt(TimSort.java:485)
at java.util.TimSort.mergeCollapse(TimSort.java:410)
at java.util.TimSort.sort(TimSort.java:214)
at java.util.TimSort.sort(TimSort.java:173)
at java.util.Arrays.sort(Arrays.java:659)
at java.util.Collections.sort(Collections.java:217)
at org.apache.cassandra.locator.AbstractEndpointSnitch.sortByProximity(AbstractEndpointSnitch.java:49)
at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithScore(DynamicEndpointSnitch.java:153)
at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithBadness(DynamicEndpointSnitch.java:171)
at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximity(DynamicEndpointSnitch.java:147)
...

Cause

This is due to a known defect in CASSANDRA-8448 where the snitch endpoint scores change while being sorted. Originally thought to have been solved, it was later discovered that the fix was incomplete and was not resolved until CASSANDRA-9519.

Issue was fixed in the following releases:

  • Cassandra 2.0.17 (DataStax Enterprise 4.6.11+)
  • Cassandra 2.1.9 (DataStax Enterprise 4.7.4+, 4.8.1+)
  • Cassandra 2.2.0

Workaround

It is possible to suppress the error by forcing the JVM to use a legacy sort methodology.

Follow the steps below to implement a temporary workaround for clusters experiencing the issue.

Step 1 - For the first node in the cluster, add the following JVM option in cassandra-env.sh:

JVM_OPTS="$JVM_OPTS -Djava.util.Arrays.useLegacyMergeSort=true"

Step 2 - Restart DSE on the node.

Step 3 - Repeat steps 1 and 2 one node at a time until all nodes in the cluster have been reconfigured.

Solution

As stated above, the workaround provided is not intended as a fix. For optimal operation of the cluster, upgrade to the following DataStax Enterprise releases which contain the fix:

  • DataStax Enterprise 4.6.11+
  • DataStax Enterprise 4.7.6+
  • DataStax Enterprise 4.8.3+

See also

Cassandra JIRA - CASSANDRA-8448 "Comparison method violates its general contract" in AbstractEndpointSnitch

Cassandra JIRA - CASSANDRA-9519 CASSANDRA-8448 not fixed

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk