DataStax Help Center

Java driver queries result in "InvalidTypeException: Not enough bytes to deserialize <type>"

Summary

This article discusses a known issue where Java driver queries result in InvalidTypeException.

Symptoms

Read queries using the Java driver fail when deserialising objects.

Here is a sample driver exception when it tries to deserialise what it expects to be a map collection:

com.datastax.driver.core.exceptions.InvalidTypeException: Not enough bytes to deserialize a map 
at com.datastax.driver.core.TypeCodec$MapCodec.deserialize(TypeCodec.java:830)
at com.datastax.driver.core.TypeCodec$MapCodec.deserialize(TypeCodec.java:780)
at com.datastax.driver.core.ArrayBackedRow.getMap(ArrayBackedRow.java:299)
at com.datastax.driver.core.ArrayBackedRow.getMap(ArrayBackedRow.java:303)
...

Here is a another sample exception for an expected long type object:

com.datastax.driver.core.exceptions.InvalidTypeException: Invalid 64-bits long value, expecting 8 bytes but got 31 
at com.datastax.driver.core.TypeCodec$LongCodec.deserializeNoBoxing(TypeCodec.java:276) [cassandra-driver-core-2.0.8.jar:]
at com.datastax.driver.core.TypeCodec$DateCodec.deserialize(TypeCodec.java:608) [cassandra-driver-core-2.0.8.jar:]
at com.datastax.driver.core.ArrayBackedRow.getDate(ArrayBackedRow.java:108) [cassandra-driver-core-2.0.8.jar:]
at com.datastax.driver.core.ArrayBackedRow.getDate(ArrayBackedRow.java:112) [cassandra-driver-core-2.0.8.jar:]

Cause

The issue relates to the cache of prepared statements not in sync with the respective table's schema after a column is added or removed resulting in prepared statements with wildcard selections (SELECT * FROM table) reading the wrong column from the partition (CASSANDRA-7910, JAVA-420, CASSANDRA-10786).

It is important to note that performing a wildcard SELECT is bad practice since the result set is unpredictable with the schema being changed over time.

Workaround

One available workaround is to perform a rolling restart of DSE in order to get rid of the cached prepared statements.

But more importantly, rewrite queries such that they explicitly select columns from tables to avoid this problem and is best practice.

Solution

CASSANDRA-7910 has been fixed in Cassandra 2.1.3 where prepared statements are invalidated for tables where columns have been added or removed and force clients to update their local copy of the metadata.

However, there are still instances where the fix may be insufficient particularly when there are multiple clients connected to the same host. There is still further work being done to address this in JAVA-420 and CASSANDRA-10786.

This issue does not affect applications which are correctly listing the columns in the queries and that is still the best way to address this problem.

See also

Jira - [CASSANDRA-7910] "Incorrect wildcard prepared statements after a column is added"

Jira - [CASSANDRA-10786] "Include hash of result set metadata in prepared statement id"

Jira - [JAVA-420] "Prepared statement reads wrong column after columns are added to table"

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk