DataStax Help Center

Apache Solr queries missing data when local Apache Cassandra node is consistent

Summary

In DSE releases after DSE4.7.1 and prior to 4.7.7 users may experience inconsistent results in their cluster. Although Apache Cassandra has consistent data across nodes, with repairs and reindexing correctly run, the data in Apache Solr is observed to be inconsistent. Only when the row is rewritten will the data appear in Solr correctly

Symptoms

Solr searches may show up with missing results, however a cqlsh query on the local node would show that the data is in the table. Running repair and re-indexing does not resolve the problem.

Using the luke request handler the index shows the following symptom using a good and bad example of a field called "createdDate":

The good query:

<lst name="createdDate">
<str name="type">tdate</str>
<str name="schema">I-S-----OF------</str>
<str name="flags">-TS-------------</str>
<str name="value">2015-11-13T12:19:48.474Z</str>
<str name="internal">1447417188474</str>
<float name="boost">1.0</float>
<int name="docFreq">1</int> <<-- doc frequency = 1
</lst>

The bad query:

<lst name="createdDate">
<str name="type">tdate</str>
<str name="schema">I-S-----OF------</str>
<str name="flags">-TS-------------</str>
<str name="value">2015-10-16T14:00:02.794Z</str>
<str name="internal">1445004002794</str>
<float name="boost">1.0</float>
<int name="docFreq">0</int> <<-- doc frequency = 0
</lst>

Cause

The problem was introduced in the following internal jira:

DSP-5948 - Make Search Node Bootstrapping More Efficient

The cassandra query pager being used was not respecting CQL row boundaries, meaning that a CQL row could be split in between pages causing the following problems:

1) In 4.7.x, the row is (re)indexed multiple times N, with N equal to the number of pages it is split into, and only the last page (subset of columns) is actually retained in the index, causing index data loss.

2) In 4.8.x, the row is just reindexed multiple times, but no data loss happens as the row is always reread from Cassandra (due to multithreaded wide row indexing introduced in 4.8.0).

 

Solution

The problem is fixed in the following internal jira:

DSP-7591 - Reindex doesn't correctly page through wide partitions

An upgrade to DSE4.7.7 or DSE4.8.x is required to get the above code fix

 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk