Summary
This article discusses a known issue where DSE Search queries do not return map collection data with warnings in the logs with Cannot find dynamic field
.
Symptoms
DSE Search queries return incomplete results, specifically missing fields with map collection data. Warning messages are repeatedly reported in system.log
. Here are sample entries from a DSE 4.7.3 cluster:
WARN [http-8983-exec-1] 2016-07-28 16:45:43,172 Cql3CassandraRowReader.java:156 - Cannot find dynamic field : [home], please make sure [addresses] is being prepended/appended accordingly to build a correct dynamic field name. Did you mean addresseshome? homeaddresses? WARN [http-8983-exec-1] 2016-07-28 16:45:43,172 Cql3CassandraRowReader.java:156 - Cannot find dynamic field : [work], please make sure [addresses] is being prepended/appended accordingly to build a correct dynamic field name. Did you mean addresseswork? workaddresses?
Cause
This issue is due to data inserted into a table which contains a map collection with incorrect map keys.
Consider the following table of users with a collection of addresses:
cqlsh> CREATE TABLE users (name text, addresses map<text,text>, PRIMARY KEY (name));
The addresses
collection would have the following equivalent dynamic field in schema.xml
:
<dynamicField indexed="true" multiValued="false" name="addresses*" stored="true" type="TextField"/>
NOTE - In this instance, name="addresses*"
indicates that the field must be prefixed with "addresses*". For example:
cqlsh> INSERT INTO users (name, addresses) VALUES ( 'John', {'addresses_home' : '1 Smith St', 'addresses_work' : '10 Doe Blvd' } );
A Search query of this data returns the following results:
"response": { "numFound": 1, "start": 0, "docs": [ { "name": "John", "addresses_home": "1 Smith St", "addresses_work": "10 Main Blvd" } ] }
When data with incorrect map keys are inserted, a Search query has incomplete results with map data missing and generates warning messages in the logs. Here is a sample INSERT
with incorrect map keys:
cqlsh> INSERT INTO users (name, addresses) VALUES ( 'Jane', {'home' : '3 Doe St', 'work' : '30 Capitol Dr' } );
A Search query fails to return the map data for "Jane":
"response": { "numFound": 2, "start": 0, "docs": [ { "name": "John", "addresses_home": "1 Smith St", "addresses_work": "10 Doe Blvd" }, { "name": "Jane" } ] }
The warning messages above are also logged as a result of the Search query.
Solution
Check the data in the map collections and update the map keys to include either a prefix or suffix depending on the dynamic field schema definition.
In the example above, the record for "Jane" is fixed as follows:
cqlsh> INSERT INTO users (name, addresses) VALUES ( 'Jane', {'addresses_home' : '3 Doe St', 'addresses_work' : '30 Capitol Dr' } );
A Search query then returns expected results with no warnings in the logs:
"response": { "numFound": 2, "start": 0, "docs": [ { "name": "John", "addresses_home": "1 Smith St", "addresses_work": "10 Doe Blvd" }, { "name": "Jane", "addresses_home": "3 Doe St", "addresses_work": "30 Capitol Dr" } ] }
See also
DataStax doc - Using dynamic fields in DSE Search
DataStax doc - Advanced DSE Search tutorial