Dropping a field from Solr schema via CQL returns SyntaxException, InvalidRequest error

Summary

This article discusses command syntax issues when managing Solr indexes using CQL commands.

Applies to

• DataStax Enterprise 5.1 or later

Symptoms

DSE 5.1.0 introduced new CQL and cqlsh commands for managing DSE Search indexes (DSP-9762).

When attempting to drop a field from the Solr schema using CQL commands, the command in cqlsh either returns a syntax exception or an invalid request error. To illustrate the issue, here is a CQL schema of a table of users:

cqlsh> CREATE TABLE contacts.users (name text PRIMARY KEY, addresses map<text,text>);

A Search index exists on the table:

cqlsh> CREATE SEARCH INDEX ON contacts.users ;

with the following Solr schema:

cqlsh> DESCRIBE ACTIVE SEARCH INDEX SCHEMA ON contacts.users ;

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<schema name="autoSolrSchema" version="1.5">
  <types>
    <fieldType class="org.apache.solr.schema.StrField" name="StrField"/>
  </types>
  <fields>
    <field indexed="true" multiValued="false" name="name" type="StrField"/>
    <dynamicField indexed="true" multiValued="false" name="addresses*" type="StrField"/>
  </fields>
  <uniqueKey>name</uniqueKey>
</schema>

Consider a situation where a field with the incorrect name was added to the Solr schema. For example:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users ADD fields.field[@name='wrongname*', @type='StrField'] ;

The Solr schema now includes the field:

<schema name="autoSolrSchema" version="1.5">
  ...
  <fields>
    ...
    <field name="wrongname*" type="StrField"/>
  </fields>
</schema>

Dropping the field via CQL either returns a syntax exception or an invalid request error:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP field wrongname* ;
<stdin>:1:SyntaxException: line 1:64 mismatched input '*' expecting EOF (... contacts.users DROP field wrongname[*]...)

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP field 'wrongname*' ;
<stdin>:1:SyntaxException: line 1:55 no viable alternative at input 'wrongname*' (...INDEX SCHEMA ON contacts.users [DROP] field...)

Cause

In the first example above, the syntax exception was due to the name of the field containing an asterisk so it needs to be quoted so the CQL parser can handle it.

In the second example, the parser was not expecting the single quote (') after the field token.

Solution

The valid syntaxes for the DROP operation are:

  ALTER SEARCH INDEX SCHEMA ON [ks.]table DROP field <field_name>

  ALTER SEARCH INDEX SCHEMA ON [ks.]table DROP <element_path>[@attribute='<value>']

In the example above, the field name needs to be enclosed in double quotation marks since it contains a special character. The "short" command for removing it is:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP field "wrongname*" ;

Alternatively, use the long form of the command:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP fields.field[@name='wrongname*'] ;

NOTE - Remember to reload and rebuild the Search index when making changes to the schema:

cqlsh> RELOAD SEARCH INDEX ON contact.users ;

cqlsh> REBUILD SEARCH INDEX ON contact.users ;

More cases

EXAMPLE 2 - Wrong XML path

In this example, a field is added in the wrong part of the XML tree, where it is not nested inside the <fields> tag:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users ADD field[@name='wrongpath', @type='StrField'] ;

  <fields>
    ...
  </fields>
  <field name="wrongpath" type="StrField"/>

Using the short form of the command will return an invalid request error because the CQL processor assumes that the field is nested in the <fields> tag:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP field wrongpath;
InvalidRequest: Error from server: code=2200 [Invalid query] message= \
  "The search index schema could not be updated because: \
  Cannot drop resource element fields.field[@name='wrongpath'] \
  because it doesn't exist"

Use the long form of the command to explicitly set the element path to <field> only:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP field[@name='wrongpath'] ;

EXAMPLE 3 - Remove a dynamicField

To remove a dynamic field from the schema, use the long form of the command since the name contains an asterisk:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP fields.dynamicField[@name='dyna_*']

EXAMPLE 4 - Remove a copyField

To remove a dynamic field from the schema, use the long form of the command since both the source and destination fields need to be specified:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP copyField[@source='name', @dest='allfields'] ;

EXAMPLE 5 - Resolve dependencies

Removing a field which has a copyField associated with it will return an invalid request error:

cqlsh> ALTER SEARCH INDEX SCHEMA ON contacts.users DROP fields.dynamicField[@name='mapcopy*'];
InvalidRequest: Error from server: code=2200 [Invalid query] message="The search index schema is not valid because: Can't load schema schema.xml: copyField dest :'mapcopy*' is not an explicit field and doesn't match a dynamicField."

Remove the "children" elements before removing the target field.