Often users may wish to drill down on a query to find out where the most time is spent in different phases. It is also useful to be able to query the percentile latency of such a query too. This note outlines how to set this up and how to view the results.
The examples here will be outlined using "jconsole", however there are many other jmx querying tools available. In many cases where a user is accessing a production system it might not be possible to open JMX ports to remote connections. It is possible to use SSH tunneling and then connect the JMX client that way. However the methods of achieving this are not discussed in this note.
The setup of the query is actually already covered on the Datastax documentation site, see the following example:
In particular for query metrics:
1. Prepare the query
The first thing that needs to be setup is the query, as outlined in the docs it takes the following form;
http://<ip>:<port>/solr/<core>/select/?<query you need to measure>&query.name=<a distinct name for your query>
2. Run the query
The query needs to run enough times to start to produce some meaningful results, the more queries, the more accurate the results will be. Generally it's a good idea to run at least 1000 iterations. The solr stress tool might also be a good way of generating these.
3. Read the results
Using a JMX client you can query the QueryMetrics mbean to pull back the required percentile value. This can be done on each node as required. Put the phase in "p0", the query name in "p1" and then percentile in "p2"
Some further information and reading