This article discusses the change in behaviour for the OpsCenter 6.1 Performance Service where historical data for slow query log is no longer displayed for clusters running DataStax Enterprise 5.1 (or later).
When using OpsCenter 6.1 to monitor DSE 5.1 cluster, the slow query log data does not survive an agent restart. For this configuration, slow query log data displayed by OpsCenter reflects slow queries seen since the agent was last started rather than all slow queries seen within the slow query log TTL.
In older versions of DataStax Enterprise, CQL queries taking longer than the configured threshold were logged to the
dse_perf.node_slow_log table. For some clusters, this led to a significant performance degradation and so from DSE 5.1, logging slow queries to Cassandra is no longer the default behaviour. Instead DSE 5.1 maintains an internal cache of the most recent slow query entries, exposing them via JMX (internal enhancement IDs DSP-5088, DSP-11171). Even for cases when slow query logs are written to Cassandra, the cached entries will still be available via JMX.
In-line with this DSE enhancement, OpsCenter 6.1 always retrieves data for the slow query log from JMX (internal enhancement ID OPSC-11702) for DSE 5.1+ clusters. Note that this holds true even if DSE is configured to log slow queries to Cassandra. The DataStax agent will not consider data stored in Cassandra and will only look at the cached data exposed via JMX. For all DSE versions prior to 5.1, the agent will use only data stored in Cassandra.
In all cases the agent maintains an internal model of slow query data it has seen, always keeping the slowest queries in sorted order within that model. DSE 5.1 provides a stream of new data to that model via JMX while prior versions of DSE provided data by way of regular CQL queries. The crucial difference comes from the fact that in older versions of DSE, slow query entries were written to Cassandra which meant the model could be seeded with previous data (subject to the configured TTL). The DSE 5.1 implementation contains no actor that has access to this historical data and thus has no way to seed the model.