FAQ - Why Java performance utilities like jps no longer work in DSE 5.0+

Overview

This article provides information on why Java performance statistics are no longer available on DataStax Enterprise nodes.

Background

The Java virtual machine (JVM) collects and logs performance statistics which are used for monitoring and tuning activities. The performance data get collected and written to /tmp/hsperfdata on a node. Command-line utilities bundled with Java such as jps and jstat read this data to report a JVM's performance.

If there is no data or the utility does not have read permissions, information about the JVM is not available to report.

Cause

For Cassandra 2.2 and later, performance statistics data collection has been disabled for the JVM (CASSANDRA-9242) in the cassandra-env.sh:

JVM_OPTS="$JVM_OPTS -XX:+PerfDisableSharedMem"

This is the default for DSE 5.0 and later (Cassandra 3.0+).

This was done as a result of the discovery that when the JVM saves the data to a memory-mapped file on /tmp/hsperfdata, the operation is synchronous and gets blocked until the IO completes. The JVM is paused during this time, sometimes for hundreds of milliseconds significantly impacting GC performance.

This is discussed in an often-cited blog post from Evan Jones, "The Four Month Bug: JVM statistics cause garbage collection pauses". Disabling performance statistics collection ironically improves the performance of the JVM.

Next steps

Enable performance data collection on nodes only when actively monitoring performance by commenting out the following line in cassandra-env.sh:

JVM_OPTS="$JVM_OPTS -XX:+PerfDisableSharedMem"

Note that this will require a restart of DSE to take effect. It is recommended to disable it again when no longer actively monitoring the JVM's performance.

See also

Support KB - Java utilities such as jps or jstat unable to monitor DSE processes