Summary
100% CPU usage on a specific node in DataStax Enterprise
Applies to
- DataStax Enterprise (DSE) 6.0 and above.
Symptoms
- While checking the CPU usage of a specific node, a constant CPU usage of 100% was observed
- The following error can also be seen repeating constantly on the faulty node:
1234567891011ERROR [CoreThread-1] 2018-09-14 07:45:16,397 EpollTPCEventLoopGroup.java:730 - Task exception encountered:
java.lang.RuntimeException: Failed to process class org.apache.cassandra.db.ReadResponse$LocalResponse$$Lambda$850/425059782
at org.apache.cassandra.utils.LineNumberInference.maybeProcessClassInternal(LineNumberInference.java:229)
Caused by: java.io.FileNotFoundException: /tmp/dse/dse.FKb2MavbG/lni/org/apache/cassandra/db/ReadResponse$LocalResponse$$Lambda$850.class (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at org.apache.cassandra.utils.LineNumberInference.maybeProcessClassInternal(LineNumberInference.java:206)
... 59 common frames omitted
ERROR [CoreThread-1] 2018-09-14 07:45:16,397 CassandraDaemon.java:117 - Exception in thread Thread[CoreThread-1,10,main]
Cause
An issue was identified in the `LineNumberInference` class
Workaround
The issue was observed only once and a restart of the node resolved the issue. If the symptoms match, attempt a restart of the node to see if that resolves the issue.
Solution
This issue is referenced under DB-2568 and the fix was delivered in DSE 6.0.5 and DSE 6.7.0.