Summary
Sometimes when starting the opscenter agents from a service, the startup might not succeed
Symptoms
When starting the service the user might see the following on the command line after they start the service:
$ sudo service datastax-agent status
datastax-agent dead but pid file exists
Cause
The datastax agent monitor process may still be running even though the agent process is not. Typically you would see both:
root 32404 1 0 00:39 pts/0 00:00:00 /bin/bash /usr/share/datastax-agent/bin/datastax_agent_monitor
496 32426 1 5 00:39 ? 00:01:27 /usr/java/jdk1.7.0_67/bin/java -Xmx128M -Djclouds.mpu.parts.magnitude=100000 -Djclouds.mpu.parts.size=16777216 -Dopscenter.ssl.trustStore=/var/lib/datastax-agent/ssl/agentKeyStore -Dopscenter.ssl.keyStore=/var/lib/datastax-agent/ssl/agentKeyStore -Dopscenter.ssl.keyStorePassword=opscenter -Dagent-pidfile=/var/run/datastax-agent/datastax-agent.pid -Dlog4j.configuration=file:/etc/datastax-agent/log4j.properties -Djava.security.auth.login.config=/etc/datastax-agent/kerberos.config -jar datastax-agent-5.2.0-standalone.jar /var/lib/datastax-agent/conf/address.yaml
Solution
Ensure the datastax_agent_monitor process is killed and then restart the datastax-agent service
To kill a process, the command is usually:
$ kill <pid>
For example:
$ kill 32404
In some cases where the process still remains running, using a kill -9 will stop it immediately. For example:
$ kill -9 32404