DataStax Help Center

Lifecycle Manager Error: "meld.core.MeldException: PID file already exists: /var/run/meld.pid"

Summary

OpsCenter Lifecycle Manager (LCM) ensures that multiple jobs can't run on the same node concurrently through several mechanisms, including the creation of a pidfile on each target node while a job is executing. In rare cirumstances, this pidfile can be left behind after the job is stopped. When this happens, LCM jobs will fail on the problem node with an error that includes the message "meld.core.MeldException: PID file already exists: /var/run/meld.pid".

Diagnosis

In addition to the error above, one can confirm that this issue is occurring by verifying that there is no LCM-related processes running on the affected node.

  1. Verify that no job is running on the problem node, its datacenter, or its cluster by reviewing the LCM Job Status Overview in the LCM web interface.
  2. Log into problem node via SSH and ensure that the meld process is not currently running.
    1. 'cat /var/run/meld.pid' -> outputs a number indicated the process-id that LCM's meld process had when it was last running.
    2. 'ps -q ${process-id}' (substituting ${process-id} for the pid you got in step-one) -> outputs information about the process. If no process is returned, or if the process-name does not include the word "meld", then no job is running and you can safely delete the pidfile.

Note that more than one target node in a cluster can be affected by this issue, you may wish to check all nodes in the cluster in order to prevent repeated

Workaround

After verifying that there is no LCM or meld related process running on the affected node, you can safely remove the pidfile by running 'sudo /var/run/meld.pid'. Once this is complete for all affected nodes, rerun your LCM job.

Details

OPSC-8257 is the OpsCenter product development ticket that tracks progress on making pidfile handling more robust so that meld does not generate an error when a stale pidfile is present.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk