Node in cluster will not start or join after being fenced.
This document (7022918) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise High Availability Extension 11 Service Pack 4
Situation
systemd[1]: Dependency failed for Pacemaker High Availability Cluster Manager.
Messages seen in SLES12 log after trying to start pacemaker.service and failing to start / join the cluster.
sbd: [34824]: WARN: Found fencing message - aborting start-up. Manual intervention required!
sbd: [34819]: WARN: Servant for pcmk (pid: 34826) has terminated
sbd: [34819]: WARN: Servant for <path to sbd device > (pid: 34824) has terminated
sbd.sh[34808]: sbd failed; please check the logs.
sbd.sh[34808]: SBD failed to start; aborting.
systemd[1]: sbd.service: Control process exited, code=exited status=1
Resolution
Need to find out <SBD_DEVICE> which can be found in /etc/sysconfig/sbd file.
Syntax to be run on node that was previously fenced:
Example: sbd -d <SBD_DEVICE> message LOCAL clear
You may also issue the command from any node in cluster by specifying the node name instead of "LOCAL"
Syntax: sbd -d <DEVICE_NAME> message <NODENAME> clear
Example: sbd -d /dev/sda1 message node1 clear
Once the node slot is cleared, you should be able to start clustering.
SLES114: rcopenais start
SLES12+: systemctl start pacemaker.service
Cause
Additional Information
SBD_STARTMODE=clean
## Default: always Options: always, clean
# Specify the start mode for sbd. Setting this to "clean" will only allow sbd to start if it was not previously fenced.
See the -S option in the man page.
Option for SLES11 /etc/sysconfig/sbd
SBD_OPTS="-W -P -S 1"
# -S 1 - set the start mode < Reference: man sbd >
# If set to one, sbd will only start if the node was previously shutdown cleanly (as indicated by an exit request message in the slot), or if the slot is empty. A reset, crashdump, or power-off request in any slot will halt the start up.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7022918
- Creation Date: 03-May-2018
- Modified Date:03-Mar-2020
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com