cluster update failed in cib checksum (COROSYNC, HEARTBEAT)

This document (7012127) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise High Availability Extension 11 Service Pack 1
SUSE Linux Enterprise High Availability Extension 11 Service Pack 2

Situation

A rolling update or update to Pacemaker Version 1.1.7 or higher was performed and the correct packages are in place. There was no error during the update. The updated node on rejoining the cluster shows errors like

mynode cib: [29793]: ERROR: retrieveCib: Checksum of /var/lib/heartbeat/crm/cib.12345 failed! Configuration contents ignored!

in the logfile. This might result in the cib daemon no longer being able to write on-disk cib files.

Resolution

The cib should be trimmed before the update, one possible script for this would be

cibadmin -Q -o configuration | sed 's/^\s*//' | sed 's/\s*$//' | tr -d '\n' | sed 's/&#10;/\\n/g'| xmllint --copy - | cibadmin -R -o configuration -p

other than that the cib could be exported via

   crm configure save /PATH/TO/FILE

then edited, the cluster stopped and the cib on disk deleted and then the cleaned cib reinserted via

   crm configure load replace /PATH/TO/FILE

The later approach would require a downtime

Cause

From pacemaker-1.1.7, when reading on-disk cib files, the whitespaces will be stripped if there are any. That could result in this mis-matched checksum.

Additional Information

Thanks to Mr Dohei for the script

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.