Pacemaker - Azure Fence Agent "fence_azure_arm" failed to start after installing the package version python-azure-core-1.23.1-2.12.8

Following OS running in Microsoft Azure Cloud: 

SUSE Linux Enterprise Server for SAP Applica­tions 12 SP5
SUSE Linux Enterprise High Availability 12 SP5
 

For an Azure SLE 12 SP5 based instance the package version python-azure-core-1.23.1-2.12.8 was installed.

Afterwards the Azure Fence Agent (AFA) failed to start in a Pacemaker cluster and the following error is shown in the /var/log/messages file:
 

Jul 08 09:12:16 [2186] x00dhnp01l0f4b stonith-ng: info: internal_stonith_action_execute:    Attempt 1 to execute fence_azure_arm (monitor). remaining timeout is 580
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: notice: operation_finished:    fence_azure_arm_monitor_2:19654:stderr [ 2024-06-08 09:12:18,799 ERROR: Azure Resource Manager Python SDK not found or not accessible ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: notice: operation_finished:    fence_azure_arm_monitor_2:19654:stderr [ ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: notice: operation_finished:    fence_azure_arm_monitor_2:19654:stderr [ 2024-06-08 09:12:18,800 ERROR: Please use '-h' for usage ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: notice: operation_finished:    fence_azure_arm_monitor_2:19654:stderr [ ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: warning: log_action:    fence_azure_arm[19654] stderr: [ 2024-06-08 09:12:18,799 ERROR: Azure Resource Manager Python SDK not found or not accessible ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: warning: log_action:    fence_azure_arm[19654] stderr: [ ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: warning: log_action:    fence_azure_arm[19654] stderr: [ 2024-06-08 09:12:18,800 ERROR: Please use '-h' for usage ]
Jul 08 09:12:18 [2186] x00dhnp01l0f4b stonith-ng: warning: log_action:    fence_azure_arm[19654] stderr: [ ]

With the new fence-agents-azure-arm package being available, the following procedure has to be used to enable its functionality:

1. Install latest patches and reboot the instance
2. Verify the installed Python version
3. Install the fence-agents-azure-arm package
4. Basic tests
5. Cleanup the Pacemaker cluster resource


Example:

1. Install latest patches if required execute this command twice and reboot the instance:

x00dhnp01l0f4b:~ # zypper -n patch
x00dhnp01l0f4b:~ # zypper -n patch
x00dhnp01l0f4b:~ # reboot

2. Verify the installed Python version:

x00dhnp01l0f4b:~ # python --version
Python 2.7.18

x00dhnp01l0f4b:~ # python3 --version
Python 3.4.10

x00dhnp01l0f4b:~ # python3.6 --version
Python 3.6.15

3. Install the required fence-agents-azure-arm package:

x00dhnp01l0f4b:~ # zypper in fence-agents-azure-arm
.
.
Resolving package dependencies...

The following 3 NEW packages are going to be installed:
  fence-agents-azure-arm python3-azure-identity python3-pycurl

The following package is going to be upgraded:
  fence-agents

1 package to upgrade, 3 new.

Package download size:   543.8 KiB

Package install size change:
              |       1.5 MiB  required by to be installed packages
   849.4 KiB  |  -  673.1 KiB  released by to be removed packages  
Continue? [y/n/...? shows all options] (y): y
.
.
Checking for file conflicts: .......................................................................................................................................[done]
(1/4) Installing: python3-azure-identity-1.5.0-2.3.4.noarch ........................................................................................................[done]
(2/4) Installing: fence-agents-4.9.0+git.1624456340.8d746be9-3.41.3.x86_64 .........................................................................................[done]
(3/4) Installing: python3-pycurl-7.43.0-1.3.x86_64 .................................................................................................................[done]
(4/4) Installing: fence-agents-azure-arm-4.9.0+git.1624456340.8d746be9-3.41.3.noarch ...............................................................................[done]

4. Basic tests:

   Azure - managed identity (MSI)

x00dhnp01l0f4b:~ # fence_azure_arm --msi --action=list --subscriptionId=<subid> --resourceGroup=<rgname>
DEV-WEEU-SAP01-X00-x00dhnp01l0f4b,
DEV-WEEU-SAP01-X00-x00dhnp01l1f4b,

x00dhnp01l0f4b:~ # fence_azure_arm --msi --action=status --subscriptionId=<subid> --resourceGroup=<rgname> --plug=DEV-WEEU-SAP01-X00-x00dhnp01l0f4b
Status: ON

   Azure - service principal (SPN)

x00dhnp01l0f4b:~ # fence_azure_arm --action=list --subscriptionId=<subid> --resourceGroup=<rgname> --tenantId=<tenantid> --username=<username> --password=<password>
DEV-WEEU-SAP01-X00-x00dhnp01l0f4b,
DEV-WEEU-SAP01-X00-x00dhnp01l1f4b,

x00dhnp01l0f4b:~ # fence_azure_arm --action=status --subscriptionId=<subid> --resourceGroup=<rgname> --tenantId=<tenantid> --username=<username> --password=<password> --plug=DEV-WEEU-SAP01-X00-x00dhnp01l0f4b
Status: ON

5. Cleanup the cluster resource:

x00dhnp01l0f4b:~ # crm resource cleanup rsc_st_azure

Important notes:

Starting with the package version fence-agents-4.9.0+git.1624456340.8d746be9-3.41.3 the specific Azure Fence Agent binary (/usr/sbin/fence_azure_arm) is not included anymore.

For Microsoft Azure the new package fence-agents-azure-arm needs to be installed.
 
The new fence-agents-azure-arm package needs the Public Cloud Module to be activated, this is needed to provide the required Python 3.6 dependencies.

For SUSE Linux Enterprise Server 12 SP5 the fence-agents-azure-arm package is available after activating the SUSE Linux Enterprise High Availability Extension 12 SP5.

SLE 15 SP4+ related TID: https://www.suse.com/support/kb/doc/?id=000021504

 

The previous implementation of the Azure fence agent was still using Python 2.
The cloud SDK python packages were updated to be only available with Python 3.
The breakage leads to a broken SAP HA in Azure since the used fence-agents package for Azure needs the Azure SDK which is now on Python 3.6.

Top Issue