Use of Filesystem resource for ASCS/ERS HA setup not possible

This document (000019944) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server for SAP Applications 15
SUSE Linux Enterprise Server for SAP Applications 12

Situation

The file /usr/sap/sapservices lists all SAP instances that could run on either cluster node. Certain SAP tools are relying on that information. The SAP hostagent and SAP start framework is starting as system service at boot time. In an HA cluster that SAP start framework service should not read the sapservices file.

The current configuration for SAP ASCS/ERS HA setups (ENSA1 or ENSA2)) is using instance work directories /usr/sap/<SID>/ASCS<nr>/ and /usr/sap/<SID>/ERS<nr>/ filesystems managed via dedicated cluster resources. This ocf:heartbeat:Filesystem resources are used for presenting or hiding profiles from the sapstartsrv running as part of the SAP start framework.

The resulting filesystem layout is complex. In public cloud environments NFS is used instead of shared storage. Project experience shows that NFS take-overs are failure prone.

Resolution

With SLES for SAP 15 the package sapstartsrv-resource-agents eliminates the need for dedicated instance work directory filesystems. Instead of that, a common NFS share can be statically mounted on all cluster nodes.

The packages contains a new resource agent ocf:suse:SAPStartSrv and auxiliary systemd services sapping and sappong. The resource agent manages sapstartsrv processes dedicated to each SAP instance. The systemd services are used to hide the /usr/sap/sapservices file from the SAP start framework at boot time.

In addition the package resource-agents has been updated. The package now contains an new ocf:heartbeat:SAPInstance resource agent, enhanced to optionally not call sapstartsrv for probing SAP instances.

The overall configuration from SUSE Setup Guides are still valid, except the instance work directories /usr/sap/<SID>/ASCS<nr>/ and /usr/sap/<SID>/ERS<nr>/ filesystems managed via dedicated cluster resources.

The solution is targeting fresh installations. Migration of existing shared-storage-based setups is out of scope for this document. More details and requirements are explained in manual page ocf_suse_SAPStartSrv(7).

Shown below is an example for the ASCS instance of an ENSA1 HA cluster. The example's SAP SID is SN2, instance name is ASCS, instance number is 00, virtual IP address is 192.168.178.188, virtual hostname is sapsn2as. The example is focusing on differences to the filesystem-based setup. Several items common for both setups are not shown. Of course an ERS is needed as well, but not shown here. The ERS instance can be configured in analogy to the ASCS.

1. Installing needed packages

The packages have to be installed and updated on all nodes.

# zypper up resource-agents
# zypper in sapstartsrv-resource-agents
# grep MINIMAL_PROBE=true /usr/lib/ocf/resource.d/heartbeat/SAPInstance

Setting MINIMAL_PROBE=true forces the resource agent to do only minimalcheck during a probe. This is needed for special file system setups. The MINIMAL_PROBE=true is only supported, if requested either by your vendor'ssupport or if described in an architecture document

2. Preparing OS basics

The OS basics have to be prepared on all nodes. Of course the NFS shares have to be prepared and exported on the NFS server. The NFS mount options are depending on the particular environment.

 # vi /etc/fstab
 ...
 nfs1:/s/SN2/sapmnt-SN2  /sapmnt/SN2  nfs rw,hard,intr,nolock,actimeo=1,proto=tcp 0 0 
 nfs1:/s/SN2/usr-sap-SN2 /usr/sap/SN2 nfs rw,hard,intr,nolock,actimeo=1,proto=tcp 0 0

 # mount -a
 # df -h /sapmnt/SN2 /usr/sap/SN2
 Filesystem              Size Used Avai Use% Mounted on
 nfs1:/s/SN2/sapmnt-SN2  3.0T 2.5T 613G 81%  /sapmnt/SN2
 nfs1:/s/SN2/usr-sap-SN2 3.0T 2.5T 613G 81%  /usr/sap/SN2

 # systemctl enable sapping sapinit sappong
 # systemctl status sapping sapinit sappong

* sapping.service - Hiding sapservices file from sapinit
   Loaded: loaded (/usr/lib/systemd/system/sapping.service; enabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:sapservices-move(8)
...
Feb 02 17:14:21 node2 systemd[1]: Started Hiding sapservices file from sapinit.

* sapinit.service - LSB: Start the sapstartsrv
   Loaded: loaded (/etc/init.d/sapinit; generated; vendor preset: disabled)
   Active: active (running) since Tue 2021-02-02 17:13:34 CET; 2 months 8 days ago
     Docs: man:systemd-sysv-generator(8)
    Tasks: 5
   CGroup: /system.slice/sapinit.service
           ├─2561 /usr/sap/hostctrl/exe/saphostexec pf=/usr/sap/hostctrl/exe/host_profile
           └─2718 /usr/sap/hostctrl/exe/saposcol -l -w60 pf=/usr/sap/hostctrl/exe/host_profile
...
Feb 02 17:13:34 node2 systemd[1]: Started LSB: Start the sapstartsrv.

* sappong.service - Unhiding sapservices file from sapinit
   Loaded: loaded (/usr/lib/systemd/system/sappong.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Tue 2021-02-02 17:13:34 CET; 2 months 8 days ago
     Docs: man:sapservices-move(8)
...
Feb 02 17:13:34 node2 systemd[1]: Started Unhiding sapservices file from sapinit.

Details are described in manual pages SAPStartSrv_basic_cluster(7) and sapping(7).

3. Configuring cluster resources

A basic cluster without SAP resources is expected to be up and running on all nodes. The resource configuration needs to be imported into the cluster's CIB only once. Differences to the former configuration are the resource SAPStartSRv replacing resource Filesystem, and the new parameter MINIMAL_PROBE=true for the resource SAPInstance.

# vi SN2_ASCS00_SAPStartSrv.txt
primitive rsc_SAPStartSrv_SN2_ASCS00 SAPStartSrv \
  params InstanceName=SN2_ASCS00_sapsn2

# vi SN2_ASCS00_SAPInstance.txt
primitive rsc_SAPInstance_SN2_ASCS00 SAPInstance \
  op monitor interval=11 timeout=60 on-fail=restart \
  params InstanceName=SN2_ASCS00_sapsn2 \
  START_PROFILE=/sapmnt/SN2/profile/SN2_ASCS00_sapsn2 \
  AUTOMATIC_RECOVER=false \
  MINIMAL_PROBE=true \
  meta resource-stickiness=5000 

# vi SN2_ASCS00_IPAddr2.txt
primitive rsc_IPAddr2_SN2_ASCS00 IPaddr2 \
  params ip=192.168.178.188

# vi SN2_ASCS00_group.txt
group grp_SN2_ASCS00 \
  rsc_IPAddr2_SN2_ASCS00 \
  rsc_SAPStartSrv_SN2_ASCS00 \
  rsc_SAPInstance_SN2_ASCS00 \
  meta resource-stickiness=3000

# crm configure load SN2_ASCS00_SAPStartSrv.txt
# crm configure load SN2_ASCS00_SAPInstance.txt
# crm configure load SN2_ASCS00_IPAddr2.txt
# crm configure load SN2_ASCS00_group.txt

# crm configure show grp_SN2_ASCS00 \
  rsc_IPAddr2_SN2_ASC00 rsc_SAPStartSrv_SN2_ASCS00 rsc_SAPInstance_SN2_ASCS00

group grp_SN2_ASCS00 \
  rsc_IPAddr2_SN2_ASC00 rsc_SAPStartSrv_SN2_ASCS00 rsc_SAPInstance_SN2_ASCS00 \
  meta resource-stickiness=3000
primitive rsc_IPAddr2_SN2_ASC00 IPaddr2 \
  params ip=192.168.178.188
primitive rsc_SAPInstance_SN2_ASCS00 SAPInstance \
  op monitor interval=11 timeout=60 on-fail=restart \
  params InstanceName=SN2_ASCS00_sapsn2as \
  START_PROFILE="/sapmnt/SN2/profile/SN2_ASCS00_sapsn2as" \
  AUTOMATIC_RECOVER=false MINIMAL_PROBE=true \
  meta resource-stickiness=5000
primitive rsc_SAPStartSrv_SN2_ASCS00 ocf:suse:SAPStartSrv \
  params InstanceName=SN2_ASCS00_sapsn2as

# crm_mon -1r

...
Online: [ node1 node2 ]

Full list of resources:

 rsc_stonith_sbd        (stonith:external/sbd): Started node2
 Resource Group: grp_SN2_ASCS00
   rsc_IPAddr2_SN2_ASC00      (ocf::heartbeat:IPaddr2):      Started node2
   rsc_SAPStartSrv_SN2_ASCS00 (ocf::suse:SAPStartSrv):       Started node2
   rsc_SAPInstance_SN2_ASCS00 (ocf::heartbeat:SAPInstance):  Started node2
...

The above configuration example is for the ASCS instance only. The needed ERS instance can be configured in analogy. A complete example is shown in manual page ocf_suse_SAPStartSrv(7).

For SLES for SAP 12 the resolution is to upgrade to 15.
On SLES for SAP 15 the here described solution can be applied.

Additional Information

man pages:

ocf_suse_SAPStartSrv(7)
SAPStartSrv_basic_cluster(7)
ocf_heartbeat_SAPInstance(7)
ocf_heartbeat_IPAddr2(7)
sap_suse_cluster_connector(8)

https://www.suse.com/c/simple-mount-structure-for-sap-application-platform/
https://documentation.suse.com/sbp/all/
https://www.suse.com/support/kb/doc/?id=000019293

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.