Best practice for applying Program Temporary Fixes (PTFs)

This document (7016640) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server

Situation

SUSE Technical Support provided a set of packages to fix a given situation (so called PTF = Program Temporary Fix).

Please note that such a PTF is fully supported by SUSE until the respective Maintenance Update containing the fix (can be reviewed within the changelog) has been released via the regular update repositories.
Important: A PTF automatically is based on the latest available Maintenance Update!
It also is important to understand the difference between a TEST package and a PTF. See below at the end of this document to learn about that.
The download directory of the PTF is provided by the Support Engineer assigned to the case and is located on
https://ptf.suse.com

To download, the PTF login credentials (different from the SCC login credentials) are necessary. These are identical with SUSE Bugzilla Login and are handled by:

https://idp-portal.suse.com

Customers who had active accounts during 2018 and 2019 were migrated and just need to set a new password at

https://idp-portal.suse.com/univention/self-service/#page=passwordreset

In case password reset doesn't work, kindly create new account at

https://idp-portal.suse.com/univention/self-service/#page=createaccount

Migration details can be found at

https://idp-portal-info.suse.com

Now the question is: How should a PTF be applied to a server? Please see the next section.

Resolution

Usually package management tasks are executed via YaST2 or zypper. These tools fetch the packages from given installation media or online repositories.

Program Temporary Fixes on the other hand are not distributed as repositories but as plain http directories and as such require a download of the packages. The easiest and most convenient solution is to download the whole
http directory via wget and to install the packages in the folder using rpm -Fvh. This procedure ensures all packages are being fetched but only those already installed on the system will be updated.

Note of caution : Please pay extra attention to the following procedure in case of Kernel PTF's.
*never* use 'zypper' and *always* use 'rpm' as detailed in the 'Special Case : kernel RPMs' section below when installing kernel PTF's

The procedure would be:

Create a folder "PTF":
mkdir /root/PTFinstall-$(date +"%F-%H-%M")
Change into the previously created directory:
cd $_
Download the provided URL recursively using wget:
wget --no-directories --recursive --reject "index.html*" --user=<SUSE_IDP/Bugzilla_account_username> --ask-password --no-parent <URL>

NOTE: If the above <URL> is pointing to a directory, be sure the URL ends with a trailing slash (/). Otherwise, wget can interpret it as a file, and may traverse up into the parent directory and then down other subdirectories. This is true despite the use of "--no=parent" on the command line.
Install the RPMs (apply kernel and other RPM's as per the 'Special Case' notes below):
rpm -Fvh *rpm

Alternatively you could also zypper:

zypper install <rpm-name>

The advantage is that zypper will also check other dependencies, which rpm will not do. The biggest disadvantage is that all packages in the directory will be installed and often from a PTF not all packages are needed (e.g. source and debug). When using "rpm -Fvh" only the packages that are installed will be updated.

Special Case : kernel RPMs

While the above procedure is very convenient, special attention should be paid to kernel RPMs. The used rpm -F option will replace the existing RPM and hence a preinstalled kernel will be replaced by the new one. This behaviour may not be desired and the TEST-/PTF should get installed in parallel to the existing one.

In this case, please identify which kernel RPMs have been installed previously (rpm -qa | grep kernel-) and install the PTF-RPMs using

rpm -ivh rpm1 rpm2 rpm3 ...

As there may be several kernel flavors in an architecture specific HTTP directory, please pay special attention to the kernel flavors and only install those kernel RPMs that are needed.

Special Case: SUSE Manager 5.x and SLE Micro

With SUSE Manager 5.x running on top of SLE Micro 5.5, there are two types of PTFs, container based PTFs and RPM based PTFs for the SLE Micro core operating system.

For container PTFs, see the TID "How to apply a Program Temporary Fix (PTF) to a containerized SUSE Manager 5 ".

For the core SLE Micro operating system RPM based PTFs, take the following steps:

1. Create a directory for the PTF:
mkdir /root/PTFinstall-$(date +"%F-%H-%M")
cd PTFinstall-<date_and_time>/

2. Download the PTF RPM files using the provided PTF URL. Aside from manually downloading the files, wget can also be used, note however this is not installed by default and if using wget is desired, it will need to be installed first, which will require an additional reboot.

3. Install the PTF RPMs:
transactional-update pkg in *PTF*.rpm

4. Reboot.

Special Case on CaasP product RPMs

When running updates through the Velum dashboard interface *after applying a PTF*, it is possible that zypper removes the PTF again.
When that happens, do the following :

Install the PTFs with the command :
transactional-update pkg install *PTF*rpm
or
transactional-update ptf install *PTF*rpm

In the case "Signature verification failed" messages are observed, please use "i" to ignore, otherwise the PTF will not be installed.

Reboot the admin note manually
# reboot
Once the system is back up, verify the rpm is installed with :
rpm -qa |grep PTF
Create a lock on the package using zypper, so that the next update will not overwrite the PTF with a possible problem version again.
zypper al <PTF_PACKAGE_NAME>
Verify the lock was properly set using :
zypper ll

Note: To install the PTF on large amount of systems, SUSE Manager Custom Channels can be used. Kindly refer to SUMA documentation .

ATTENTION: Once the PTF is superseded with a regular maintenance update, and that update is released / included in the regular update repositories, please make sure to remove the lock that was set on the package before applying these updates, so the package(s) can be properly updated again going forward.

Additional Information

How to install a kernel RPM:

Situation:

1. The packages have been downloaded using the above described procedure and the directory contains the following files:

sles11sp3:~/PTFinstall-2016-05-26-10-00 # l
total 1159804
drwxr-xr-x  2 root root      4096 May 26 10:07 ./
drwx------ 10 root root      4096 May 26 10:00 ../
-rw-r--r--  1 root root   7058123 May 12 13:59 kernel-default-3.0.101-0.47.79.1.10806.1.PTF.974428.nosrc.rpm
-rw-r--r--  1 root root  24281811 May 12 13:59 kernel-default-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  15551478 May 12 13:59 kernel-default-base-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root 303585616 May 12 13:59 kernel-default-debuginfo-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  52651476 May 12 13:59 kernel-default-debugsource-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root   6353063 May 12 13:59 kernel-default-devel-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  26063477 May 12 13:59 kernel-default-devel-debuginfo-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  14489809 May 12 13:59 kernel-default-extra-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root   4759014 May 12 13:59 kernel-default-hmac-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root 119311862 May 12 12:30 kernel-source-3.0.101-0.47.79.1.10806.1.PTF.974428.src.rpm
-rw-r--r--  1 root root  91798255 May 12 12:30 kernel-source-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  88043183 May 12 12:30 kernel-source-vanilla-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root   7057543 May 12 14:46 kernel-xen-3.0.101-0.47.79.1.10806.1.PTF.974428.nosrc.rpm
-rw-r--r--  1 root root  24034382 May 12 14:46 kernel-xen-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  14660596 May 12 14:46 kernel-xen-base-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root 286017607 May 12 14:46 kernel-xen-debuginfo-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  51501488 May 12 14:46 kernel-xen-debugsource-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root   6292081 May 12 14:46 kernel-xen-devel-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  23449988 May 12 14:46 kernel-xen-devel-debuginfo-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root  14508510 May 12 14:46 kernel-xen-extra-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm
-rw-r--r--  1 root root   4754534 May 12 14:46 kernel-xen-hmac-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm

2. Identify the installed kernel RPMs

sles11sp3:~/PTFinstall-2016-05-26-10-00 # rpm -qa | grep kernel-
kernel-default-base-3.0.101-0.47.71.1
kernel-firmware-20110923-0.52.3
kernel-default-3.0.101-0.47.71.1

3. Install the required packages only:

rpm -ivh kernel-default-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm kernel-default-base-3.0.101-0.47.79.1.10806.1.PTF.974428.x86_64.rpm

As the kernel-firmware was was not provided, it does not need to get updated. As mentioned, this will install the PTF kernel beside the existing kernel and not replace it so it can act as fallback. So a rpm -qa | grep kernel- will look afterwards:

sles11sp3:~/PTFinstall-2016-05-26-10-00 # rpm -qa | grep kernel-
kernel-default-3.0.101-0.47.79.1.10806.1.PTF.974428
kernel-default-base-3.0.101-0.47.71.1
kernel-default-base-3.0.101-0.47.79.1.10806.1.PTF.974428
kernel-firmware-20110923-0.52.3
kernel-default-3.0.101-0.47.71.1

Kindly check TID 000018545 for information on How to import suse_key for signed PTF packages

Note the difference between a TEST package and a PTF

Before releasing a fully supported PTF package engineering may decide to run some TEST packages, either to gather further information on the defect by enabling debugging, or to determine if the chosen fix is on the spot.

Those package names contain a TEST suffix to differentiate them from a PTF.
TEST packages are not to be used in production unless advised by support staff.
One reason is that TEST packages are not supported, whereas a PTF is fully supported.
In order to fix a bug it may be necessary to run multiple TEST packages to determine the best fix.

Roughly the order of events to fix a bug is:

Support qualifies and ideally reproduces the issue reported and hands over to engineering.
Engineering fixes the issue and produces a TEST package if necessary
Customer gets the TEST package from support and confirms the issue as fixed or not
Based on customer feedback Engineering releases a fully supported PTF
Customer reports issue as fixed by the PTF
Engineering hands over to Maintenance
Maintenance invokes QA and releases the fix via official repositories.

The PTF is fully supported until a Maintenance Update containing it's fix is released via the regular update repositories.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.