NFS v3 performance degradation present after kernel update on SLES 12 or 15

SUSE Linux Enterprise Server 15 SP3 General maintenance and LTSS
SUSE Linux Enterprise Server 15 SP2 LTSS
SUSE Linux Enterprise Server 15 SP1 LTSS
SUSE Linux Enterprise Server 15 SP0 LTSS 
SUSE Linux Enterprise Server 12 SP5 General maintenance
SUSE Linux Enterprise Server 12 SP4 LTSS

A SLES 12 SP4 - SP5 or SLES 15 SP0 - SP3 NFS Server received maintenance updates which were released from December 2022 thru April 2023.  After the updates, NFS v3 clients using exports from those SLES NFS Servers experience slowness while reading files.  The larger the file, the more noticeable the performance loss is.  Reads of files 4096 bytes or smaller are not affected at all.

This does not affect NFS v4.x.

This does not affect SLES 15 SP4 (or 15 SP5 beta).  Refer to the "Environment" section above for the affected SPx levels.  Other variations of those SPx levels, such as SLES for SAP, Cloud, Real Time are also affected.

The issue has been corrected in newer maintenance kernels.  In May 2023, these fixed kernels were released:

SLES 15 SP3 LTSS kernel 5.3.18-150300.59.121
SLES 15 SP2 LTSS kernel 5.3.18-150200.24.151.1
SLES 15 SP1 LTSS kernel 4.12.14-150100.197.145.1
SLES 12 SP5 General kernel 4.12.14-122.159.1
SLES 12 SP4 LTSS kernel 4.12.14-95.125.1

Because they already reached end-of-life, SLES 15 SP0 LTSS and SLES 15 SP3 General Maintenance will not receive fixes.  See the "Additional Information" section for help identifying kernels which need or contain the fix, and other options that exist.
 

When the security fix for CVE-2022-43945 was backported into certain older SUSE Kernels, an error was introduced.  This was later corrected.  The original security fix in SLES 15 SP4 (kernel 5.14.21) did not have this problem.

Top Issue

To determine if the running kernel may have this problem, use the following commands (example commands and data returned from SLES 15 SP3 LTSS are shown):

To see what version and flavor of the kernel is currently running:

# uname -a
Linux dp-sles15sp3 5.3.18-150300.59.121-default #1 SMP Wed May 3 13:54:48 UTC 2023 (f717660) x86_64 x86_64 x86_64 GNU/Linux

The date shown above, "May 3 ... 2023," is the build date, and is a clue that the kernel is likely new enough to contain the fix.

To see the full version number of that kernel (and any others installed) use the command below.   Note that "default" may need to be replaced by whatever flavor is in use, as shown by the above "uname -a".  Two kernels are often shown because grub allows you to boot the current or previously installed kernel.

# rpm -q kernel-default
kernel-default-5.3.18-150300.59.118.1.x86_64
kernel-default-5.3.18-150300.59.121.2.x86_64

Search the kernel's change log for relevant information.  Use the kernel version number as reported by the above "rpm -q" command.  The version as reported by "uname -a" will not be complete enough.  If only one "kernel-default" is present, just specifying "kernel-default" (without the version string) will be enough:
[Output is shortened]

# rpm -q --changelog kernel-default-5.3.18-150300.59.121.2 | grep CVE-2022-43945
  (bsc#1205128 CVE-2022-43945 bsc#1210124).
  (bsc#1205128 CVE-2022-43945 bsc#1210124).
  (bsc#1205128 CVE-2022-43945).
  (bsc#1205128 CVE-2022-43945).

If the results include the longer entry "(bsc#1205128 CVE-2022-43945 bsc#1210124)" then the fix is present.  (Exception:  On SLES 15 SP4 or higher, this longer entry is not needed.)

If the results include ONLY "(bsc#1205128 CVE-2022-43945)" then the problem is present, not the fix.  (Exception:  On SLES 15 SP4 or higher, this does not signify that the problem is present.)

If the results do not include either of the above, then the kernel is likely older, before the problem was introduced.  However, it is also possible that the rpm command was entered incorrectly and no changelog was generated.  If no output is obtained, try the command again, replaceing "grep CVE-2022-43945" with "less" to verify that the changelog is actually being produced:

rpm -q --changelog kernel-default-5.3.18-150300.59.121.2 | less

NOTE: In a minority of cases, the problem was introduced in the last kernel available in a maintenance channel, and that channel reached end-of-life before the problem was discovered and fixed.  Those channels and further options include:

SLES 15 (SP0) LTSS

Kernel 4.12.14-150000.150.109.1, the last kernel released for SLES 15 (SP0) LTSS, has this problem.  No additional kernel will be released with a fix.  Choose one of these options:
a.  Move backwards to the previous LTSS kernel (4.12.14-150000.150.104.1).
b.  Upgrade to a Service Pack which was still being maintained as of May 2023, and apply current maintenance.
c.  Have clients mount with NFS v4.x instead of v3.

SLES 15 SP3 (General)

Kernel 5.3.18-150300.59.106.1, the last kernel released for SLES 15 SP3 General maintenance, has this problem.  No additional kernel will be released in general maintenance.  Choose one of these options:
a.  Move backwards to the previous kernel (5.3.18-150300.59.101.1).
b.  Subscribe to and register 15 SP3 LTSS to obtain long term support, and apply current LTSS maintenance.
c.  Upgrade to Service Pack 4 and apply current maintenance (though there is no specific fix needed for this issue, on SP4.)
d.  Have clients mount with NFS v4.x instead of v3.