System fails to boot or losses functionality after kernel update when third party kernel modules are used
This document (000020799) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 15 SP2
SUSE Linux Enterprise Server 15 SP3
SUSE Linux Enterprise Server 15 SP3
Situation
A system requires a Kernel Module Package (KMP) driver to boot.
After upgrading to a new kernel while using external kernel modules from a KMP package one of two scenarios appear:
Scenario One: The system fails to boot and drops into the dracut shell upon reboot
The system enters into maintenance mode after kernel update.
The dracut version is older than dracut-049.1+suse.238.gd8dbb075-150200.3.60.1
sh-4.4# dracut-cmdline -h
dracut-15-SP3 dracut-049.1+suse.234.g902e489c-150200.3.57.1
...
Instead of the third-party driver, the system booted with the in-box version of this driver. For example, if qla2xxx was required for boot:
sh-4.4# modinfo qla2xxx
...
supported: yes
NOTE: It would say "no" or "external" if it was the third party driver.
After booting the system from the SUSE installation media and attempting to chroot the installed system, the installed system's root device cannot be mounted (See How to Chroot in Rescue Mode).
Scenario Two: The system boots using SUSE's native in-box driver instead of the KMP driver causing functionality loss
The dracut version is older than dracut-049.1+suse.238.gd8dbb075-150200.3.60.1
# dracut -h | grep Version
Version: 049.1+suse.234.g902e489c-150200.3.57.1
Instead of the third-party driver, the system booted with the in-box version of this driver. For example, if qla2xxx was required for boot:
sles15sp3:~ # modinfo qla2xxx | grep -i 'supported:'
supported: yes
NOTE: It would say "no" or "external" if it was the third party driver.
After upgrading to a new kernel while using external kernel modules from a KMP package one of two scenarios appear:
Scenario One: The system fails to boot and drops into the dracut shell upon reboot
The system enters into maintenance mode after kernel update.
The dracut version is older than dracut-049.1+suse.238.gd8dbb075-150200.3.60.1
sh-4.4# dracut-cmdline -h
dracut-15-SP3 dracut-049.1+suse.234.g902e489c-150200.3.57.1
...
Instead of the third-party driver, the system booted with the in-box version of this driver. For example, if qla2xxx was required for boot:
sh-4.4# modinfo qla2xxx
...
supported: yes
NOTE: It would say "no" or "external" if it was the third party driver.
After booting the system from the SUSE installation media and attempting to chroot the installed system, the installed system's root device cannot be mounted (See How to Chroot in Rescue Mode).
Scenario Two: The system boots using SUSE's native in-box driver instead of the KMP driver causing functionality loss
The dracut version is older than dracut-049.1+suse.238.gd8dbb075-150200.3.60.1
# dracut -h | grep Version
Version: 049.1+suse.234.g902e489c-150200.3.57.1
Instead of the third-party driver, the system booted with the in-box version of this driver. For example, if qla2xxx was required for boot:
sles15sp3:~ # modinfo qla2xxx | grep -i 'supported:'
supported: yes
NOTE: It would say "no" or "external" if it was the third party driver.
Resolution
Avoiding Both Scenarios:
Install dracut-049.1+suse.238.gd8dbb075-150200.3.60.1 or greater prior to installing the KMP package or updating the kernel with the KMP package requirements.
Scenario One: Fails to boot and drops to the ram disk shell
Install dracut-049.1+suse.238.gd8dbb075-150200.3.60.1 or greater prior to installing the KMP package or updating the kernel with the KMP package requirements.
Scenario One: Fails to boot and drops to the ram disk shell
- You must find a way to chroot the installed system (see How to Chroot in Rescue Mode), which includes mounting the installed system's root device. Some possibilities include:
- Chroot the installed system from custom installation media that already includes the third party KMP driver.
- If the installed system's disk resides on shared media, like a storage area network (SAN), chroot the installed system from a working server that can access the shared media.
- Temporarily move the system disk into a working server and chroot the installed system there. When fixed, move it back to the original server.
- Assuming the system is registered for updates, activate the network (see SLE 15 - Configure a static network address while in the Rescue System ). Adjust the example to your environment:
- Run ip address add 192.168.1.200/24 dev eth0
- Run ip route add default via 192.168.1.254
- Run ip link set dev eth0 up
- Run mkdir -p /run/netconfig/
- Run echo 'nameserver 192.168.1.254' > /etc/resolv.conf
- NOTE: Instead of using the network, you could copy the updated dracut package to a USB disk and install from it.
- Update the dracut package
- Run zypper up dracut
- Reboot the server
- Run rm -rf /run/netconfig/
- Run exit
- Run reboot
- zypper up dracut
- dracut -h | grep Version shows dracut-049.1+suse.238.gd8dbb075-150200.3.60.1 or greater
- reboot
Cause
An issue with dracut failed to include the third party KMP driver in the ram disk.
Status
Reported to Engineering
Additional Information
Refer to the Recommend update for dracut SUSE-RU-2022:2644-1 announcement.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020799
- Creation Date: 05-Oct-2022
- Modified Date:18-Oct-2022
-
- SUSE Linux Enterprise Server
- SUSE Linux Enterprise Server for SAP Applications
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
