Logical Volumes Not Activated During Boot
This document (000021342) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 15 SP5
SUSE Linux Enterprise Server for SAP Applications 15 SP5
SUSE Linux Enterprise Server for SAP Applications 15 SP5
Situation
When booting a SLES 15 SP5 system, it may be observed that some logical volumes do not get activated, which can cause filesystems to not be mounted automatically as expected.
If these logical volumes correspond to filesystems in /etc/fstab that do not have _netdev or nofail specified as mount options, then the system may fail to boot and will go into emergency mode.
Messages such as the following are observed on the console during boot, with details corresponding to your filesystem and device:
If the system is able to boot, then activating the volumes manually works without making any other changes to the system.
The installed version of the lvm2 package starts with 2.03.16.
The filter setting in /etc/lvm/lvm.conf rejects paths directly to a device, but accepts a path that is a symlink to a device. An example filter setting that rejects /dev/dm-1, but accepts a symlink to it under /dev/disk/by-id/ is shown below.
Journal entries such as the following can be found either from emergency mode or from the booted system:
If these logical volumes correspond to filesystems in /etc/fstab that do not have _netdev or nofail specified as mount options, then the system may fail to boot and will go into emergency mode.
Messages such as the following are observed on the console during boot, with details corresponding to your filesystem and device:
[ TIME ] Timed out waiting for device /dev/mapper/mpathvg-mpathlv. [DEPEND] Dependency failed for /multipath
If the system is able to boot, then activating the volumes manually works without making any other changes to the system.
# vgchange -ay mpathvg 1 logical volume(s) in volume group "mpathvg" now active
The installed version of the lvm2 package starts with 2.03.16.
# rpm -q lvm2 lvm2-2.03.16-150500.7.6.1.x86_64
The filter setting in /etc/lvm/lvm.conf rejects paths directly to a device, but accepts a path that is a symlink to a device. An example filter setting that rejects /dev/dm-1, but accepts a symlink to it under /dev/disk/by-id/ is shown below.
# grep -E "^[[:space:]]*filter" /etc/lvm/lvm.conf filter = [ "a|/dev/disk/by-path/pci-0000:08:00.0|", "a|/dev/disk/by-id/dm-uuid-.*mpath-.*|" , "r|.*|"]
# ls -l /dev/dm-1 brw-rw---- 1 root disk 254, 1 Jan 30 16:01 /dev/dm-1
# ls -l /dev/disk/by-id/dm-uuid-mpath-360014056fff5e15713144e991a0b4ca1 lrwxrwxrwx 1 root root 10 Jan 30 16:01 /dev/disk/by-id/dm-uuid-mpath-360014056fff5e15713144e991a0b4ca1 -> ../../dm-1
Journal entries such as the following can be found either from emergency mode or from the booted system:
# journalctl --no-pager -b 0 | grep lvm (...) Jan 30 15:51:49 localhost lvm[1667]: finding all devices for filter symlinks. Jan 30 15:51:49 localhost lvm[1667]: /dev/dm-1 excluded: device is rejected by filter config. (...)
Resolution
If the system is able to boot, then use zypper to upgrade to an lvm2 package that is version 2.03.22 or later.
If the system will not boot, then additional steps must be taken to get the system to boot first, then the above resolution can be applied. These may vary from system to system depending on the scenario. One workaround that is likely to get your system to boot can be found below under the heading "Additional Information".
# zypper up lvm2
# rpm -q lvm2 lvm2-2.03.22-150500.7.9.1.x86_64
If the system will not boot, then additional steps must be taken to get the system to boot first, then the above resolution can be applied. These may vary from system to system depending on the scenario. One workaround that is likely to get your system to boot can be found below under the heading "Additional Information".
Cause
An optimization was made in the upstream lvm2 project that is present in SUSE's lvm2-2.0.3.16-* packages. This optimization causes event-based auto-activation of volumes to fail under certain conditions such as those described here. This problem was identified and fixed in a later upstream release, and that fix is found in SUSE's updated lvm2 packages which were re-based to use the upstream 2.03.22 code.
Additional Information
If the server ends up in emergency mode as a result of this problem, then we have to get it to boot normally before the fix can be applied.
It is very unlikely that the server will be unable to mount / and /usr since the devices for these filesystems are activated during the initrd phase of boot using a different activation method that is not subject to this issue. This means that it is probable that the system will be able to boot temporarily by commenting out other entries in /etc/fstab.
For example, if the logical volume that will not activate is /dev/mapper/testvg-testlv, then from emergency mode you can comment the corresponding entry in /etc/fstab by adding a # in front of it. The resulting entry would look like this:
Once all of the affected filesystems are commented, you can reboot the server. It should get past emergency mode this time, allowing you to upgrade lvm2 as described above.
Once the new lvm2 package is installed, the filesystem entries in /etc/fstab can be un-commented by removing the #, and the server can be rebooted normally.
It is very unlikely that the server will be unable to mount / and /usr since the devices for these filesystems are activated during the initrd phase of boot using a different activation method that is not subject to this issue. This means that it is probable that the system will be able to boot temporarily by commenting out other entries in /etc/fstab.
For example, if the logical volume that will not activate is /dev/mapper/testvg-testlv, then from emergency mode you can comment the corresponding entry in /etc/fstab by adding a # in front of it. The resulting entry would look like this:
#/dev/mapper/testvg-testlv /mnt xfs defaults 0 0
Once all of the affected filesystems are commented, you can reboot the server. It should get past emergency mode this time, allowing you to upgrade lvm2 as described above.
Once the new lvm2 package is installed, the filesystem entries in /etc/fstab can be un-commented by removing the #, and the server can be rebooted normally.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000021342
- Creation Date: 31-Jan-2024
- Modified Date:07-Feb-2024
-
- SUSE Linux Enterprise Server
- SUSE Linux Enterprise Server for SAP Applications
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
