System exit to emergency shell at boot with multipath enabled
This document (000019607) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 12
Situation
[ OK ] Listening on multipathd control socket. Starting Device-Mapper Multipath Device Controller... [ OK ] Listening on Device-mapper event daemon FIFOs. Starting Device-mapper event daemon... Expecting device dev-disk-by\x2duuid-34be48b2\x2dc21...32dd9.device... Expecting device dev-sda2.device... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Expecting device dev-disk-by\x2duuid-1172afe0\x2d63c...5d0a7.device... Expecting device dev-disk-by\x2duuid-c4a3d1de\x2d4dc...ef77d.device... [ OK ] Started Create list of required static device nodes ...current kernel. Starting Create static device nodes in /dev... [ OK ] Started Collect Read-Ahead Data. [ OK ] Started Device-mapper event daemon. [ OK ] Started udev Coldplug all Devices. Starting udev Wait for Complete Device Initialization... [ OK ] Started Replay Read-Ahead Data. Starting Load Kernel Modules... Starting Remount Root and Kernel File Systems... [ OK ] Started Create static devices [ 13.682489] floppy0: no floppy controllers found [* ] (1 of 4) A start job is running for dev-disk-by\x2du...(7s / 1min 30s) [* ] (1 of 4) A start job is running for dev-disk-by\x2du...(7s / 1min 30s) ... Timed out waiting for device dev-disk-by\x2duuid-c4a...cfef77d.device. [DEPEND] Dependency failed for /opt. [DEPEND] Dependency failed for Local File Systems. [DEPEND] Dependency failed for Postfix Mail Transport Agent. Welcome to emergency shell Give root password for maintenance (or press Control-D to continue):This is the exit to the emergency shell
This issue is logical consequence of the multipath integration in systemd and occurs if the root filesystem is not on multipath but multipath is enabled.
In this setup multipath tries to setup its paths for all devices that are not blacklisted, as the device of the root filesystem is mounted already it is inaccessible causing multipath to fail.
Resolution
/etc/multipath.conf
Issue for example :
multipath -v2and identify what is the device for the root filesystem.
This will result in an output similar to:
Jul 29 10:10:03 | 3600508b1001030343841423043300400: ignoring map
The string between '| ' and ':' is the wwid, which should be added to /etc/multipath.conf as:
blacklist { wwid "3600508b1001030343841423043300400" }
Once this is added multipath will not try to access the wrong (the non-multipathed) device for the root file system during boot.
Another way of dealing with this would be to enable multipath in the initrd; re-run 'mkinitrd' and include the 'multipath' dracut module. Then the system will always come up multipathed, which is clearly not the intention if the root device is not multipathed.
As an immediate short term workaround one could also disable multipath in the root filesystem as this :
systemctl stop multipathd systemctl stop multipathd.socket systemctl disable multipathd
CAVEAT: Please keep in mind that this wwid is most likely local to the specific machine.
As such :
/etc/multipath.confshould then be removed from syncing of configuration files between different machines. Which applies for example to csync2.
Cause
Additional Information
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000019607
- Creation Date: 01-Aug-2014
- Modified Date:17-Jun-2021
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com