How to Chroot in Rescue Mode

This document (7018126) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11

Situation

The system is unable to boot and the only way to run the necessary fix is to chroot into it from rescue mode on the media.

Resolution

1. First boot the system into Rescue Mode from the media. For best results use media that matches the version and service pack of the system.

2. Once it's finished loading, log in as root.

3. Mount the root partition or logical volume to /mnt

mount /dev/<device name> /mnt

If it's not known where the root file system is, it needs to be guessed until found by mounting each possibility and checking its contents. Run the following command to list all available partitions:

cat /proc/partitions

List all available logical volumes with this command:

lvdisplay

After mounting each one check its contents by listing the mounted directory. For example if mounted to /mnt run:

ls /mnt

Some directories that should be listed to verify it is a root file system are /root, /boot and /home.

4. Once mounted use the following for-loop to rbind the virtual file systems:

for i in proc sys dev run; do mount --rbind /$i /mnt/$i ; done

Alternatively, mount them each separately rather than using the for-loop:

mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
mount --rbind /dev /mnt/dev
mount --rbind /run /mnt/run

5. After the root file system and virtual file systems are mounted run this command to chroot in:

chroot /mnt

After chrooted in it is a best-practice to run this command to make sure all other necessary partitions are also mounted:

mount -a

Example of corrective actions that can be done in rescue chroot:

If the initrd has been damaged or invalid , it can be recreated :

Command to use in SLES 11 :

mkinitrd

Command to use in SLES 12 and SLES 15 :

dracut

Note, that you need to explicitly define the kernel version for 'dracut(8)' since without defining one, the auto-detected version uses 'uname(2)' syscall which would return the kernel version from the (ISO) rescue environment, and thus it would fail with 'Cannot file module directory...'.

Steps to identify the installed kernel version and run the dracut command using that information:
First check the kernel version(s) in installed system using 'ls -1 /boot/vmlinuz-* /lib/modules/' command.

Example:

rescue:/ # ls -1 /boot/vmlinuz-* /lib/modules/
/boot/vmlinuz-5.3.18-150300.59.101-default
/boot/vmlinuz-5.3.18-150300.59.147-default

/lib/modules/:
5.3.18-150300.59.101-default
5.3.18-150300.59.112-default
5.3.18-150300.59.147-default

Then run the dracut command targetting installed kernel-version .

dracut -f --kver 5.3.18-150300.59.147-default

Cause

System does no longer boot.
There are several possible causes for that behavior.

Additional Information

Keep in mind that a chrooted system was not booted.

This means that services aren't running upon chroot into the system.

In SLE 12 systemd does not allow services to be started within the chrooted environment. Executable commands, so long as they don't rely on those services, should still work though. If you absolutely need to have a systemd service running in rescue mode look into systemd-nspawn

When using multipath with lvm, you should start multipathd before using lvm. If you do not, lvm may not detect the physical volumes needed to mount filesystems due to the lvm filter.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.