Slow server startup due to buffer I/O errors on attached LUNs

This document (7005566) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 9
SUSE Linux Enterprise Server 10
SUSE Linux Enterprise Server 11

Situation

Buffer I/O errors on passive paths to one or more LUNs can cause significant delays during server startup. These errors occur after loading the HBA driver and appear similar to the following:

    sd 6:0:0:0: [sdc] Device not ready
    sd 6:0:0:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
    sd 6:0:0:0: [sdc] Sense Key : Not Ready [current]
    sd 6:0:0:0: [sdc] Add. Sense: Logical unit not ready, manual intervention required
    sd 6:0:0:0: [sdc] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
    end_request: I/O error, dev sdc, sector 0
    Buffer I/O error on device sdc, logical block 0

Resolution

These buffer I/O messages are very dependent upon the server hardware, and are usually seen during the hardware scan that takes place during server startup. This is before multipathing has started and recognized these paths as being passive.

In many environments it is not possible to completely eliminate these messages, but they can be substantially reduced by removing the HBA driver from the initrd, and loading the module at a later stage in the boot process. This can be accomplished through the following steps:

- Edit /etc/sysconfig/kernel.
- Remove the HBA driver (i.e. qla2xxx or lpfc) from the INITRD_MODULES line.
- Add the HBA driver to the MODULES_LOADED_ON_BOOT line.
- Save the modified /etc/sysconfig/kernel file.
- Rebuild the initrd using `mkinitrd`. Monitor the output of this command to ensure the HBA driver is not being added to the kernel for some other reason.
- Reboot

NOTE - If the server is booting from a SAN, this solution should not be used.

Additional Information

Using this workaround is very dependent upon the environment of the server. However, if the server is not using devices on the SAN for system partitions, there should be no issues implementing this change. Using this configuration, server boot times should be substantially faster. In one extreme case, the server startup time was reduced from over 45 minutes, to just a few minutes.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.