Using a large number of LUNs and/or paths with MPIO

SUSE Linux Enterprise Server 11 SP1
SUSE Linux Enterprise Server 11

In environments with large numbers of LUNs and/or paths to those LUNs, multipathd may be unable to create the multipath maps for all the devices. When this occurs, the following message (or a similar message) is seen in /var/log/messages:

  multipathd:  error creating pipe for callout: Too many open files

 

Multipathd requires open file descriptors for each managed path. In the startup script for multipathd (/etc/init.d/multipathd), the maximum number of open file descriptors is increased from the system default of 1024 to 4096. However, in environments with large numbers of LUNs and/or paths to those LUNs, this value may be insufficient.

As a general rule of thumb, there should be two available file descriptors per path. For example, if 1000 LUNs were available through 4 paths each, multipathd would need 8000 open file descriptors (#LUNs x #paths x 2).

To increase the maximum number of file descriptors, the following entry must be place in /etc/multipath.conf:
 

  defaults {
            max_fds   <VALUE>
           }

Note - It is very common for no /etc/multipath.conf file to exist. In this case, create the file using only the configuration option listed above. Otherwise, add the max_fds setting to the 'defaults' section of the existing multipath.conf file. If the server is booting from the SAN, run `mkinitrd` to ensure the new multipath.conf is added to the initrd.

After modifying (or creating) the /etc/multipath.conf file, restart multipathd to cause the setting to take effect.

SUSE has tested multipathd with 16,000 available LUN/path combinations. As long as the system has enough resources to support the required open files, and the max_fds is set appropriately, large numbers of LUNs and/or paths should work properly.