NFS mount attempt fails, complains of statd or lock problems

This document (7022429) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11

Situation

An NFS client machine attempts to perform an NFS v3 mount (or in some cases, a different NFS v3 operation after mount). This fails. For a failed mount attempt, the following errors appear at the command line:

mount.NFS: rpc.statd is not running but is required for remote locking.
mount.NFS: Either use '-o nolock' to keep locks local, or start statd.
mount.NFS: an incorrect mount option was specified

For a failed mount attempt or other NFS failures, one or more of the following errors may be seen in /var/log/messages:

rpcbind: connect from 127.0.0.1 to getport/addr(status): request from unauthorized host

rpcbind: connect from 127.0.0.1 to unset(status): request from unauthorized host

rpc.statd[18142]: Could not bind name to socket: Address already in use

These kinds of failures are not expected with NFS v4.x, though in some rare and unusual configurations, there might still be some concerns between NFS 4.x and rpcbind.

Resolution

More than one thing can be responsible:

1. If this NFS client machine has had a new kernel installed without being rebooted, this can cause one variation of this issue. Compare the output of "rpm -qa | grep kernel-" [to see the installed kernel(s)] and the output of "uname -a" [to see the running kernel] and see if there is a discrepancy. Reboot to put the currently installed kernel into effect, if needed.

2. The NFS client machine's /etc/hosts.allow and/or /etc/hosts.deny may be restricting access to the local portmapper (rpcbind), or those files may not be readable by the rpcbind service.

The NFS client machine must be able to talk to it's own rpcbind (portmapper) service in order to perform various functions. This is allowed by default, but in many cases, administrators have changed these files or their permissions, without realizing the impact of those changes.

a. The permissions of both /etc/hosts.allow and /etc/hosts.deny are normally 644, which allows all entities to read them. These permissions can be restored with:

chmod 644 /etc/hosts.allow
chmod 644 /etc/hosts.deny

Then test whether the problem still exists.

b. Once permissions are correct, if the problem still exists, the contents of those files should be checked. The exact modifications needed may depend upon the configuration desired. Returning to the "default" is the simplest way to correct this issue. Also, returning to the default (at least temporarily) is a good way to confirm or deny the involvement of these files.

The default /etc/hosts.allow file will contain only comments and no active lines.

The default /etc/hosts.deny file contains comments plus one line:

http-rman : ALL EXCEPT LOCAL

With those files in their default condition, test whether the problem still exists.

c. Optional, instead of returning to the default contents: If many things are being denied in hosts.deny, and it is desired to keep those restrictions, then it may be enough to add the following entry to /etc/hosts.allow:

#/etc/hosts.allow

rpcbind : ALL : ALLOW

CAUTION:
Some administrators might be tempted to "increase security" by denying some or all services with lines such as:

#/etc/hosts.deny

ALL : ALL

rpcbind : ALL

And once this causes problems for NFS mounts, they may believe that it would be sufficient solution to change those entries to make an exception for "local" access to those services, such as:

#/etc/hosts.deny

ALL : ALL EXCEPT LOCAL

rpcbind : ALL EXCEPT LOCAL

These settings would indeed allow more operations to complete successfully. HOWEVER, it is still not enough for proper functioning of NFS v3. An NFS client sometimes need to be contacted by an NFS Server, to announce the NFS Server's status or to perform lock call-backs. If the NFS client's rpcbind is only allowing "local" access, then the NFS Server will not be able to reach those services on the client.

The NFS client's rpcbind should be open to everyone, or at least to LOCAL plus any NFS Server that it deals with.

So if a rule in /etc/hosts.allow is to be used to correct this, it should be:

#/etc/hosts.allow

rpcbind : ALL : ALLOW

Alternatively, if a rule in /etc/hosts.deny is used with an exception, it should be something like:

#/etc/hosts.deny

rpcbind : ALL EXCEPT LOCAL 10.10.10.10

#(where 10.10.10.10 is the IP address of the NFS Server with which this client machine interacts)

or possibly:

#/etc/hosts.deny

ALL : ALL EXCEPT LOCAL 10.10.10.10

#(where 10.10.10.10 is the IP address of the NFS Server with which this client machine interacts)

Cause

/etc/hosts.allow and hosts.deny are files used by the "TCP Wrappers" library. This library can enforce rules about who can access various services. However, these rules are not automatically enforced. A service will only be protected by these rules if it has been written to use them. A service can be checked by doing "ldd" against the binary.

For example, the command:

ldd /sbin/rpcbind

will output a list of library dependencies for "rpcbind", and one of them will be:

libwrap.so.0 => /lib64/libwrap.so.0

The presence of this library is the signal that the program (which was checked with ldd) is written to use TCP Wrappers.

Previously, SLES 10 used a package called "portmap" to serve rpc functions, and the portmap binary did not use TCP Wrappers. In contrast, SLES 11 and beyond uses the rcpbind package for portmap services, and rpcbind does use TCP Wrappers. So customers who have TCP Wrapper rules configuration may not have run into this problem on SLES 10, but may find that moving from SLES 10 to a newer distribution triggers this problem.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.