NFS Stale File Handle error and solution

In linux machine we have NFS mounted directories sometimes contain stale file handles. If you run command such as ls or vi you will see an error

# ls

.: Stale File Handle

Before moving to fix this issue first we need to understand the concept of Stale File Handle.

A filehandle becomes stale whenever the file or directory referenced by the handle is removed by another host.

while your client still holds an active reference to the object. A typical example occurs when the current directory of a process, running on your client, is removed on the server (either by a process running on the server or on another client).

So this can occur if the directory is modified on the NFS server, but the directories modification time is not updated.

To fixed this issue, the best solution is to remount directory from the NFS client using mount command.

# umount -f /test

# mount -t nfs nfsserver:/path/to/share /test

Job for nfs-server.service failed because the control process exited with error code. See "systemctl status nfs-server.service" and "journalctl -xe" for details. Resolution

On Red Hat Linux 7 operating system, some time you will get below NFS service failed error message.

"Job for nfs-server.service failed because the control process exited with error code. See "systemctl status nfs-server.service" and "journalctl -xe" for details."

The above error occur when you are taking restart of nfs service.

# systemctl restart nfs.service
Job for nfs-server.service failed because the control process exited with error code. See "systemctl status nfs-server.service" and "journalctl -xe" for details.
Resolution

# systemctl status nfs-server.service
nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Fri 2018-04-27 09:56:08 IST; 8s ago
  Process: 21370 ExecStopPost=/usr/sbin/exportfs -f (code=exited, status=0/SUCCESS)
  Process: 21366 ExecStopPost=/usr/sbin/exportfs -au (code=exited, status=0/SUCCESS)
  Process: 21362 ExecStop=/usr/sbin/rpc.nfsd 0 (code=exited, status=0/SUCCESS)
  Process: 21273 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
  Process: 2714 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=1/FAILURE)
 Main PID: 21273 (code=exited, status=0/SUCCESS)

Apr 27 09:56:07 localhost.redhat.com systemd[1]: Starting NFS server and services...
Apr 27 09:56:08 localhost.redhat.com exportfs[2714]: exportfs: Failed to resolve foobar.com <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Apr 27 09:56:08 localhost.redhat.com exportfs[2714]: exportfs: Failed to resolve foobar.com <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Apr 27 09:56:08 localhost.redhat.com systemd[1]: nfs-server.service: control process exited, code=exited status=1
Apr 27 09:56:08 localhost.redhat.com systemd[1]: Failed to start NFS server and services.
Apr 27 09:56:08 localhost.redhat.com systemd[1]: Unit nfs-server.service entered failed state.
Apr 27 09:56:08 localhost.redhat.com systemd[1]: nfs-server.service failed.

To resolve such issue you need to follow the below process.

You need to check which NFS version rpm is installed on the server.

#rpm -qa | grep nfs

the above command show you the installed nfs version.

Normally the above error occur due to lower version on nfs, so to resolve the issue you need to upgrade the nfs version.

installed the latest nfs rpm nfs-utils-1.3.0-0.33.el7 or later

#yum install nfs

The above command upgrade the nfs rpm, or you can download the manually rpm file and installed it through rpm -uvh command.

After that restart the nfs service again.

# systemctl restart nfs.service

it is started successfully, you can check the status via below command.

# systemctl status nfs.service

[root@localhost ~]# systemctl status nfs.service
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)
  Drop-In: /run/systemd/generator/nfs-server.service.d
           └─order-with-mounts.conf
   Active: active (exited) since Thu 2018-04-27 03:47:57 IST; 18h ago
  Process: 32477 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
  Process: 32472 ExecStartPre=/bin/sh -c /bin/kill -HUP `cat /run/gssproxy.pid` (code=exited, status=0/SUCCESS)
  Process: 32469 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
 Main PID: 32477 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/nfs-server.service

Apr 26 03:47:57 localhost systemd[1]: Starting NFS server and services...
Apr 26 03:47:57 localhost systemd[1]: Started NFS server and services.

Kernel Panic Error - not syncing: VFS:Unable to mount root fs on unknown-block(0,0)

Kernel Panic crash your system if it is occur. In Unix system, normally such kernel panic occur while we are upgrading the kernel or firmware of any Unix operating system.

In simple words, its look like a blue screen when occur on your personal laptop etc. but if we talk about Unix, the below error message appear during boot time.

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
Pid: 1, comm: swapper Not tainted 2.6.32-502.el6.x86_64 #1
Call Trace:
[<ffffffff815292bc>] ? panic+0xa7/0x16f
[<ffffffff81c2b432>] ? mount_block_root+0x216/0x2cb
[<ffffffff81002930>] ? bstat+0x2b0/0x980
[<ffffffff81c2b53d>] ? mount_root+0x56/0x5a
[<ffffffff81c2b6b1>] ? prepare_namespace+0x170/0x1a9
[<ffffffff81c2a92a>] ? kernel_init+0x2e1/0x2f7
[<ffffffff8100c20a>] ? child_rip+0xa/0x20
[<ffffffff81c2a649>] ? kernel_init+0x0/0x2f7
[<ffffffff8100c200>] ? child_rip+0x0/0x20

when such above error occur that means your system is in dangerous state and if you not rectify this issue you may lost your data as well. 

We will provide to you one of cause and solution of this error, why it is occur and how we can resolve it.

In one of case, above error message occur when you upgrade your kernel and by mistake if it is not properly upgrade then kernel version get different with previous one.

For resolving this issue you need to boot you unix machine in grub mode. You can take a reboot of server and press any key once the boot screen appear. In the below screen it is showing different option, please select the grub menu option.

In grub mode, check the kernel and initrd parameter, if both parameter kernel version and initrd version are not matched that's means this is the root cause. So you need to correct the kernel version and make a same version entry and boot the operating system after saving the grub entry for temporary purpose.

To fix the permanent this issue, you need to change the kernel version in grub.conf file otherwise once you take a reboot of server again, you have to face same issue. So avoiding such issue again, please correct the version in grub configuration file.

[root@localhost]# vi /etc/grub.conf

#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS 6 (2.6.32-504.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-504.el6.x86_64 ro root=/dev/mapper/vstorage-root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vstorage/root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
        initrd /initramfs-2.6.32-504.el6.x86_64.img

Please do same version entry on bold highlighted place. The kernel version should be same at kernel and initrd line. After changing the entry save the entry and take a reboot of server again to check the kernel panic problem again occur or not.