Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Definition of the advanced NFS options (1007909)

Details

This article provides information on Advanced NFS settings.

Warning: VMware does not recommend making changes to the default settings.

Solution

To access the NFS advanced options:
  1. Log into the VMware vSphere Client.
  2. Select the ESXi/ESX host.
  3. Click the Configuration tab.
  4. Click Advanced Settings.
  5. Select NFS.
This is a list of each setting with a brief description:

Note: If the NFS HeartBeat advanced option values are modified, a reboot of the ESXi/ESX host is required.
  • NFS.DiskFileLockUpdateFreq

    The time between updates to the NFS lock file on the NFS server. Increasing this value increases the time it takes to recover stale NFS locks. For more information, see NFS Locking.

  • NFS.HeartbeatFrequency

    This sets how often the NFS heartbeat world runs to see if any NFS volumes need a heartbeat request. For more information, see NFS Heartbeats.

  • NFS.HeartbeatTimeout

    The amount of time before aborting a heartbeat request. For more information, see NFS Heartbeats.

  • NFS.HeartbeatDelta

    The amount of time after a successful GETATTR request before the heartbeat world issues a heartbeat request for a volume. If an NFS volume is in an unavailable state, an update is sent every time the heartbeat world runs (NFS.HeartbeatFrequency seconds). For more information, see NFS Heartbeats.

  • NFS.HeartbeatMaxFailures

    The number of consecutive heartbeat requests that must fail before the server is marked as unavailable. For more information, see NFS Heartbeats.

  • NFS.IndirectSend

    This is a debugging option. Do not use this for any other reason. It causes all NFS I/O to be queued to a helper world and decreases performance if enabled.

  • NFS.LockDisable

    This is a debugging option. This option can cause data corruption and should not be used without assistance from VMware Technical Support. This option has been removed from ESX 4.0.

  • NFS.LockRenewMaxFailureNumber

    The number of lock update failures that must occur before the lock is marked as stale. For more information, see NFS Locking.

  • NFS.LockUpdateTimeout

    The amount of time before a lock update request is aborted.
    For more information, see NFS Locking.

  • NFS.MaxConnPerIP

    This option defines the maximum number of TCP/IP connection per NFS Server.

  • NFS.MaxVolumes

    The maximum number of NFS volumes that can be mounted. The TCP/IP heap must be increased to accommodate the number of NFS volumes configured. For more information, see TCP/IP Heap Size.

  • NFS.ReceiveBufferSize

    This is the size of the receive buffer for NFS sockets. This value was chosen based on internal performance testing. VMware does not recommend adjusting this value.

  • NFS.SendBufferSize

    The size of the send buffer for NFS sockets. This value was chosen based on internal performance testing. VMware does not recommend adjusting this value.

  • NFS.SyncRetries

    The number of retries before synchronous IO fails (10 seconds per retry).

  • NFS.UDPRetransmitDelay

    This option is not used. VMware does not currently support NFS over UDP.

  • NFS.VolumeRemountFrequency

    This determines how often an NFS volume attempts to mount when it is initially unable to mount. After a volume is mounted it never needs to be remounted. The volume may be marked unavailable if it loses connectivity to the NFS server but it still remains mounted.

    Note: Some of these advanced options might be missing or additional options added depending on the version of ESXi/ESX host.

NFS Locking

NFS locking on ESX does not use the NLM protocol. VMware has its own locking protocol. NFS locks are implemented by creating lock files on the NFS server. Lock files are named .lck-fileid, where fileid is the value of the fileid field returned from a GETATTR request for the file being locked.

When a lock file is created, an update is periodically (every
NFS.DiskFileLockUpdateFreq seconds) sent to the lock file to let other ESX hosts know that the lock is still active. The lock file updates generate small (84 byte) WRITE requests to the NFS server. Changing any of the NFS locking parameters changes how long it takes to recover stale locks. This formula can be used to calculate how long it takes to recover a stale NFS lock:

(NFS.DiskFileLockUpdateFreq * NFS.LockRenewMaxFailureNumber) + NFS.LockUpdateTimeout

This example demonstrates this equation using the default values in VMware ESX 3.5:
X is the length of time it takes to recover from a stale NFS lock.

X = (NFS.DiskFileLockUpdateFreq * NFS.LockRenewMaxFailureNumber) + NFS.LockUpdateTimeout
X = (10 * 3) + 5
X = 35 seconds

 
If any of these parameters are modified, it is very important that all ESX hosts in the cluster use identical settings. Having inconsistent NFS lock settings across ESX hosts can result in data corruption.

NFS Heartbeats

NFS heartbeats are used to determine whether or not an NFS volume is still available. NFS heartbeats are actually GETATTR requests on the root file handle of the NFS Volume. There is a system world that runs every NFS.HeartbeatFrequency seconds to check if it needs to issue heartbeat requests for any of the NFS volumes. If a volume is marked available, it only issues a heartbeat if it has been NFS.HeartBeatDelta seconds since it issued a successful request (for example, a successful GETATTR, READ, READDIR, etc) that does not modify the attribute for that volume. The NFS heartbeat world always issues heartbeats for NFS volumes which are marked unavailable. Here is the formula to calculate the Minimum and the Maximum time for an ESX to mark an NFS volume as unavailable:

Minimum time to mark a volume down(minTime) = (HeartbeatFrequency * (HeartbeatMaxFailures - 1)) + HeartbeatTimeout

Maximum time to mark a volume down(maxTime) = HeartbeatDelta + HeartbeatFrequency + minTime


Default values for these variables in ESX/ESXi 4.x:
  • HeartbeatFrequency 9
  • HeartbeatTimeout 5
  • HeartbeatDelta 5
  • HeartbeatMaxFailures 3
Default values for these variables in ESX/ESXi 5.x:
  • HeartbeatFrequency 12
  • HeartbeatTimeout 5
  • HeartbeatDelta 5
  • HeartbeatMaxFailures 10

TCP/IP Heap size

The TCP/IP stack in versions of ESX prior to ESX 4.0 does not handle mbuf allocation failures gracefully (purple diagnostic screens). Because of this, it is very important that the TCP/IP heap size is adjusted to accommodate usage by NFS. Unfortunately there are many consumers of the TCP/IP stack, and it is difficult to precisely calculate how large the TCP/IP heap needs to be. The safest way to calculate the TCP/IP heap size, given the number of NFS volumes configured, is to linearly scale the default values up to the supported maximum specific to the version of ESXi/ESX. For more information, see Increasing the default value that defines the maximum number of NFS mounts on an ESXi/ESX host (2239).

For example, the default number of NFS volumes is 8 and the default min/max sizes of the TCP/IP heap are 6 MB / 30 MB respectively. This means the TCP/IP heap size for a host configured with 32 NFS volumes should have the min/max TCP/IP heap sizes set to 24 MB / 120 MB.

Tags

NFS-advanced-options

Update History

12/28/2012 - Added ESXi 5.1

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 36 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 36 Ratings
Actions
KB: