Increasing the default value that defines the maximum number of NFS mounts on an ESXi/ESX host

Products

VMware vSphere ESXi

Issue/Introduction

This article describes how to change the default value for the configuration parameter that defines the maximum number of NFS mounts on an ESXi/ESX host.

This article provides information on:

Configuring NFS TCP and NFS HEAP settings
Identifying appropriate settings for TcpipHeapSize and TcpipHeapMax.

Note: vSphere 6.0 has increased the maximum heap size and added support for the NFS 4.1 protocol. There are some differences between NFS3 and NFS4.1 support in vSphere 6.0. For more information, see the Understanding Network File System Datastores section of vSphere 6.0 Storage.

Environment

VMware vSphere ESXi 5.5
VMware ESX 4.1.x
VMware vSphere ESXi 6.0
VMware ESX Server 3.0.x
VMware ESX Server 3.5.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.1
VMware ESX 4.0.x
VMware vSphere ESXi 6.5
VMware ESXi 4.0.x Embedded
VMware vSphere ESXi 7.0.x
VMware ESXi 4.1.x Installable
VMware ESXi 3.5.x Embedded
VMware ESXi 3.5.x Installable
VMware vSphere ESXi 6.7

Resolution

The vSphere ESXi/ESX default configuration allows for only eight NFS mounts per ESXi/ESX host. There are three advanced configuration options which control the maximum number of NFS mounts:

To edit advanced configuration options, select the ESXi/ESX host in the Inventory Panel, then navigate to Configuration > Software > Advanced Settings to launch the Settings window.
Select and highlight any of the advance options listed below the User Interface page will display, the range from Default to the Maximum value that can be set.

Set these values:

Under NFS, Select NFS.MaxVolumes: Limits the number of NFS datastores which can be mounted by the vSphere ESXi/ESX host concurrently.Till 6.7 the default value is 8 and from 7.0, the default value is 32 which can be increased to a maximum specific to the version of ESXi/ESX:
- ESXi/ESX 3.x : Set NFS.MaxVolumes to 32
- ESXi/ESX 4.x : Set NFS.MaxVolumes to 64
- ESXi 5.0/5.1/5.5 : Set NFS.MaxVolumes to 256
- ESXi 6.0/6.7/7.0 : Set NFS.MaxVolumes to 256

From Esxi 6.0, its not required to increase Net.TcpipHeapSize and Net.TcpipHeapMax. Following are the values to be updated for the earlier versions:

Under Net, Select Net.TcpipHeapSize: The amount of heap memory, measured in megabytes, which is allocated for managing VMkernel TCP/IP network connectivity. When increasing the number of NFS datastores, increase the default amount of heap memory as well:

ESXi/ESX 3.x: Set Net.TcpipHeapSize to 30
ESXi/ESX 4.x: Set Net.TcpipHeapSize to 32
ESXi 5.0/5.1/5.5: Set Net.TcpipHeapSize to 32

Under Net, Select Net.TcpipHeapMax: The maximum amount of heap memory, measured in megabytes, which can be allocated for managing VMkernel TCP/IP network connectivity. When increasing the number of NFS datastores, increase the maximum amount of heap memory as well, up to the maximum specific to the version of ESXi/ESX host:

ESXi/ESX 3.x : Set Net.TcpipHeapMax to 120
ESXi/ESX 4.x: Set Net.TcpipHeapMax to 128
ESXi 5.0/5.1: Set Net.TcpipHeapMax to 128
ESXi 5.5: Set Net.TcpipHeapMax to 512
ESXi 6.0: Set Net.TcpipHeapMax to 1536

For ESXi 6.0, 6.5, 6.7: Default Net.TcpipHeapMax is 512MB which is sufficient even for 256 mounts. Its not required to increase.
These settings enable the maximum number of NFS mounts for vSphere ESXi/ESX.
Changing Net.TcpipHeapSize and/or Net.TcpipHeapMax requires a host reboot for the changes to take effect.

Modify the three advanced configuration options using the vSphere Client or other method. For a full list of methods, see Configuring advanced options for ESX/ESXi (1038578).

For more information on advanced NFS settings, see Definition of the advanced NFS options (1007909) and the Best Practices for running VMware vSphere on Network Attached Storage whitepaper.

Additional Information:

ESXi 6.5U3, 6.7U3 and 7.0:

In these releases, NFSv3 socket sizes are increased. Default socket size is 1MB(earlier 256K) for both send and receive sockets. Max that can be configured is 5MB(earlier 512K).

Default Net.TcpipHeapMax is increased to 1024MB. Max Net.TcpipHeapMax that can be configured is 3200MB.

Following equation gives the TCPIP heap required for number of shares.
TCPIPHeapMax required = shares * (send socket size + receive socket size)

Example 1:

Shares = 256. Socket size = 1MB (default)
TCPIPHeapMax required = 256 * (1M + 1M) = 512MB

So, for the default socket sizes of 1MB, existing default TCPIPHeapMax of 1024MB is sufficient.

Example 2:

Shares = 256. Configured Socket size = 5MB (NFS.SendBufferSize, NFS.ReceiveBufferSize)
TCPIPHeapMax = 256 * (5M + 5M) = 2560MB
In this case, one needs to increase the TCPIPHeapMax to atleast 2560MB.

These settings enable the maximum number of NFS mounts for vSphere ESXi/ESX.
Changing Net.TcpipHeapSize and/or Net.TcpipHeapMax requires a host reboot for the changes to take effect.

For translated versions of this article, see:

Additional Information

Possible Errors

2020-01-24T23:34:06.268Z cpu0:2099888 opID=4c099e1c)NFS: 205: NFS mount 1x.xxx.xxx.xx:/nfs_datastore failed: NFS has reached the maximum number of supported volumes