How VMware ESXi determines the order in which names are assigned to devices (2091560)
Similarly, storage adapters have aliases in the form of vmhbaN, and a graphics device alias looks like vmgfxN. See the additional information section for more information on the numbers used.
On initial install, the new naming scheme tries to assign names in a predictable order by physical location. Thereafter it keeps a device's name stable even across the addition or removal of other devices to the system. At install time, typically devices that are integrated on the system's motherboard receive the lowest numbered names, in the order that their output ports are numbered on the system's case, and devices that are in pluggable slots receive the next highest numbered names, in the order that the slots are numbered on the case. The reader should note that this process ignores devices for which no driver is found.
After the ESXi device manager has an ordered list of information for devices in the system, it can assign aliases to these devices according to their types: network, storage, or graphics adapter.
In the case of a new installation or autodeploy with no host profile, the device manager will query the machine's firmware to get location information for each port, generally including which ports are onboard, which are in pluggable slots, and how those ports and slots are numbered on the system's case. ESXi then assigns aliases to the ports from 0 upwards in order of onboard ports first, then ports not described by the firmware, then ports in pluggable slots.
If a system has:
- Two onboard network ports
- One single-port NIC in slot #3
- One dual-port NIC in slot#5
Then devices names should be assigned as:
|Physical Port||Device Alias|
|Onboard port 1||vmnic0|
|Onboard port 2||vmnic1|
|Slot #3 port 1||vmnic2|
|Slot #5 port 1||vmnic3|
|Slot #5 port 2||vmnic4|
Changes to the hardware configuration are subject to these rules:
- When cards are added or removed after the initial installation, ports remaining retain their assigned names.
- ESXi assigns a new port to the first available unassigned alias for its type.
- The alias for a removed port becomes available.
- A replaced port may be seen either as being the same as the original port or as a removal followed by an addition. In the former case, the new port has the same alias. In the later case, it may acquire the same alias.This occurs if the old alias is the lowest-numbered free alias.
- In the event the replacement device has more ports, these will be assigned the next available unassigned alias(es) of its type. A replacement device with fewer ports will release the unused aliases for re-assignment.
Where the system is configured with a host profile the system administrator should capture an updated profile after the hardware changes.
Upgrades from one release of ESXi to the next, including autodeploy configurations, do not change the aliases previously assigned to the system's ports. This includes upgrading from prior releases to ESXi 5.5 and later.
To upgrade a cluster of homogenous stateless hosts from one release to another:
- Upgrade the auto-deploy server with the new ESXi image.
- Remove the reference host from the cluster.
- Reboot the reference host with the new ESXi image.
- Extract a host profile.
- If any device names are not what you expect, edit the device alias section of the host profile.
- Reboot the reference host with the new profile.
- If the host is now configured as desired, attach the host profile to the remainder of the hosts, otherwise repeat from step 5.
- Reattach the reference host back to the cluster.
- Reboot the rest of the cluster (typically a rolling reboot).
Most drivers create device objects that map one to one with physical PCI functions. The range of aliases reserved for these physical devices begins at 0. Some drivers create objects that are not mapped one to one with physical functions. For native drivers, these aliases are recorded persistently, in local storage (on stateful systems) and/or in the host profile.
Drivers built on ESXi's native driver framework can create logical devices with additional devices that do not correspond with PCI functions. These devices are assigned higher numbered aliases at the time their driver first creates them, in an unspecified order. Just as with aliases for physical devices, aliases for logical devices are persistent across reboots in local storage and/or in the host profile. As an example, consider a converged network adapter (CNA) whose PCI class identifies it as a network device, but whose native driver exports both a network uplink object and an iSCSI storage adapter object. In this case, the network uplink would be a physical device, perhaps with alias
vmnic1, while the iSCSI adapter would be a logical device, perhaps with the alias
Drivers built on ESXi's Linux driver compatibility layer vmklinux can create pseudo-logical devices. As with true logical devices, these devices do not correspond one-to-one with PCI functions. In general, their aliases are not guaranteed persistent across reboots. However, certain vmklinux drivers may use special algorithms to provide persistence, such as inserting digits into the corresponding persistent physical device alias in a fixed pattern. With other vmklinux drivers, the pseudo-logical device aliases are generally assigned in a deterministic way and thus do not change from boot to boot if the set of hardware and drivers installed in the system remains the same, but again this is not guaranteed.
This section discusses a few special situations where the results may seem unexpected. It is not exhaustive.
Devices Without Drivers
As stated, aliases are assigned only for devices that have drivers. For example, if the ESXi release that is initially installed on a machine that does not have a driver for one of the machines NICs, that NIC's ports do not receive vmnic aliases. Numbers will not be reserved for them in the sequence. If ESXi is later upgraded to a release that does support the NIC, those ports receive the next available unused aliases, as if the NIC is newly added. This can put the aliases in a different order than would be seen in a fresh install of the new ESXi release on the same machine.
On certain systems, ports that a user may think of as being integrated are actually on pluggable PCI/PCIe cards. For example, on systems from at least one vendor, several network ports that are standard equipment included with the base system and may be perceived as onboard by the end customer, are actually on a PCIe card that is factory installed in the highest numbered slot. Therefore, they appear later in the vmnic alias order than any NICs that may be added in other slots.
As another example, on blade systems there is often not a clear distinction between which devices are onboard and which are pluggable. The blade chassis may provide a way to move devices between different blades, or even to create and remove multiple virtual devices and assign them to different blades. In these cases, the detailed rules in the next section still apply.
Typically, a hardware vendor is likely to provide M1/M2 information for devices that are hardwired to a single blade and S1/S2/S3 for devices that are assignable. Alternatively, the hardware vendor can provide M1 information for all devices, which gives the vendor full control over the order in which ESXi assigns names. See the next section, Device Information and Naming for an explanation of the information source tags, M1, M2, S1, S2, and S3.
ESXi assigns names to all the devices having drivers that it finds.
ESXi obtains device location information from these sources. For each device, these sources are consulted in order until information is found from one of them.
|Tag||Device Data Source||Device Information Specification|
|M1||PCIe _DSM for Device Naming||PCIe Firmware Specification, Rev. 3.1, section 4.6.7, DSM for Naming a PCI or PCI Express Device Under Operating Systems|
|M2||SMBIOS Type 41 Structure||System Management BIOS (SMBIOS) Reference Specification, Version 2.6 or later, section Onboard Devices Extended Information (Type 41)|
|S1||ACPI _SUN Object||Advanced Configuration and Power Interface Specification (ACPI), Revision 5.0, section 6.1.11 _SUN (Slot User Number). The _SUN method for the PCI function is tried first; then if that does not exist, the _SUN method for function 0 of the same PCI device is tried|
|S2||Legacy $PIR Table||Legacy $PIR table: PCI System Architecture, 4th Edition, MindShare, Inc., Tom Shanley and Don Anderson, Interrupt Routing Table, pages 233-238|
|S3||SMBIOS Type 9 Structure||System Management BIOS (SMBIOS) Reference Specification, Version 2.6 or later, section System Slots (Type 9)|
|P||PCI SBDF Address||The device's current PCI SBDF (segment:bus:device.function) address|
After these information sources are consulted for all devices having drivers, devices within each type are sorted and named in this order:
- Devices whose information came from M1 or M2, in the port number order given by that information.
- Devices whose information came from P. These are often onboard devices for which the hardware platform vendor has not supplied M1 or M2 information.
- Devices whose information came from S1, S2, or S3, in slot number order. If more than one device is in the same slot such as with a typical multi-port NIC, devices within the slot are sorted in PCI SBDF order.
Changing the names assigned by the ESXi host
In some cases you may want to change the aliases assigned by the ESXi host. The procedure is as follows:
- If you are using Host Profiles
If the affected system is attached to a Host Profile, edit the Device Alias Configuration component of the profile, which is found under the General System Settings in the profile. Then remediate the affected ESXi host.
Note: The host must be rebooted to complete the remediation.
- If you are not using Host Profiles
From the ESXi shell, run the command to see the current assignment of aliases to device locations:
localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list
Bus type Bus address Alias
pci m01000208 vmnic0
pci m01000218 vmnic1
pci p0000:00:07.1 vmhba0
pci s00000002.00 vmhba1
logical pci#p0000:00:07.1#1 vmhba64
logical pci#p0000:00:07.1#0 vmhba0
For each alias that you want to reassign, use the command:
localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --bus-type pci --alias vmnicN --bus-address B
where the vmnicN and B are the new assignment that you want to make. For instance, in the above example you could swap vmnic0 and vmnic1 using the commands:
localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --bus-type pci --alias vmnic0 --bus-address m01000218
localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --bus-type pci --alias vmnic1 --bus-address m01000208
After giving all the necessary alias store commands, perform a clean shut down and reboot the system.
Currently, these alias numbering assignments are used:
- For physical PCI aliases:
vmnic: 0-31, 64-127
- For native driver logical aliases:
vmhba: 64-95, 128-159, ...
vmgfx: 32 - 1000000
- For vmlinux logical aliases:
vmnic: 32-63, 1000000-MAX
vmhba: 32-63, 96-127, ...
Counting physical PCI functions, VMware supports names for 32 HBA and GFX adapters, and 96 NICs.
Some devices make multiple ports available at the same PCI address. This is a brief description of the algorithm used to assign aliases in this case.
- Standard Naming convention (for example, vmnicX where X is a decimal number)
- Need to have persistence across reboots for logical uplinks.
- Algorithm: for each device with multiple ports of the same type, indicated here by relative port numbers port 1, port 2, etc:
- port 1 - Assign the PCI alias (the number will be from the range of 0-31 or 64-127)
- port 2 (and higher) - Gets its alias number in million range, as explained in step 3
- In order to enable persistence for additional ports, perform this approach. The number for each alias is obtained using this formula:
1000000 + (pciIndex*100) + p#
pciIndex: is the number in the PCI alias.
p#: is the relative port number
For example: If PCIe function gets a name as
lspci : 00:04:00.0 - vmnic4
This method allows defining up to 100 pseudo devices for a single PCIe PF (physical function).