ESXi IO connectivity issues or PSOD with VT-d interrupt remapper disabled
Article ID: 317585
Updated On:
Products
VMware
Issue/Introduction
Symptoms:
- The ESXi host has the VT-d interrupt remapper disabled (ESXi boot option iovDisableIR set to TRUE).
- In the /var/log/vmkernal log file, there are entries similar to:
ALERT: IntrCookie: 3411: Interrupt received on invalid vector (cpu 0, vector 72); ignoring it.
Note:
- The cpu and vector number mentioned are for illustrative purposes only.
- ESXi 5.5p10, 6.0p04, 6.0u3, and 6.5 by default disable use of the VT-d interrupt remapper due to a known issue. For more information, see ESXi host fails with a diagnostic screen due to an Intel Virtualization Technology Erratum (2147325).
Cause
The vmkernel alert shown indicates that ESXi has received an interrupt on an unexpected processor interrupt vector and has ignored the interrupt because ESXi does not know the source of the interrupt or which software handler to invoke.
Reception of such an interrupt is not normal and may cause loss of I/O connectivity and/or ESXi PSODs.
In order to distribute the interrupt load evenly across processors, ESXi performs interrupt re-balancing. As part of this process, ESXi may modify the target processor and interrupt vector assigned to an I/O device. This process is known as I/O device interrupt migration.
When the VT-d interrupt remapper is not present in a platform or its use is disabled in ESXi (boot option iovDisableIR=TRUE), ESXi performs I/O device interrupt migration by modifying the PCI MSI/MSI-X registers in the I/O device(s).
VMware has recently received reports that some I/O devices do not react well when ESXi performs interrupt migration by programming the device's MSI/MSI-X registers. Specifically, some I/O devices may interrupt in a processor vector which ESXi no longer uses and considers invalid, causing the generation of the alert message and its subsequent effects.
Resolution
Alternatively as a workaround enable VT-d interrupt remapper by setting vmkernel boot option iovDisableIR to FALSE.
- Connect to the ESXi host with an SSH session and root credentials.
- Run this command:
esxcli system settings kernel set --setting=iovDisableIR -v FALSE
- Reboot the ESXi host.
- Ensure the iovDisableIR setting is set to FALSE with this command:
esxcli system settings kernel list -o iovDisableIR
For example:
esxcli system settings kernel list -o iovDisableIR
Name Type Description Configured Runtime Default
------------ ---- --------------------------------------- ---------- ------- -------
iovDisableIR Bool Disable Interrupt Routing in the IOMMU... FALSE FALSE TRUE
If this issue is observed with other ESXi versions:
- Ensure the VT-d interrupt remapper is enabled in the BIOS.
- Run this command to verify if it is enabled:
vsish -e get /hardware/iov/IntrRemappingEnabled
This command returns the value 1 if the VT-d interrupt remapper is enabled in the BIOS and in use by ESXi, and 0 otherwise.
- If the alert persists, contact VMware Technical Support to file a support request. For more information, see How to file a Support Request in Customer Connect (2006985).
Additional Information
Feedback
Yes
No
Powered by
