VMCI DMA datagrams do not work with IOMMU and can fail on vMotion
search cancel

VMCI DMA datagrams do not work with IOMMU and can fail on vMotion

book

Article ID: 318491

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

VMs may experience either of the two issues,

  • Linux guest operating system cannot complete booting.

  • On vMotion, snapshot restore, VM resume, or VM hot-add, services using VMCI or vSockets may experience unexpected communication or loss of communication, or the guest operating system may become unresponsive.



Environment

VMware vSphere ESXi 8.0
VMware vSphere ESXi 8.0.x

Cause

These issues occur on a hardware version 20 VM with a Linux distribution that has specific patches introduced in Linux kernel 5.18 for a VMCI feature, including but not limited to up-to-date versions of RHEL 8.7, Ubuntu 22.04 and 22.10, and SLES15 SP3 and SP4.

For VMs stuck during the boot process, this will happen when the VM also has Advanced Processor Option "Enable IOMMU in this virtual machine" set and the guest operating system has DMA remapping enabled.

For the unexpected communication or loss of communication in services using VMCI or vSockets, this may happen when a vSphere vMotion operation on a virtual machine runs in parallel with an operation that sends or receives VMCI datagrams. Under the same conditions, the issue can also happen when restoring a memory snapshot, resuming a suspended VM or using Hot Add.

Resolution

This issue is resolved in vSphere ESXi 8.0b (build number 21203435).


Workaround:

Set `vmci.dmaDatagramSupport=FALSE` in Advanced VM Settings.

For Linux guest operating systems unable to complete booting, another option is to disable Advanced Processor Option "Enable IOMMU in this virtual machine"

 


Additional Information

Impact/Risks:
This issue will have an impact on Virtual machine Guest OS or VMs itself.