1. Orderly/Planned hot-plug: hot-plug operation is initiated via notification of hardware or software after all of required proprietary steps, such as unmounting datastore, unclaiming device, etc. are done.
2. Surprise hot-plug: Operation is initiated without prior notification. Such as hot-unplug device manually.

• ESXi supports hot-plug for only NVMe device which is in normal mode, i.e. the device isn't configured as RDM, PCI passthrough, or used as OS installation volume.
• It is recommended to do orderly hot-plug whenever as possible. Supporting surprise hot-plug is precaution against incorrect user operation. 

Best practices

• Choose a hot-plug capable slot and then insert NVMe device. Wait for power indicator being on, if the platform supports driver power control.
• Issue esxcli hardware PCI list and check if target nvme device is listed. Output is similar to following:

• Check readiness of SCSI devices for the NVMe device: run esxcli storage core device list. Output is similar to following:

• If NVMe device contains VMFS datastores, these datastores will be mounted automatically. Run 'esxcli storage filesystem list' to list all datastores claimed. 

Removal

• Make sure no IO is running on the NVME device to be removed before starting hot-plug, no matter orderly or surprise.
• On orderly hot-unplug scenario,  user needs to run proprietary steps before unplugging. They are:
  • Power off VMs, if any.
  • Unmount VMFS and detach devices. For more information see vSphere support for hot-swap of local disks.
• In surprise hot-unplug scenario, user stop IOs running on NVMe device and then plug device out immediately not running the above steps.
• Hot-unplug device. Check that the device is removed from the system device listing:
  • Issue "esxcli storage core device list", check that the device is not listed.
  • Issue "esxcli storage nmp path list", check that there is no path for this device.
• Plug in the device, check that the device becomes available. Steps are:
  1. If the device was hot-unplugged orderly, it needs to re-enable device being detected during rescan via command "esxcli storage core claiming autoclaim --enabled=True".
  2. Issue "esxcli hardware pci list|grep -w <vmhba>", check that new adapter is listed.
  3. Issue "esxcli storage core device list", check that the device is listed.
  4. Issue "esxcli storage nmp path list", check that the path to device is listed.
• Steps to remount VMFS volumes:
  1. Mount VMFS volume: esxcli storage filesystem mount -l <datastore>
  2. List all VMFS volumes: esxcli storage filesystem list