Virtual machines using EFI firmware fails to start from certain storage devices
search cancel

Virtual machines using EFI firmware fails to start from certain storage devices

book

Article ID: 318783

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

When powering on a Windows virtual machine using EFI from an Dell EMC PowerStore and EMC XtremIO storage arrays, you see these events:

  • Virtual machine fails to start, reports an I/O error or an error indicating that boot files are missing or corrupt.
  • In the /var/log/vmkernel.log file, you see this error indicating disk I/O failures on the storage array:

    Cmd(0x#####) 0x##, CmdSN 0x##### from world ##### to dev "####" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0
     
  • Running the esxcli system settings advanced list -o "/Disk/DiskMaxIOSize" command on the host containing the virtual machine reports the default disk IO block size:

    Path: /Disk/DiskMaxIOSize
    Type: integer
    Int Value: 32767
    Default Int Value: 32767
    Min Value: 32
    Max Value: 32767
    String Value:
    Default String Value:
    Valid Characters:
    Description: Max Disk READ/WRITE I/O size before splitting (in KB)
     
  • When running the esxcli storage core path list command, you see that the storage array reports a default IO block size not equal to 32767.
  • You may also encounter an issue where after installing some vibs in a boot from SAN environment  ESXi fails to  with message 
       Loading /sb.v00
      Error loading /sb.v00
      Compressed MD%: beb813d57fd3215416e8b87a4db2e4a2
      Decompressed MD5: 00000000000000000000000000000000
      Fatal erorr: 6 (Buffer too small)
  • You may also encounter an issue when snapshot is attempted on a VM it fails with message "An error occurred while saving: 38 (Function not implemented)"


Environment

VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5
VMware vSphere ESXi 5.5

Cause

The default DiskMaxIOSize in ESXi is 32767 (32 MB). Even if the storage array reports a different block size to the host, ESXi continues to use the default DiskMaxIOSize of 32 MB. Because of this, block transfers between the virtual machine and the storage array may be interrupted or incomplete.The result is that the virtual machine may fail to boot using EFI.

Resolution

This issue is resolved in VMware ESXi 6.0, Patch Release ESXi600-201909001, VMware ESXi 6.5, Patch Release ESXi650-201811002 and VMware ESXi 6.7 Update 1 Release.

Refer also to release notes at https://docs.vmware.com/en/VMware-vSphere/index.html
To download patches go to MyVMware
 
 
 


Workaround:
To work around this issue, you must set the ESXi DiskMaxIOSize parameter on each host that sees the storage array's LUNs to the IO block size value advertised by the array.
Note: With Dell EMC PowerStore storage, this parameter should be set to 1 MB . With XtremIO storage, this parameter should be set to 4 MB.

To set the ESXi DiskMaxIOSize parameter:
  1. Put the host in the Maintenance Mode.
  2. Set the ESXi DiskMaxIOSize parameter to 1 MB (for PowerStore) and 4 MB(for XtremIO) by running this command:
 
For PowerStore:

esxcli system settings advanced set -o "/Disk/DiskMaxIOSize" --int-value 1024
 
For XtremIO:

esxcli system settings advanced set -o "/Disk/DiskMaxIOSize" --int-value 4096
  1. Run this command to verify if the parameter is 1 MB or 4 MB:

    esxcli system settings advanced list -o "/Disk/DiskMaxIOSize"

    You see output similar to:

    Path: /Disk/DiskMaxIOSize
    Type: integer
    Int Value: 1024 (for PowerStore) or 4096(for XtremIO)
    Default Int Value: 32767
    Min Value: 32
    Max Value: 32767
    String Value:
    Default String Value:
    Valid Characters:
    Description: Max Disk READ/WRITE I/O size before splitting (in KB)

Alternatively, you can reduce the size of the IO requests passed to the storage device using the VMware vSphere Client program:

  1. Navigate to Host > Configuration.
  2. Click Advanced Settings.
  3. Navigate to Disk.
  4. Change the Disk.DiskMaxIOSize value to 1024 (for PowerStore) or 4096(for XtremIO).

Note: You can change this setting without rebooting the host or putting it in the maintenance mode.


Additional Information

For more information about ESXCLI commands, see vSphere Command-Line Interface Documentation for your Vsphere release.

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box.
EMC XtremIO ストレージからの EFI を使用した Windows 仮想マシンの起動が失敗し、次のエラーが表示される： 起動システムが無効です。イメージ データが破損しています

Impact/Risks:
If there are other devices on the host which are capable of servicing large IOs, the IOs to these devices will also be split as per DiskMaxIOSize setting and can  affect all VMs on the ESXi host and possibly affect the performance

Powered by Wolken Software