Bootbank cannot be found at path '/bootbank' errors being seen after upgrading to ESXi 7.0 U2
book
Article ID: 317914
calendar_today
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
To resolve the bootbank cannot be found.
Symptoms:
ESXi host boots off of USB SD card.
ESXi goes into non-responding status.
The following vCenter alarm is generated
Alarm 'Host error' on esxi.gsslabs.org triggered by event 75855264 'Issue detected on esxi.gsslabs.org in DATACENTER: Bootbank cannot be found at path '/bootbank'
vmkwarning.log may show entries similar to:
2021-05-17T03:53:57.655Z cpu22:2097681)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-05-17T03:54:17.280Z cpu22:2097681)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...
vmkernel.log may show entries similar to:
2021-05-16T22:54:20.120Z cpu40:7579038)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-05-16T22:54:29.999Z cpu68:2097681)ScsiPath: 8058: Cancelled Cmd(0x45c900f21240) 0x1a, cmdId.initiator=0x4538f041a598 CmdSN 0x1407e8 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-05-16T22:54:29.999Z cpu68:2097681)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-05-16T22:54:29.999Z cpu68:2097681)ScsiDeviceIO: 4315: Cmd(0x45c900f21240) 0x1a, cmdId.initiator=0x4538f041a598 CmdSN 0x1407e8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-05-16T22:54:31.120Z cpu42:2097682)ScsiPath: 8058: Cancelled Cmd(0x45c900e60040) 0x0, cmdId.initiator=0x4538e9b1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.
NOTE:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment
Environment
VMware vSphere ESXi 7.0.x
Cause
USB devices have a small queue depth and due to a race condition in the ESXi storage stack, some I/O operations might not get to the device. Such I/Os queue in the ESXi storage stack and ultimately time out.