Symptoms:
Host configured SW iscsi hit PSOD randomly.
 In the host equipped with Qlogic QL41xx or QL45xx CNA, esx might encounter PSOD randomly when SW iscsi is running. The backtrace looks like below.

2023-01-14T13:51:21.183Z cpu10:13955491)World: 3072: PRDA 0x420042800000 ss 0x0 ds 0x10b es 0x10b fs 0x10b gs 0x0
2023-01-14T13:51:21.183Z cpu10:13955491)World: 3074: TR 0xf58 GDT 0x453880014000 (0xf77) IDT 0x420010b50000 (0xfff)
2023-01-14T13:51:21.183Z cpu10:13955491)World: 3075: CR0 0x80010031 CR3 0x8048138000 CR4 0x142768
2023-01-14T13:51:21.224Z cpu10:13955491)Backtrace for current CPU #10, worldID=13955491, fp=0x453913c1bd60
2023-01-14T13:51:21.224Z cpu10:13955491)0x453913c1bcb8:[0x420010ca533b]vmk_PktListPopFirstPkt@vmkernel#nover+0xb stack: 0x0, 0x45d95ca81480, 0x420011a9b949, 0x1, 0x420010c6932b
2023-01-14T13:51:21.224Z cpu10:13955491)0x453913c1bcc0:[0x420011a9b9e5]qedi_UplinkTx@(qedi)#<None>+0xf6 stack: 0x45d95ca81480, 0x420011a9b949, 0x1, 0x420010c6932b, 0x453913c1bd40
2023-01-14T13:51:21.224Z cpu10:13955491)0x453913c1bd70:[0x420010c8f495]UplinkDevDoTransmit@vmkernel#nover+0x372 stack: 0x4305b510bd80, 0xa00004410b0885b, 0x453913c1bef0, 0xc5, 0x0
2023-01-14T13:51:21.224Z cpu10:13955491)0x453913c1be60:[0x420010c8fb9f]UplinkDevTransmit@vmkernel#nover+0x178 stack: 0x4301914579c0, 0x420042800210, 0x420042800000, 0x420042800218, 0x0

VMware vSphere ESXi 7.0.3

The wrong configuration of SW iscsi makes PSOD randomly. For example, command 'esxcfg-nics -l' lists NICs of qedi as below.

   vmnic2 0000:2b:00.4 qedi Up Up 50000 Full a2:0d:c7:70:00:04 9000 QLogic Corp QLogic FastLinQ QL45xxx Series 10/25/40/50/100 GbE Controller (iSCSI)
   vmnic3 0000:2b:00.5 qedi Up Up 50000 Full a2:0d:c7:70:00:05 9000 QLogic Corp QLogic FastLinQ QL45xxx Series 10/25/40/50/100 GbE Controller (iSCSI)

   These are iSCSI NICs; which are not supposed to be used as normal NICs. But user are not aware of this and configures SW iscsi networkportal using above NICs, which make uplink device unstable and crashed randomly.

 

 

•  Validate the NIC configuration using the below steps

esxcfg-scsidevs -a

HBA Name Driver Link State UID Capabilities Description
-------- --------- ---------- ------------------------------------ ------------------- -----------
vmhba0 nvme_pcie link-n/a pcie.200 (0000:02:00.0) Marvell Technology Group Ltd Marvell NR2241 NVMe Controller
vmhba64 qedi unbound iscsi.vmhba64 Second Level Lun ID QLogic FastLinQ QL45xxx Series 10/25/40/50/100 GbE Controller (iSCSI)
vmhba65 qedi unbound iscsi.vmhba65 Second Level Lun ID QLogic FastLinQ QL45xxx Series 10/25/40/50/100 GbE Controller (iSCSI)
vmhba66 iscsi_vmk online iqn.1998-01.com.vmware:am4-esx-vh001 Second Level Lun ID iSCSI Software Adapter

• Make sure these Vmnic's are not being used for software ISCSI traffic 
• These are iSCSI NICs; which are not supposed to be used as normal NICs.

Use the NICs of qedentv as networkportal for SW ISCSI or switch to HW ISCSI

vmnic4 0000:86:00.0 qedentv Up 10000Mbps Full f4:e9:d4:ed:a0:7a 1500 QLogic Corp. QLogic FastLinQ QL41xxx 1/10/25 GbE Ethernet Adapter
vmnic5 0000:86:00.1 qedentv Up 10000Mbps Full f4:e9:d4:ed:a0:7b 1500 QLogic Corp. QLogic FastLinQ QL41xxx 1/10/25 GbE Ethernet Adapter

Impact/Risks:

• During this PSOD ,the host crashes, and all services and VMs running on the host are terminated. The VMs don't get a chance to gracefully shut down, but are instead powered off abruptly. 
• They are all caused by packet (list) corruption. Since the packets are either received or transmitted via NICs managed by the qedi drivers if its an ISCSI connection