ESXi host fails with PF Exception 14 PSOD.

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

ESXi host crashes with PF Exception 14 PSOD.

You see the PSOD backtraces similar to either of these:

2017-04-01T13:44:24.863Z cpu21:32908)@BlueScreen: #PF Exception 14 in world 32908:retireWld.00 IP 0x418019e95f5e addr 0x0

PTEs:0x146a6b027;0x148d7c027;0x0;

2017-04-01T13:44:24.863Z cpu21:32908)Code start: 0x418019600000 VMK uptime: 8:21:16:47.899

2017-04-01T13:44:24.863Z cpu21:32908)0x4390c461b600:[0x418019e95f5e]lpfc_scsi_cmd_iocb_cmpl@<None>#<None>+0x5da stack: 0x0

2017-04-01T13:44:24.864Z cpu21:32908)0x4390c461b740:[0x418019e9cf8e]lpfc_sli4_fcp_process_wcqe@<None>#<None>+0xce stack: 0x3ffffff

2017-04-01T13:44:24.864Z cpu21:32908)0x4390c461b900:[0x418019e9d2bc]lpfc_sli4_fcp_handle_wcqe@<None>#<None>+0xfc stack: 0x43917a627b00

2017-04-01T13:44:24.865Z cpu21:32908)0x4390c461b960:[0x418019ea524c]lpfc_sli4_handle_eqe@<None>#<None>+0x734 stack: 0x6d1cad6b392ec

2017-04-01T13:44:24.865Z cpu21:32908)0x4390c461ba20:[0x418019ea5d41]lpfc_sli4_intr_bh_handler@<None>#<None>+0x89 stack: 0x22

2017-04-01T13:44:24.865Z cpu21:32908)0x4390c461ba50:[0x418019658e09]IntrCookieBH@vmkernel#nover+0x299 stack: 0x0

2017-04-01T13:44:24.866Z cpu21:32908)0x4390c461baf0:[0x41801963307e]BH_DrainAndDisableInterrupts@vmkernel#nover+0xe2 stack: 0x1010000000

2017-04-01T13:44:24.866Z cpu21:32908)0x4390c461bb80:[0x418019656e22]IDT_IntrHandler@vmkernel#nover+0x1c6 stack: 0x20

2017-04-01T13:44:24.866Z cpu21:32908)0x4390c461bbb0:[0x4180196c8044]gate_entry_@vmkernel#nover+0x0 stack: 0x0

2017-04-01T13:44:24.867Z cpu21:32908)0x4390c461bc70:[0x4180199048aa]Power_HaltPCPU@vmkernel#nover+0x1ee stack: 0x417fd9883f20

2017-04-01T13:44:24.867Z cpu21:32908)0x4390c461bcc0:[0x418019811c48]CpuSchedIdleLoopInt@vmkernel#nover+0x2f8 stack: 0x6d1cae3406547

2017-04-01T13:44:24.868Z cpu21:32908)0x4390c461bd40:[0x4180198153a3]CpuSchedDispatch@vmkernel#nover+0x16b3 stack: 0x0

2017-04-01T13:44:24.868Z cpu21:32908)0x4390c461be60:[0x418019815f68]CpuSchedWait@vmkernel#nover+0x240 stack: 0x0

2017-04-01T13:44:24.869Z cpu21:32908)0x4390c461bee0:[0x41801981649f]CpuSched_SleepUntilTC@vmkernel#nover+0x8f stack: 0x2001

2017-04-01T13:44:24.869Z cpu21:32908)0x4390c461bf40:[0x418019658934]IntrCookieRetireLoop@vmkernel#nover+0x214 stack: 0x418045400224

2017-04-01T13:44:24.869Z cpu21:32908)0x4390c461bfd0:[0x418019816bfe]CpuSched_StartWorld@vmkernel#nover+0xa2 stack: 0x0

2017-04-01T13:44:24.875Z cpu21:32908)base fs=0x0 gs=0x418045400000 Kgs=0x0

OR

2017-07-17T13:27:38.637Z cpu35:66338)@BlueScreen: #PF Exception 14 in world 66338:Cmpl-vmhba0- IP 0x418017b17c85 addr 0x0 PTEs:0x2af8774027;0x2af87bc027;0xbfffffffff001;

2017-07-17T13:27:38.638Z cpu35:66338)Code start: 0x418017200000 VMK uptime: 82:02:57:56.159

2017-07-17T13:27:38.638Z cpu35:66338)0x43919911b780:[0x418017b17c85]lpfc_scsi_cmd_iocb_cmpl@(lpfc)#<None>+0x561 stack: 0x43919911b840

2017-07-17T13:27:38.638Z cpu35:66338)0x43919911b8a0:[0x418017b225e7]lpfc_sli_handle_fast_ring_event@(lpfc)#<None>+0x48b stack: 0x805004

2017-07-17T13:27:38.639Z cpu35:66338)0x43919911bad0:[0x418017b340ca]lpfc_sli_fp_intr_handler@(lpfc)#<None>+0x3e stack: 0x1a

2017-07-17T13:27:38.639Z cpu35:66338)0x43919911baf0:[0x4180172d0a54]IntrCookieBH@vmkernel#nover+0x1e0 stack: 0x0

2017-07-17T13:27:38.640Z cpu35:66338)0x43919911bb90:[0x4180172b0d30]BH_DrainAndDisableInterrupts@vmkernel#nover+0x100 stack: 0x43919911bc70

2017-07-17T13:27:38.640Z cpu35:66338)0x43919911bc20:[0x4180172d2862]IntrCookie_VmkernelInterrupt@vmkernel#nover+0xc6 stack: 0x73

2017-07-17T13:27:38.641Z cpu35:66338)0x43919911bc50:[0x41801732df1d]IDT_IntrHandler@vmkernel#nover+0x9d stack: 0x418048c00640

2017-07-17T13:27:38.641Z cpu35:66338)0x43919911bc70:[0x41801733c044]gate_entry_@vmkernel#nover+0x0 stack: 0x0

2017-07-17T13:27:38.641Z cpu35:66338)0x43919911bd30:[0x41801728aec2]Power_ArchSetCState@vmkernel#nover+0x106 stack: 0x7fffffffffffffff

2017-07-17T13:27:38.642Z cpu35:66338)0x43919911bd60:[0x4180174c44b3]CpuSchedIdleLoopInt@vmkernel#nover+0x39b stack: 0x800

2017-07-17T13:27:38.642Z cpu35:66338)0x43919911bdd0:[0x4180174c6d6a]CpuSchedDispatch@vmkernel#nover+0x114a stack: 0x410000000001

2017-07-17T13:27:38.643Z cpu35:66338)0x43919911bf00:[0x4180174c7fe2]CpuSchedWait@vmkernel#nover+0x27a stack: 0x100000000000200

2017-07-17T13:27:38.643Z cpu35:66338)0x43919911bf80:[0x4180174c80b5]CpuSched_NoEvqWait@vmkernel#nover+0x19 stack: 0x4306d83a0b28

2017-07-17T13:27:38.643Z cpu35:66338)0x43919911bf90:[0x418017534790]SCSICompletionWorld@vmkernel#nover+0x10c stack: 0x0

2017-07-17T13:27:38.644Z cpu35:66338)0x43919911bfe0:[0x4180174c8c95]CpuSched_StartWorld@vmkernel#nover+0x99 stack: 0x0

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.0

Resolution

To resolve this issue, Upgrade brcmfcoe driver to 11.2.1153.13 and lpfc driver to 11.2.156.20 or newer version.To resolve this issue on Huawei servers, upgrade to lpfc 11.1.183.641.

For more information on driver see, Broadcom site.

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

Additional Information

ESXi 主机崩溃并出现 lpfc 驱动程序所致的 PF 异常 14 PSOD
lpfc ドライバによって PF 例外 14 PSOD が発生して ESXi ホストがクラッシュする