Search the VMware Knowledge Base (KB)
View by Article ID

"hpsa_update_scsi_devices or detect_controller_lockup_thread" PSOD error (2075978)

  • 43 Ratings
Language Editions

Symptoms

Running the host on HP hardware fails with a purple diagnostic screen error, you see one of these error:
  • hpsa_update_scsi_devices@<None>#<None>+0x39c
    hpsa_scan_start@<None>#<None>+0x187
    hpsa_kickoff_rescan@<None>#<None>+0x20f
    kthread@com.vmware.driverAPI#9.2+0x185
    LinuxStartFunc@com.vmware.driverAPI#9.2+0x97
    vmkWorldFunc@vmkernel#nover+0x83
    CpuSched_StartWorld@vmkernel#nover+0xfa

  • Panic: 892: Saved backtrace: pcpu X TLB NMI
    _raw_spin_failed@com.vmware.driverAPI#9.2+0x5
    detect_controller_lockup_thread@#+0x3a9
     kthread@com.vmware.driverAPI#9.2+0x185
     LinuxStartFunc@com.vmware.driverAPI#9.2+0x97
     vmkWorldFunc@vmkernel#nover+0x83                     
     CpuSched_StartWorld@vmkernel#nover+0xfa
     PCPU X locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU9s): X)

Note: For additional symptoms and log entries, see Additional Information section.

Purpose

To resolve the hpsa_update_scsi_devices or detect_controller_lockup_thread purple diagnostic screen error on an ESXi host running on HP server, apply the updated driver provided by HP.

Resolution

This is a known issue affecting VMware ESXi 5.x. 

To resolve this issue, apply the updated driver provided by HP. 

For more information, see: 
For all BL685c G7 blades and DL360p Gen8 servers, HP recommends to update to ESXi 5.5 update1 to the June 2014 version.

The reasons for the recommendation are:
  1. smx-provider memory leak issue is resolved.
  2. Several issues for the hpsa driver are resolved in the .60 version found in new June 2014 version of ESXi 5.5 update1. The previous version of the hpsa driver was .50 and was problematic.
For the DL360p Gen8 servers, it is recommended to update the Firmware on all servers to 1.51. This is a critical update to avoid NMI events which would cause PSOD in your environment. For more information, see HP Support Center.

It is also recommended to check the DL360p Gen8 servers to make sure that they are at least at Feb 2014 system ROM. This is to correct a possible IPMI issue.
 
If this issue persists after the driver upgrade:
  • Open a HP Support Request, reference HP case 4648045806.
  • If this issue persists, open a support request with VMware Support. For more information, see Filing a Support Request in My VMware (2006985).
  • Provide VMware support your HP case number.
Disclaimer : VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

Additional Information

You see these additional symptoms:
  • Cannot run the host on Hewlett Packard (HP) hardware.
  • Before host becomes unresponsive, in the /var/log/vmkernel.log file, you see entries similar to:

    WARNING: LinDMA: Linux_DMACheckConstraints:149: Cannot map machine address = 0xfffffffffff, length = 49160 for device 0000:03:00.0; reason = buffer straddles device dma boundary (0xffffffff)
    WARNING: Heap: 4089: Heap_Align(vmklnx_hpsa, 32768/32768 bytes, 8 align) failed.  caller: 0x41802dcb1f91cpu4:1696102)<4>hpsa 0000:09:00.0: out of memory in adjust_hpsa_scsi_table

  • Before you see a purple diagnostic screen, in the /var/log/vmkernel.log file, you see entries similar to:

    Note
    : These are multiple memory error messages from the hpsa driver.

    out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    WARNING: Heap: 3622: Heap vmklnx_hpsa (39113576/39121768): Maximum allowed growth (8192) too small for size (20480)


    cpu7:1727675)<4>hpsa 0000:06:00.0: out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    cpu2:1727677)<4>hpsa 0000:0c:00.0: out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    cpu4:1727676)<4>hpsa 0000:09:00.0: out of memory at vmkdrivers/src_9/drivers/hpsa/hpsa.c:3562
    cpu3:1727738)WARNING: LinDMA: dma_alloc_coherent:726: Out of memory
    cpu3:1727738)<3>hpsa 0000:06:00.0: cmd_special_alloc returned NULL!

Tags

ESXi host is unresponsive after losing access to storage, ESXi Spamming Interrupt received on invalid vector, ignoring it,pcpu 22 locked up failed to ack tlb invalidate,hpsa_update_scsi_devices or detect_controller_lockup_thread

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 43 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 43 Ratings
Actions
KB: