Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Lost or degraded connectivity to storage device (1009553)

Details

Summary
 
The event indicates a loss in connectivity to the specified storage device. The path indicated is the last path that went down.
 
 
Example
 
Virtual Center reports these errors in the vpx.log file: 
  • Lost connectivity to storage device naa.60a9800043346534645a433967325334. Path vmhba35:C1:T0:L7 is down. Affected datastores: "Storage1" 
  • Path redundancy to storage device or Lost path redundancy to storage device naa.60a9800043346534645a433967325334. Path vmhba35:C1:T0:L7 is down. 
 
Impact
 
Any virtual machines using the affected datastore may become unresponsive.
 
If the service console root file system is on the affected datastore, then the service console becomes unresponsive and you cannot manage your ESX host.

Solution

The storage device reading (vmhba35:C1:T0:L7) mentioned in the example earlier contains several potential failure points:

  • vmhba35 - HBA (Host Bus Adapter)
  • C1 - Channel
  • T0 - Target (Storage processor port)
  • L7 -  LUN (Logical Unit Number or Disk Unit)
To determine the actual failure or to eliminate possible issues:
Note: All mentioned commands are available via vSphere Command-Line or VMware vSphere Management Assistant.
  1. Identify the available storage paths to the reported storage device by running the esxcfg-mpath -l command . For more information, see Obtaining LUN pathing information for ESX hosts (1003973).
  2. Check that a rescan does not restore visibility to the targets. Follow the steps provided in Performing a rescan of the storage (1003988) to rescan the storage device by using the command-line interface and VMware vSphere Client.
  3. Determine whether the connectivity issue is with the iSCSI storage or the fiber storage. Perform one of the following depending on what your connectivity issue is:
    • To troubleshoot the connectivity to the iSCSI storage using the software initiator:
      1. Check whether a ping to the storage array fails from ESX. For more information, see Testing network connectivity with the Ping command (1003486).
      2. Check whether a vmkping to each network portal of the storage array fails. For more information, see Testing VMkernel network connectivity with the vmkping command (1003728).
      3. Check that the initiator is registered on the array. Contact your storage vendor for instructions on this procedure.
      4. Check that the following physical hardware is correctly functioning:
        • Ethernet switch
        • Ethernet cables between the switch and the ESX host
        • Ethernet cables between the switch and the storage array

    • To troubleshoot the connectivity to the fiber attached storage, check the following:
    1. The fiber switch zoning configuration permits the ESX host to see the storage array. Consult your switch vendor if you require assistance.
    2. The fiber switch propagates RSCN messages to the ESX hosts. For more information on configuring the fiber switch, Configuring fibre switch so that ESX Server doesn't require a reboot after a zone set change (1002301).
  4. Check the physical hardware for the following:
  • The storage processors on the array.
  • The fiber switch and the Gigabit Interface Converter (GBIC) units in the switch.
  • The fiber cables between the fiber switch and the array.
  • The array itself.

Note: You must rescan after any change is made to verify whether the targets are detected.

Update History

06/03/2010 - Specifed that the logs are located in virtual center and updated the title for KB 1003973

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 24 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 24 Ratings
Actions
KB: