Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Troubleshooting poor local storage controller performance (1007688)

Symptoms

  • When cloning or migrating virtual machines to the local storage of an ESX host, the process takes much longer to complete compared to other hosts. 
  • Performing writes into a dump file may take a lot longer on this host. 
  • Performing reads from a file may take longer than other hosts, but not as long as the above write performance test. 
  • There may be no errors in the logs (such as /var/log/messages or /var/log/vmkernel) to follow up with.

Resolution

You can use the utility esxtop to help assess the bottleneck.
 
To use esxtop to help assess the bottleneck:
  1. Log into the ESX server console.
  2. Execute the command:

    esxtop
     
  3. Press d to go to the disk statistics screen.
  4. Press f to go to the field selector.
  5. If the J does not have an asterisk (*) beside it, press j to add that field to the view. You can remove E from the view.
  6. Look for the DAVG/cmd field (device latency). This gives you an idea of how long the ESX host is waiting (in milliseconds) for SCSI commands submitted to the storage to come back with a response.
If the commands appear to have a considerable amount of latency (more than 50 ms) for the local vmhba, device contention is being encountered.
  • Write performance is degraded on RAID controllers that have a discharged or absent battery because the write cache is disabled.

    Check the battery to ensure that it is present and/or charged. Your SCSI controller may not necessarily be faulty.
       
  • Your RAID type may have some bearing on performance, particularly if it is not extremely poor, yet lower than your expected baseline.

    The number of spindles/disks used per volume, along with the speed of the disks has an impact on performance. 
     
  • If a volume is degraded, such as having a failed drive in a RAID-5 disk set, read and write performance is severely hampered. You can check your RAID status in your array manager, or by observing the server's indicator/warning lights, when and where applicable. 
     
  • As a troubleshooting measure, it may be worth breaking the disks into a RAID-less configuration, to eliminate this as being a possibility. This may require a reinstallation of VMware ESX.

For further troubleshooting, engage your server hardware vendor.

Additional Information

For information about write back caching on Storage Area Networks (SAN), see Write-cache disabled on storage array causing performance issues or failures (1002282). 

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 3 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 3 Ratings
Actions
KB: