Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Large I/O block size operations show high latency in Windows 2008 in ESX 4.x and ESXi 4.x / 5.x (2036863)

Symptoms

In VMware ESX 4.x and ESXi 4.x/5.x, when performing a task that utilizes large I/O block sizes in Windows 2008, performance charts in vCenter or ESXTOP display very high device latency or DAVG, even though the actual throughput is excellent.

Cause

Unlike previous Windows releases, Windows 2008 can issue much larger I/O block sizes for certain operations, such as file copy. However, this can be reproduced with other applications, such as IOmeter with large I/O block size test parameters.

This issue can also be reproduced in a Linux virtual machine with the dd command and issuing commands with a large block size (i.e. - 1M).

Resolution

High latency is due to a large I/O block size that needs to be split up into smaller I/O sizes in order to be transmitted to the storage device. When this I/O split occurs, the measure of latency is against the entire command, not the individual chunks.
 
Therefore, you must wait for all chunks to complete to the array before reporting back on the latency. This results in a false positive when high latency is observed to the array, when a performance problem does not actually exist. 

Note: Reducing the Disk.DiskMaxIOSize advanced setting in the ESX/ESXi host does not improve the latency results because the Guest operating system is the one issuing large I/O block size. For more information, see Tuning ESX/ESXi for better storage performance by modifying the maximum I/O block size (1003469).

The Windows registry may be altered to issue smaller I/O block sizes, resulting in lower latency. However, this is merely a false positive. For more information, see:

While one may think that issuing a virtual machine migration or virtual machine deployment can be considered a large block operation, in reality the vmkernel issues I/Os block size of 32k. So, this is significantly smaller than what the guest operating system issues, resulting in no high  latency for this operation.

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 5 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 5 Ratings
Actions
KB: