Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Abnormal DAVG and KAVG values observed during VAAI operations (2012288)

Symptoms

When performing VAAI operations (such as clone, template deploy, or eagerzeroedthick disk creation), abnormal DAVG and/or KAVG values may be observed on the disk (u) or device (d) views within esxtop such as:

DEVICE                                PATH/WORLD/PARTITION DQLEN WQLEN ACTV QUED %USD  LOAD   CMDS/s  READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/cmd GAVG/cmd QAVG/cmd
mpx.vmhba0:C0:T0:L0                            -               1     -    0    0    0  0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
naa.6006016061711d0080eebbe70cb1dd11           -              64     -    0    0    0  0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
naa.60060160e4a02c00a06219199aa3e011           -              64     -    0    0    0  0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
naa.60060160e4a02c00e4cba4ebaba9e011           -              64     -    8    0   12  0.12    62.15     0.39    20.85     0.00     0.06     0.15    94.91    95.06     0.00
naa.600605b002f26c6014f779d81b90cc94           -             128     -    0    0    0  0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
 
 
 9:02:45pm up  2:36, 350 worlds, 0 VMs, 0 vCPUs; CPU load average: 0.00, 0.00, 0.00

 ADAPTR PATH                 NPTH   CMDS/s  READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/cmd GAVG/cmd QAVG/cmd
 vmhba0 -                       1     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
 vmhba1 -                       1     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
 vmhba2 -                      10    64.21     0.00    21.47     0.00     0.07    96.74     0.00    96.75     0.00
 vmhba3 -                      10     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
vmhba33 -                       0     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00

Resolution

When VAAI commands are issued via VAAI Filter, there are actually 2 commands sent. These are top-layer commands which are issued and are never sent to the actual device (they stay within the ESX kernel). These commands are intercepted by the VAAI filter and the VAAI plugin, and are replaced by the vendor-specific commands, which are issued to the device.
 
This is why esxtop shows device statistics for the top-level commands only, and as a result the values for DAVG and KAVG seem unusual when compared to results obtained when VAAI is not enabled . 

In this instance (and only for this instance), the DAVG and KAVG observed in esxtop should not be interpreted as a performance issue, absent of any other symptoms. The DAVG and KAVG values observed for non-VAAI workloads are still valid.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 4 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 4 Ratings
Actions
KB: