Knowledge Base
The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides

|
Changing the Queue Depth for QLogic and Emulex HBAs
Details
If you are having unsatisfactory performance of your hardware bus adapters (HBAs), or your SAN storage processors or heads are over-utilized, you can adjust the ESX hosts' maximum queue depth value. The maximum value refers to the queue depths reported for various paths to the LUN. When you lower this value, it throttles the ESX host's throughput and alleviates SAN contention concerns if multiple hosts are over-utilizing the storage and are filling its command queue.
When one virtual machine is active on a LUN, you only need to set the maximum queue depth. When multiple virtual machines are active on a LUN, the value of Disk.SchedNumReqOutstanding value is also relevant. The queue depth value, in this case, is equal to whichever value is the lowest of the two settings: adapter queue depth or Disk.SchedNumReqOutstanding. The recommendation is to set both Disk.SchedNumReqOutstanding and the adapter queue depth to the same value.
To set the Disk.SchedNumReqOutstanding value, see Setting the Maximum Outstanding Disk Requests per Virtual Machine (1268).
For more information about setting queue depth for an Emulex host bus adapter (HBA) on ESX 3.0, see Setting the Queue Depth for an Emulex HBA on ESX Server 3.0 (8535211).
For more information about how storage queues affect performance, see http://communities.vmware.com/docs/DOC-6490.
For more information about the default values and configuration options for the QLogic VMware drivers, see Fiber Channel Adapter for VMware ESX User's Guide.
For more information about the default values and configuration options for the Emulex VMware drivers, see Emulex Driver for VMware ESX User Manual.
Note: The preceding links were available as of June 25, 2010. If you find the links to be broken, provide feedback on the article and a VMware employee will update the article as necessary.
Solution
VMware ESX 5.x
To adjust the queue depth for an HBA:
- Verify which HBA module is currently loaded by entering one of these commands on the service console:
For QLogic:
esxcli system module list | grep qla
For Emulex:
esxcli system module list | grep lpfc - Run one of these commands:
Note: The examples show the QLogic qla2xxx and Emulex lpfc820 modules. Use the appropriate module based on the outcome of the previous step.
For QLogic:
esxcli system module parameters set -p ql2xmaxqdepth=64 -m qla2xxx
For Emulex:
esxcli system module parameters set -p lpfc0_lun_queue_depth=64 -m lpfc820
In this case, the HBAs represented by ql2x and lpfc0 have their LUN queue depths set to 64.
Note: If all Emulex cards on the host need to be updated, apply the global parameter, lpfc_lun_queue_depth instead.
- Reboot your host.
-
Run this command to confirm if your changes are applied:
esxcli system module parameters list -m <driver>
Where <driver> is your QLogic or Emulex adapter driver module, such as lpfc820 or qla2xxx.
The output appears similar to:
Name Type Value Description
-------------------------- ---- ----- --------------------------------------------------
.....
ql2xmaxqdepth int 64 Maximum queue depth to report for target devices.
.....
VMware ESX 4.x
To adjust the queue depth for an HBA:
- Verify which HBA module is currently loaded by entering one of the following commands on the service console:
For QLogic:
vmkload_mod -l | grep qla
For Emulex:
vmkload_mod -l | grep lpfc - Run one of these commands:
Note: The examples show the QLogic qla2xxx and Emulex lpfc820 modules. Use the appropriate module based on the outcome of the previous step.
For QLogic:
esxcfg-module -s ql2xmaxqdepth=64 qla2xxx
For Emulex:
esxcfg-module -s lpfc0_lun_queue_depth=64 lpfc820
In this case, the HBAs represented by ql2x and lpfc0 will have their LUN queue depths set to 64.
- Reboot your host.
- Run this command to confirm if your changes are applied:
esxcfg-module -g <driver>
Where <driver> is your QLogic or Emulex adapter driver module, such as lpfc820 or qla2xxx.
ESX 3.5
To set the queue depth for a QLogic HBA:
- Log in to the service console as root.
- Verify which QLogic HBA module is currently loaded:
vmkload_mod -l | grep qla
Depending on the model of the HBA, the module can be one of the following:
qla2300_707_vmw
qla4010
qla4022 - Run these commands:
esxcfg-module -s ql2xmaxqdepth=64 qla2300_707_vmw
esxcfg-boot –b
The HBA represented by ql2x will have its LUN queue depth set to 64.
The example shows the qla2300_707 module. Use the appropriate module based on the outcome of Step 2.
- Reboot the host.
- Run this command to confirm if your changes are applied:
esxcfg-module -g <driver>
Where <driver> is your QLogic or Emulex adapter driver module, such as lpfc820 or qla2xxx.
To change the queue depth of an Emulex HBA:
- Log on to the service console as root.
- Verify which Emulex HBA module is currently loaded:
vmkload_mod -l | grep lpfc
- For a single instance of an Emulex HBA on the system, run these commands:
Note: The example shows the lpfc_740 module. Use the appropriate module based on the outcome of Step 2.
esxcfg-module -s lpfc0_lun_queue_depth=64 lpfc_740
esxcfg-boot –b
In this case, the HBA represented by lpfc0 will have its LUN queue depth set to 64.
- For multiple instances of an Emulex HBA being present on the system, run these commands:
esxcfg-module -s "lpfc0_lun_queue_depth=64 lpfc1_lun_queue_depth=64" lpfc_7xx
esxcfg-boot –b
In this case, both HBAs lpfc0 and lpfc1 will have their LUN queue depths set to 64.
- Reboot the host.
- Run this command to confirm if your changes are applied:
esxcfg-module -g <driver>
Where <driver> is your QLogic or Emulex adapter driver module, such as lpfc820 or qla2xxx.
ESX 2.x
To set the queue depth of the QLogic adapter on ESX 2.x:
- Edit the file /etc/vmware/hwconfig and locate the line for the QLogic adapter. It looks similar to:
device.3.12.0.name = "QLogic Corp QLA2312/2340 (rev 02)"
Make note of the string of three numbers between device and name. -
In the /etc/vmware/hwconfig file, add one of the following lines to the end of the file:
-
For ESX Server 2.0, add this line:
device.vmnix.3.12.0.options = "ql2xmaxqdepth=nn" -
For ESX Server 2.1 and later versions, add this line:
device.esx.3.12.0.options = "ql2xmaxqdepth=nn"
In the example, 3.12.0 is the string of numbers taken from the QLogic line located in the previous step, and nn is the desired queue depth maximum. Be sure to save your changes.
Note: The second character in ql2xmaxqdepth is a lowercase " L".
-
-
Reboot the host.
Tags
Keywords
Request a Product Feature
To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.
Actions
- KB Article:
- Updated:
- Categories:
- Product Family:
- Products:
- Product Versions:

