VMware
 

Knowledge Base

Search the Knowledge Base:
Products:
Search In:
 

Changing the Queue Depth for QLogic and Emulex HBAs

Details

If you notice unsatisfactory performance of your HBAs, you can change their maximum queue depths. The maxiumum value refers to the queue depths reported for various paths to the LUN.
 
When one virtual machine is active on a LUN, you only need to set the maximum queue depth. When multiple virtual machines are active on a LUN, the value of Disk.SchedNumReqOutstanding value is also relevant. The queue depth value, in this case, is equal to whichever value is the lowest of the two settings: adapter queue depth or Disk.SchedNumReqOutstanding. The recommendation is to set both Disk.SchedNumReqOutstanding and the adapter queue depth to the same value.
 
The following instructions describe how to set the queue depth. To set the Disk.SchedNumReqOutstanding value, see Setting the Maximum Outstanding Disk Requests per Virtual Machine (1268).
 
For more information about how storage queues affect performance, see http://communities.vmware.com/docs/DOC-6490.

For more information about the default values and configuration options for the QLogic VMware drivers, see Fiber Channel Adapter for VMware ESX User's Guide.
 
For more information about the default values and configuration options for the Emulex VMware drivers, see Emulex Driver for VMware ESX User Manual.
 
Note: The preceding links were available as of December 22, 2009. If you find the links to be broken, provide feedback on the article and a VMware employee will update the article as necessary.
 

Solution

ESX 4.0

To adjust the queue depth for an HBA:

  1. Verify which HBA module is currently loaded by entering one of the following commands on the service console:

    For QLogic:

    vmkload_mod -l | grep qla2xxx

    For Emulex:

    vmkload_mod -l | grep lpfcdd

  2. Run one of the following commands. The examples show the QLogic qla2300_707 and Emulex lpfcdd_7xx modules. Use the appropriate module based on the outcome of the previous step.

    For QLogic:

    esxcfg-module -s ql2xmaxqdepth=64 qla2300_707

    For Emulex:

    esxcfg-module -s lpfc0_lun_queue_depth=16 lpfcdd_7xx

    In this case, the HBAs represented by ql2x and lpfc0 will have their LUN queue depths set to 64.
     
  3. Reboot your host.

ESX 3.0/3.5

To set the queue depth for a QLogic HBA:

  1. Log in to the service console as root.
  2. Verify which QLogic HBA module is currently loaded:

    vmkload_mod -l | grep qla2300

    Depending on the model of the HBA, the module can be one of the following:

    qla2300_707 (ESX 3.0.x)
    qla2300_707_vmw (ESX 3.5)
     
  3. Run the following commands.

    esxcfg-module -s ql2xmaxqdepth=64 qla2300_707
    esxcfg-boot –b

    The HBA represented by ql2x will have its LUN queue depth set to 64.

    The example shows the qla2300_707 module. Use the appropriate module based on the outcome of Step 2.
     
  4. Reboot.

To change the queue depth of an Emulex HBA:

  1. Log on to the service console as root.
  2. Verify which Emulex HBA module is currently loaded:

    vmkload_mod -l | grep lpfcdd

    Depending on the model of the HBA, the module can be one of the following:

    lpfcdd_7xx
    lpfcdd_732
     
  3. For a single instance of an Emulex HBA on the system, run the following commands. The example shows the lpfcdd_7xx module. Use the appropriate module based on the outcome of Step 2.

    esxcfg-module -s lpfc0_lun_queue_depth=16 lpfcdd_7xx
    esxcfg-boot –b

    In this case, the HBA represented by lpfc0 will have its LUN queue depth set to 16.
     
  4. For multiple instances of an Emulex HBA being present on the system, run the following commands:

    esxcfg-module -s "lpfc0_lun_queue_depth=16 lpfc1_lun_queue_depth=16" lpfcdd_7xx
    esxcfg-boot –b

    In this case, both HBAs lpfc0 and lpfc1 will have their LUN queue depths set to 16.
     
  5. Reboot. 

ESX 2.x

To set the queue depth of the QLogic adapter on ESX 2.x:
  1. Edit the file /etc/vmware/hwconfig and locate the line for the QLogic adapter. It looks similar to this:

    device.3.12.0.name = "QLogic Corp QLA2312/2340 (rev 02)"

    Take note of the string of three numbers between device and name.
     
  2. Continuing in /etc/vmware/hwconfig, add one of the following lines to the end of the file:

    1. For ESX Server 2.0, add this line:

      device.vmnix.3.12.0.options = "ql2xmaxqdepth=nn"
    2. For ESX Server 2.1 and later versions, add this line:

      device.esx.3.12.0.options = "ql2xmaxqdepth=nn"

      In the example, 3.12.0 is the string of numbers taken from the QLogic line located in the previous step, and nn is the desired queue depth maximum. Be sure to save your changes.

      Note: The second character in       ql2xmaxqdepth is a lowercase "L".
       
  3. Reboot the server.

Keywords

1267; 1269; 1268; Fibre; Channel; storage; urlz

Feedback

Rate this article:
(6 Ratings)

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
Email address (optional)
Submit
Rate this article:
(6 Ratings)
Actions
  • KB Article: 1267
  • Updated: Dec 22, 2009
  • Products:
    VMware ESX
  • Product Versions:
    VMware ESX 2.0.x
    VMware ESX 2.1.x
    VMware ESX 2.5.x
    VMware ESX 3.0.x
    VMware ESX 3.5.x
    VMware ESX 4.0.x