Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides

Search the VMware Knowledge Base (KB)

View by Article ID

Enabling Support for NetQueue on the Intel 82598 and 82599 10 Gigabit Ethernet Controller (1004278)

Details

This article provides steps to enable support for NetQueue on the Intel 82598 and 82599 10 Gigabit Ethernet Controller.

Note: NetQueue is enabled by default on ESX 4.0 at the system level.

Solution

The ixgbe driver supports adapters based on the Intel 82598 and 82599 10 Gigabit Ethernet controller. This driver utilizes VMware's NetQueue technology to enable Intel Virtual Machine Device Queues (VMDq).

To enable VMDq:

Enable NetQueue in VMkernel using VMware Infrastructure (VI) Client.
1. Choose Configuration > Advanced Settings > VMkernel.
2. Select the checkbox for VMkernel.Boot.netNetqueueEnabled.
Enable the ixgbe module in the service console of the ESX Server host:

# esxcfg-module -e ixgbe
Set the required NetQueue options for the ixgbe module:
- MSI-X enables the Ethernet controller to direct interrupt messages to multiple processor cores. This feature is required for NetQueue to function with VMDq. In the examples below, the parameter InterruptType with a value of 2 specifies MSI-X.
- A value for VMDQ must exist to indicate the number of receive queues. The range of values for the VMDQ parameter is 1 to 16. A value of 16 for VMDQ sets the number of receive queues to the maximum. The Intel 82598/82599 10 Gigabit Ethernet Controller provides 32 transmit queues and 64 receive queues per port, which can be mapped to a maximum of 16 processor cores.
  
  Note: For optimal performance, ensure that the number of each kind of queue is smaller than the number of CPU cores.
  
  Note: To check the current options for the ixgbe module in case you need to revert later, run the command:
  
  # esxcfg-module -g ixgbe
- For a single port and the maximum number of receive queues, use the command:
  
  # esxcfg-module -s "InterruptType=2 VMDQ=16" ixgbe
- For two ports, add the values in a comma-separated list for each parameter. For example:
  
  # esxcfg-module -s "InterruptType=2,2 VMDQ=16,16" ixgbe
Reboot the ESX host.

Notes: If you are using jumbo frames:
- You also need to change the values for netPktHeapMinSize to 32 and netPktHeapMaxSize to 128. For more information, see Set netPktHeapMinSize=32 and netPktHeapMaxSize=128 to Enable Jumbo Frames and NetQueue (1004593).
- The maximum number of RX queues supported by the ixgbe driver is 4.

To verify that VMDq has been successfully enabled:

Verify NetQueue has been enabled:

# cat /etc/vmware/esx.conf

Confirm that this line has been added into the file:
/vmkernel/netNetqueueEnabled = "TRUE"
Verify the options configured for the ixgbe module:

# esxcfg-module -g ixgbe

The output is similar to:

ixgbe enabled = 1 options = 'InterruptType=2,2 VMDQ=16,16'

The enabled value should be equal to 1, which indicates the ixgbe module is enabled. InterruptType should be equal to 2 and include an entry for each port. (This is a 2-port example.) VMDQ should be 16 to enable the maxiumum number of receive queues and should also have an entry for each port.
Query which ports have loaded the driver using esxcfg-nics -l. Then query the statistics using ethtool. If VMDq is enabled, statistics for multiple receive queues are shown (rx_queue_0 through rx_queue_15 in the example below).

# esxcfg-nics -l Name PCI Driver Link Speed Duplex MTU Description vmnic0 04:00.00 bnx2 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T vmnic1 08:00.00 bnx2 Down 0Mbps Half 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T vmnic2 0b:00.00 ixgbe Up 10000MbpsFull 9000 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection vmnic3 0b:00.01 ixgbe Up 10000MbpsFull 9000 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection vmnic4 0d:00.00 ixgbe Up 10000MbpsFull 1500 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection vmnic5 0d:00.01 ixgbe Up 10000MbpsFull 1500 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection # ethtool -S vmnic2 NIC statistics: rx_packets: 0 tx_packets: 56 rx_bytes: 0 tx_bytes: 3584 lsc_int: 1 tx_busy: 0 non_eop_descs: 0 rx_errors: 0 tx_errors: 0 rx_dropped: 0 tx_dropped: 0 multicast: 0 broadcast: 0 rx_no_buffer_count: 0 collisions: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 rx_fifo_errors: 0 rx_missed_errors: 0 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 tx_tcp4_seg_ctxt: 0 tx_flow_control_xon: 0 rx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_flow_control_xoff: 0 rx_csum_offload_good: 0 rx_csum_offload_errors: 0 tx_csum_offload_ctxt: 0 rx_header_split: 0 low_latency_interrupt: 0 alloc_rx_page_failed: 0 alloc_rx_buff_failed: 0 tx_queue_0_packets: 56 tx_queue_0_bytes: 3360 rx_queue_0_packets: 0 rx_queue_0_bytes: 0 rx_queue_1_packets: 0 rx_queue_1_bytes: 0 rx_queue_2_packets: 0 rx_queue_2_bytes: 0 rx_queue_3_packets: 0 rx_queue_3_bytes: 0 rx_queue_4_packets: 0 rx_queue_4_bytes: 0 rx_queue_5_packets: 0 rx_queue_5_bytes: 0 rx_queue_6_packets: 0 rx_queue_6_bytes: 0 rx_queue_7_packets: 0 rx_queue_7_bytes: 0 rx_queue_8_packets: 0 rx_queue_8_bytes: 0 rx_queue_9_packets: 0 rx_queue_9_bytes: 0 rx_queue_10_packets: 0 rx_queue_10_bytes: 0 rx_queue_11_packets: 0 rx_queue_11_bytes: 0 rx_queue_12_packets: 0 rx_queue_12_bytes: 0 rx_queue_13_packets: 0 rx_queue_13_bytes: 0 rx_queue_14_packets: 0 rx_queue_14_bytes: 0 rx_queue_15_packets: 0 rx_queue_15_bytes: 0

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

1 2 3 4 5
18 Ratings

1 2 3 4 5
18 Ratings

Actions

Bookmark Document

KB:

Did this article help you?
This article resolved my issue. This article did not resolve my issue. This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)