Enabling Support for NetQueue on the Intel 82598 10 Gigabit Ethernet Controller

To enable VMDq:

1. Enable NetQueue in VMkernel using VI3 Client.
   
   
   1. Choose Configuration > Advanced Settings > VMkernel.
      
      
   2. Select the checkbox for VMkernel.Boot.netNetqueueEnabled.
      
      
2. Enable the ixgbe module in the service console of the ESX Server host:
   
   # esxcfg-module -e ixgbe
   
   
3. Set the required NetQueue options for the ixgbe module:
   
   
   • MSI-X enables the Ethernet controller to direct interrupt messages to multiple processor cores. This feature is required for NetQueue to function with VMDq. In the examples below, the parameter InterruptType with a value of 2 specifies MSI-X.
     
     
   • A value for VMDQ must exist to indicate the number of receive queues. A value of 16 for VMDQ sets the number of receive queues to the maximum. The Intel 82598 10 Gigabit Ethernet Controller provides 32 transmit queues and 64 receive queues per port, which can be mapped to a maximum of 16 processor cores. The range of values for the VMDQ parameter is 1 to 16.
   
   For a single port and the maximum number of receive queues, use the command:
   
   # esxcfg-module -s "InterruptType=2 VMDQ=16" ixgbe
   
   For two ports, add the values in a comma-separated list for each parameter as shown in the following example:
   
   # esxcfg-module -s "InterruptType=2,2 VMDQ=16,16" ixgbe
    
4. Reboot the ESX Server system.
   
   
   Note: If you are using jumbo frames, you also need to change the values for netPktHeapMinSize to 32 and netPktHeapMaxSize to 128. For details, see http://kb.vmware.com/kb/1004593.

1. Verify NetQueue has been enabled:
   
   # cat /etc/vmware/esx.conf
   
   Confirm the following line has been added into the file:
   
   /vmkernel/netNetqueueEnabled = "TRUE"
   
   
2. Verify the options configured for the ixgbe module:
   
   # esxcfg-module -g ixgbe
   
   The output should be similar to the following example:
   
   ixgbe enabled = 1 options = 'InterruptType=2,2 VMDQ=16,16'
   
   The enabled value should be equal to 1, which indicates the ixgbe module is enabled. InterruptType should be equal to 2 and include an entry for each port. (This is a 2-port example.) VMDQ should be 16 to enable the maxiumum number of receive queues and should also have an entry for each port.
   
   
3. Query which ports have loaded the driver using esxcfg-nics -l. Then query the statistics using ethtool. If VMDq is enabled, statistics for multiple receive queues will be shown (rx_queue_0 through rx_queue_15 in the example below).
   
   # esxcfg-nics -l
   
   Name PCI Driver Link Speed Duplex MTU Description
   vmnic0 04:00.00 bnx2 Up 1000Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
   vmnic1 08:00.00 bnx2 Down 0Mbps Half 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
   vmnic2 0b:00.00 ixgbe Up 10000MbpsFull 9000 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection
   vmnic3 0b:00.01 ixgbe Up 10000MbpsFull 9000 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection
   vmnic4 0d:00.00 ixgbe Up 10000MbpsFull 1500 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection
   vmnic5 0d:00.01 ixgbe Up 10000MbpsFull 1500 Intel Corporation 82598EB 10 Gigabit AF Dual Port Network Connection
   
   # ethtool -S vmnic2
   
   NIC statistics:
   rx_packets: 0
   tx_packets: 56
   rx_bytes: 0
   tx_bytes: 3584
   lsc_int: 1
   tx_busy: 0
   non_eop_descs: 0
   rx_errors: 0
   tx_errors: 0
   rx_dropped: 0
   tx_dropped: 0
   multicast: 0
   broadcast: 0
   rx_no_buffer_count: 0
   collisions: 0
   rx_over_errors: 0
   rx_crc_errors: 0
   rx_frame_errors: 0
   rx_fifo_errors: 0
   rx_missed_errors: 0
   tx_aborted_errors: 0
   tx_carrier_errors: 0
   tx_fifo_errors: 0
   tx_heartbeat_errors: 0
   tx_timeout_count: 0
   tx_restart_queue: 0
   rx_long_length_errors: 0
   rx_short_length_errors: 0
   tx_tcp4_seg_ctxt: 0
   tx_flow_control_xon: 0
   rx_flow_control_xon: 0
   tx_flow_control_xoff: 0
   rx_flow_control_xoff: 0
   rx_csum_offload_good: 0
   rx_csum_offload_errors: 0
   tx_csum_offload_ctxt: 0
   rx_header_split: 0
   low_latency_interrupt: 0
   alloc_rx_page_failed: 0
   alloc_rx_buff_failed: 0
   tx_queue_0_packets: 56
   tx_queue_0_bytes: 3360
   rx_queue_0_packets: 0
   rx_queue_0_bytes: 0
   rx_queue_1_packets: 0
   rx_queue_1_bytes: 0
   rx_queue_2_packets: 0
   rx_queue_2_bytes: 0
   rx_queue_3_packets: 0
   rx_queue_3_bytes: 0
   rx_queue_4_packets: 0
   rx_queue_4_bytes: 0
   rx_queue_5_packets: 0
   rx_queue_5_bytes: 0
   rx_queue_6_packets: 0
   rx_queue_6_bytes: 0
   rx_queue_7_packets: 0
   rx_queue_7_bytes: 0
   rx_queue_8_packets: 0
   rx_queue_8_bytes: 0
   rx_queue_9_packets: 0
   rx_queue_9_bytes: 0
   rx_queue_10_packets: 0
   rx_queue_10_bytes: 0
   rx_queue_11_packets: 0
   rx_queue_11_bytes: 0
   rx_queue_12_packets: 0
   rx_queue_12_bytes: 0
   rx_queue_13_packets: 0
   rx_queue_13_bytes: 0
   rx_queue_14_packets: 0
   rx_queue_14_bytes: 0
   rx_queue_15_packets: 0
   rx_queue_15_bytes: 0