Optimizing iSCSI software initiator performance and NIC teaming

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information related to optimizing iSCSI software initiator performance with teamed NICs on ESX/ESXi 3.x hosts.

Environment

VMware ESX Server 3.5.x
VMware ESX Server 3.0.x
VMware ESXi 3.5.x Embedded
VMware ESXi 3.5.x Installable

Resolution

Note: This procedure using NIC teaming is applicable only to ESX/ESXi 3.x hosts, with ESX/ESXi 4.x and 5.x it is strongly recommended to use port binding instead of NIC teaming. For more information refer to the iSCSI SAN Configuration guide at the VMware Documentation site

There are several factors involved in optimizing iSCSI performance when using the software iSCSI initiator.

The only area where iSCSI performance can be easily optimized on an ESX host is in the configuration of the network.

Separate the network traffic:

On an ESX server where you have virtual machine traffic, VMotion traffic, and iSCSI traffic, consider a configuration like:

vSwitch1 – for virtual machines
vSwitch2 – for VMotion
vSwitch3 – for iSCSI

Segregating and isolating the iSCSI network traffic by VLAN configuration on virtual switch or physical switch is required to ensure iSCSI data integrity.

Note: There may be several variations of this principle, but the principle is to isolate the iSCSI traffic from other network traffic in the network.

Note: If you have multiple iSCSI targets you can further improve the iSCSI throughput but putting two NICs on the vSwitch you have set up for iSCSI. This only works with multiple targets. VMware suggests no more than two NICs in a team for iSCSI. It may be possible to have a single NIC for each iSCSI target.

For any given target (iSCSI target IP), you establish one link through a NIC in the team.An additional session to a separate target (different iSCSI target IP) can establish a connection through the second NIC in the team.This is not guaranteed because it is dependent on a number of variables we cannot control.All LUNs presented on one IP address passes data through the connection established during the iSCSI session login.

The combination of IP hash on the server and the Source Address/Destination Address management (or similar) on the switch enables traffic to a given target to go over one NIC and the traffic to the second IP address to go over the other NIC.To take advantage of the teaming and the IP hash, you must have a one to many or many to many IP address/traffic pattern.If a link detect drops on the NIC in use for a given session, it fails over to the other NIC in the team.All NICs using IP hash, have to be connected to one physical switch.

Additional Information

For translated versions of this article, see: