Changing RDMA NIC's RoCE Version in ISER Environments
search cancel

Changing RDMA NIC's RoCE Version in ISER Environments

book

Article ID: 319612

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:
  • Connection loss from ESXi to target array and users might see logs similar to:

    YYYY-MM-DDTHH:MM:SS.946Z cpu47:1001391035)iser: iser_set_roce: Backing RNIC for vmhba69 supports RoCE v2 only
    YYYY-MM-DDTHH:MM:SS.947Z cpu51:1001391107)Jumpstart plugin software-iscsi activation failed: software-iscsi->start() failed: error while executing the cli.


Environment

VMware vSphere 7.0.x
VMware vSphere ESXi 7.0.0

Cause

  • When user sets the RoCE version that iSER is supposed to connect with the target, that data is stored internally on the server. If the user, changes the version on the RoCE driver and not make a corresponding change in iSER using iSER's config manager command, it will lead to a mismatch between running and stored configuration.
  • When the host is rebooted, the configuration mismatch is detected by the system which leads to failure of iSER configuration restore. This will lead to connection failures between ESXi and iSER target array.
Note: User is not supposed to change the RoCE version of the RDMA (NIC) driver, before setting the RoCE parameter of iSER adapter to the value the RDMA (NIC) driver RoCE version is to be changed to.

Resolution

  • If RDMA (NIC) driver is configured to run in v1 mode, set iSER vmhba parameters for the same
esxcli rdma iser params set -a vmhba69 -r 1
  • If the user decides to change the RDMA (NIC) driver's personality to run in v2 mode, the iSER adapter's ROCE parameter must be set to the new version before it's set in the RDMA (NIC) driver.
esxcli rdma iser params set -a vmhba69 -r 2

The same applies if changing the RDMA (NIC) driver's RoCE version from v2 to v1 and the above command(s) have to executed for all iSER adapters on the host.

Powered by Wolken Software