Deploying VMware NSX for vSphere 6.x through Auto Deploy
search cancel

Deploying VMware NSX for vSphere 6.x through Auto Deploy

book

Article ID: 327400

calendar_today

Updated On:

Products

VMware NSX Networking

Issue/Introduction

This article provides steps to prepare ESXi hosts in vSphere clusters as part of Auto Deploy configurations in an NSX for vSphere 6.x

Environment

VMware NSX for vSphere 6.3.x
VMware NSX for vSphere 6.1.x
VMware NSX for vSphere 6.2.x
VMware NSX for vSphere 6.0.x
VMware NSX for vSphere 6.4.x

Resolution

Requirements:

Before proceeding, ensure to:
  • Auto deploy infrastructure
  • VXLAN infrastructure (NSX Manager and a DHCP server)
  • Windows host has PowerCLI installed
  • Copy of the ESXi offline depot. For example, VMware-ESXi-5.1.0-799733-depot.zip
  • Copy of an additional vxlan offline depot. For example, vxlan.zip.

    You can download this by connecting to the NSX Manager at:

    https://vsm-ip/bin/vdn/vibs/5.5/vxlan.zip

    Notes:
     
    • Prior to NSX 6.2.0, the vxlan.zip contains 3 VIBs, esx-vxlan, esx-vsip and esx-dvfilter-switch-security. In NSX 6.2.x and 6.3.x, the vxlan.zip contains only 2 VIBs, esx-vxlan and esx-vsip. The contents of the esx-dvfilter-switch-security and the new traceflow VIBs were folded into the esx-vxlan VIB.
    • Also prior to NSX 6.2.0, there was a single URL on NSX Manager from which VIBs for a certain version of the ESXi Host could be found. In other words, the administrator only needed to know a single URL, regardless of NSX version.
    • Starting in NSX 6.2.4, a third VIB, esx-vdpi, is provided along with the esx-vsip and esx-vxlan NSX VIBs.
    • In NSX 6.4.x, the vxlan.zip file contains a single VIB file "VMware_bootbank_esx-nsxv.vib".
      Example: For ESXi 6.5, the name of the VIB file is "VMware_bootbank_esx-nsxv_6.5.0-0.0.13168956.vib".
    • In NSX 6.2.x, 6.3.x, and 6.4.x, the NSX VIBs are available at different URLs. To find the correct VIBs, find the new VIB URL from https://NSX_Manager_IPaddress/bin/vdn/nwfabric.properties.The nwfabric.properties returns the correct download paths based on versions:

    For NSX for vSphere 6.2.x

    # 5.1 VDN EAM Info
    VDN_VIB_PATH.1=/bin/vdn/vibs-6.2.0/5.1-2107743/vxlan.zip
    VDN_VIB_VERSION.1=2107743
    VDN_HOST_PRODUCT_LINE.1=embeddedEsx
    VDN_HOST_VERSION.1=5.1.*

    # 5.5 VDN EAM Info
    VDN_VIB_PATH.2=/bin/vdn/vibs-6.2.0/5.5-2983935/vxlan.zip
    VDN_VIB_VERSION.2=2983935
    VDN_HOST_PRODUCT_LINE.2=embeddedEsx
    VDN_HOST_VERSION.2=5.5.*

    # 6.0 VDN EAM Info
    VDN_VIB_PATH.3=/bin/vdn/vibs-6.2.0/6.0-2984108/vxlan.zip
    VDN_VIB_VERSION.3=2984108
    VDN_HOST_PRODUCT_LINE.3=embeddedEsx
    VDN_HOST_VERSION.3=6.0.*

    For NSX for vSphere 6.3.x

    # 5.5 VDN EAM Info
    VDN_VIB_PATH.1=/bin/vdn/vibs-6.3.0/5.5-4744075/vxlan.zip
    VDN_VIB_VERSION.1=4744075
    VDN_HOST_PRODUCT_LINE.1=embeddedEsx
    VDN_HOST_VERSION.1=5.5.*

    # 6.0 VDN EAM Info
    VDN_VIB_PATH.2=/bin/vdn/vibs-6.3.0/6.0-4744062/vxlan.zip
    VDN_VIB_VERSION.2=4744062
    VDN_HOST_PRODUCT_LINE.2=embeddedEsx
    VDN_HOST_VERSION.2=6.0.*

    # 6.5 VDN EAM Info
    VDN_VIB_PATH.3=/bin/vdn/vibs-6.3.0/6.5-4744074/vxlan.zip
    VDN_VIB_VERSION.3=4744074
    VDN_HOST_PRODUCT_LINE.3=embeddedEsx
    VDN_HOST_VERSION.3=6.5.*


    For more information, see Troubleshooting vSphere ESX Agent Manager (EAM) with NSX (2122392).

Steps to prepare ESXi hosts for NSX as part of Auto Deploy

To include VXLAN as a part of Auto Deploy:
  1. Create a new image profile
  2. Update Auto Deploy rules
  3. Configure VXLAN
  4. Update the host profiles
Creating a new image profile

To create a new image profile:
  1. Start PowerCLI and connect to vCenter Server by running this command:

    Connect-VIServer x.x.x.x –User user -Password password
     
  2. Import offline depots by running these commands:

    C:\Add-EsxSoftwareDepot C:\path\VMware-ESXi-5.1.0-799733-depot.zip
    C:\Add-EsxSoftwareDepot C:\path\vxlan.zip


    Note: Repeat these commands for all depots
     
  3. Run this command to list the default Image Profiles and note the name of profile to be cloned, such as ESXi-5.1.0-799733-no-tools:

    C:\Get-EsxImageProfile
     
  4. Run this command to create new Image Profile by cloning an existing image profile, such as ESXi-5.1.0-799733-no-tools:

    C:\New-EsxImageProfile -Name NewImageProfile -CloneProfile ESXi-5.1.0-799733-no-tools VMware –AcceptanceLevel PartnerSupported
     
  5. For NSX 6.3.x, run this command to add the two required NSX for vSphere VIBS to the new Image Profile. Ensure that the Image Profile includes both VIBs, or you may experience unexpected results.

    C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vxlan
    C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vsip


    Note: Starting in NSX for vSphere 6.2.4, a third VIB, esx-vdpi, is provided along with the esx-vsip and esx-vxlan NSX VIBs. Also note that this does not apply to NSX for vSphere 6.3.x.

    C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vdpi
     
  6. Run this command to list all VIBs in the new Image Profile:

    C:\Get-EsxImageProfile NewImageProfile | Select -ExpandProperty VibList | sort
     
  7. (Optional) Run this command to save the new Image Profile as an offline depot:

    C:\ESXi-Depots> Export-EsxImageProfile -ImageProfile NewImageProfile -ExportToBundle -FilePath C:\path\FileName.zip
     

Updating Auto Deploy rules

To update the Auto Deploy rules:

  1. Run this command to list the current rules:

    C:\> Get-DeployRuleSet
     
  2. Run this command to remove the old rule:

    C:\> Remove-DeployRule old_rule

    Note: This command removes the rule, but does not delete it.
     
  3. Create a new rule for the new Image Profile.

    For example, run this command to create the rule CustomImageRule using the NewImageProfile created earlier and to assign it to all hosts that boot using Auto Deploy.

    C:\> New-DeployRule CustomImageRule -Item NewImageProfile –AllHosts
     
  4. Run this command to activate the new rule:

    C:\> Add-DeployRule CustomImageRule
     
  5. Run these commands to update the existing hosts to use the new rule:

    C:\> foreach ($esx in Get-VMHost) {$esx | test-deployrulesetcompliance | Repair-DeployRuleSetCompliance}
    C:\> foreach ($esx in Get-VMHost) {$esx | test-deployrulesetcompliance}


    Note: The changes take affect after the next reboot.

Configuring VXLAN
Note:
This step is not required if there is no need for VXLAN overlay services in the environment.

If this is a new NSX cluster, open NSX Manager and install the Network Virtualization Components on all hosts in the NSX cluster, see the Install Network Virtualization Components section in the NSX Installation and Upgrade Guide.

To prepare the hosts for VXLAN, configure these parameters in the preparation workflow:

  • Choose the cluster and the vSphere Distributed Switch (vDS)
  • Assign the VLAN for VXLAN transport traffic
  • Select the Teaming policy and MTU

Updating the host profiles
Note:
This step is always required, even if the hosts will not be prepared for VXLAN.

To avoid unexpected results, ensure that the initial host profile used before NSX preparation is created after the ESXi host has booted with an image profile containing the NSX VIBs. This order of operation ensures that the host profile contains the VXLAN netstack and is not removed when the profile is applied.
 
Note: vSphere releases which support VXLAN use separate TCP/IP stack or netstack instances to provide isolation and abstraction. The esxcli network ip netstack get command retrieves the netstacks - defaultTCPipStack and VXLAN.
 
After the hosts are prepared for NSX and VXLAN is configured, update the host profile from the reference host and remove certain host-specific parameters.
 
To update the host profile from the reference host and remove certain host-specific parameters:
  1. Right-click the host profile and click Enable/Disable Profile Configuration.
  2. Under Networking configuration > Host Virtual NIC, deselect the VXLAN vmkernel interface to ensure that NSX Manager always creates the VTEP on boot.

    Note: if your reference profile contains multiple VTEPs, ensure that all such VTEPs are removed.
     
  3. Under Networking configuration > NetStack Instance > VXLAN, deselect the ipRouteConfig option. (If vSphere 6.5 version)
    Under Networking configuration > NetStack Instance > VXLAN, ensure the ipRouteConfig option is selected. (if vSphere 6.7 version)
     
  4. Under Advanced configuration option, deselect all UserVars.Rmq* variables. These variables are host-specific.

    # grep Rmq /etc/vmware/esx.conf
    /advUserOptions/options[0030]/name = "RmqClientPeerName"
    /advUserOptions/options[0031]/name = "RmqHostId"
    /advUserOptions/options[0036]/name = "RmqClientResponseQueue"
    /advUserOptions/options[0037]/name = "RmqClientExchange"
With these changes you can refresh the host profile as required, and the NSX parameters are ignored when the host profile is applied.
 
Note: When setting the hostname, apply the default value to all netstack instances except for the default TCP/IP stack. To do so, VMware recommends Networking configuration > NetStack Instance > VXLAN > DNS Configuration > Hostname, select Obtain hostname from DCHP.
Note: When configuring a host for VXLAN in the NSX Manager UI, ensure that the configured LACP setting matches the setting on the virtual switch. If the settings differ, in the NSX Manager logs, you see entries similar to:
 
<YYYY-MM-DD>T<time>.650 GMT WARN http-443-exec-22 DefaultExceptionLogger:35 - The following exception occurred during request processing by the BlazeDS MessageBroker and will be serialized back to the client: </time>
flex.messaging.MessageException: Invalid LACP version selected. Distributed Virtual Switch vDS01 is configured with different LACP version.
 
Unpreparing a Host for VXLAN
 
To unprepare a host for VXLAN, you must remove the VXLAN network configuration parameters (IP route config and VTEP interfaces) as well as remove the three NSX VIBs from the image profile and create a new image profile. If the VIBs are not removed from the image profile, the host will show as Not Ready in the Host Preparation tab after reboot. Clicking on the red error next to the Not Ready link displays the reason for the error.
 


Additional Information

Troubleshooting vSphere ESX Agent Manager (EAM) with NSX
Troubleshooting VMware NSX for vSphere 6.x
通过 Auto Deploy 部署 VMware NSX for vSphere 6.x
Auto Deploy による VMware NSX for vSphere 6.x のデプロイ