NSX preparation of a cluster fails but the VIBs install on the ESXi hosts
search cancel

NSX preparation of a cluster fails but the VIBs install on the ESXi hosts

book

Article ID: 341890

calendar_today

Updated On:

Products

VMware NSX Networking VMware vCenter Server

Issue/Introduction

The purpose of this article is to provide a workaround to complete the initial preparation of NSX inside a cluster.

Symptoms:

During initial installation of NSX 6.4.1 in a vCenter 6.7 environment, NSX may fail to fully install.  The VIBs install successfully on the ESXi hosts, but the NSX Version shows "Not Ready", VXLAN shows "Not Configured" with no option to install.  ESXi hosts show green check marks under "NSX Installation," "Firewall," and "Communication Channels."


 

Verification of Symptom:

  1. Using vCenter Web Client, navigate to Home -> Administrator -> vCenter Server Extensions -> vSphere ESX Agent Manager -> Configure -> ESX Agencies
  2. Highlight the Agency with a status of "Alert."   Under the Issue column the message reads: "The software defined by an Agency cannot be staged in VUM."
  3. Navigate to the EAM Mob by opening a web browser and navigating to: https://vcenterserver/eam/mob
  4. In the EAM/Mob navigate to: Agency ID -> Runtime -> issue[x]
  5. Inside the section marked "Description" you see text similar to:

"The software defined by an Agency cannot be staged in VUM. Cause : Could not upload the following VIBs in Integrity server: [https://192.168.110.30/bin/vdn/vibs-6.4.1/6.0-8590012/vxlan.zip, https://192.168.110.30/bin/vdn/vibs-6.4.1/6.7-8590012/vxlan.zip, https://192.168.110.30/bin/vdn/vibs-6.4.1/6.5-8590012/vxlan.zip]"

 

 

 


Environment

VMware vCenter Server Appliance 6.7.x
VMware NSX for vSphere 6.4.x

Cause

This is caused by EAM (ESX Agent Manager) being unable to push the NSX VIBs into Update Manager. In many cases, this is because VUM is configured to use a proxy Internet access.

Verify a proxy is in use by navigating to:
  1. Home -> Hosts and Clusters -> Select a Cluster -> Update Manager  -> Go to Admin View
  2. Under "Download Settings" view the "Proxy settings."  If a URL or IP address is configured in the Proxy field, a proxy is in use.


This error "The software defined by an Agency cannot be staged in VUM. Cause : Could not upload the following VIBs in Integrity server:" may also occur if the customer has a Windows vCenter 6.7 with Update Manager installed onto the vCenter.
If such a setup exists, see "Resolution for Windows VC + VUM" in the Resolution section.

Resolution

Resolution for Proxy Configuration
  1. Remove the proxy configuration.
    1. Click Edit in "Download Settings" section of Update Manager.
    2. Remove the check from "Use Proxy" in the Edit Proxy Settings dialog box
​​​​
  1. Navigate to Home -> Administration -> vCenter Server Extensions -> Configure -> ESX Agencies
  2. Highlight the Agency with the Alert.
  3. Right Click the Agency
  4. Click "Resolve All Issues."
​​​
 
Resolution for Windows VC + VUM
 
  1. Verify vCenter is a Windows machine
  2. Verify Update Manager is installed, and on the vCenter itself
    1. Hosts and Clusters -> Select the vCenter from the Navigator -> Manage -> Network Connectivity -> Update Manager settings should include "IP address or host name for the patch store" with a value of the vCenter
  3. Log into the vCenter to check the VUM logs
    1. cd /var/log/vmware/vmware-updatemgr/vum-server/
    2. less vmware-vum-server-log4cpp.log
    3. Verify these error messages in VUM

vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:804 'httpDownload' 2024 INFO]  [httpDownload, 577] Downloading https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02 via proxy 10.60.33.96:8080
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:832 'httpDownload' 2024 ERROR]  [httpDownload, 738]
Error 12175 from WinHttpSendRequest for url https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:832 'httpDownload' 2024 INFO]  [httpDownload, 920] Retrying https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02 with proxy off
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:832 'httpDownload' 2024 INFO]  [httpDownload, 562] Downloading https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:836 'httpDownload' 2024 ERROR]  [httpDownload, 738]
Error 12175 from WinHttpSendRequest for url https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:836 'httpDownload' 2024 WARN]  [httpDownload, 423] Download https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02 failed: Error 12175 from WinHttpSendRequest for url https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:836 'httpDownload' 2024 ERROR]  [httpDownload, 438] Reached retry download limit
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:836 'DownloadMgr' 2024 ERROR]  [downloadMgr, 629] Executing download job {807237733952} throws error: Reached retry download limit
vmware-vum-server-log4cpp.log.01:[2019-01-14 13:22:17:836 'DownloadMgr' 2024 ERROR]  [downloadMgr, 718] Download failed for url: https://VWH01MGM143.csssgatineau.reg07.rtss.qc.ca:443/eam/vib?id=9db70df5-15f1-4e90-b7b5-9ffe3f50ed02


4. If these error messages are present, then the issue is due to Update Manager being unable to communicate correctly to the Windows vCenter. This is a Windows error, and the fix is a Windows registry change:
https://kb.vmware.com/s/article/2009000



Additional Information

Impact/Risks:
The NSX cluster will not prepare fully.  VXLAN cannot be configured.