Operations against an existing NSX-T workload domain fails after upgrading VMware Cloud Foundation to version 3.5.1
Article ID: 316932
Updated On:
Products
VMware Cloud Foundation
Issue/Introduction
Symptoms:
- Operations against an existing NSX-T workload domain fails on the "Gather input to add host to NSX-T Fabric and migrate networking from dvs to nvds" task. These operations can include creating a new cluster, expanding a cluster or removing a host from a cluster.
- VMware Cloud Foundation was upgraded from version 3.5 to version 3.5.1 after the NSX-T workload domain was created.
- The default NSX-T certificate has not been replaced.
- You see messages similar to the following in the /var/log/vmware/vcf/domainmanager/domainmanager.log file on the SDDC Manager VM.
2018-12-19 12:10:56.492 [vcf_dm,7ef3e7f82436c1bc,878d3675a0613279] [-thread-15] ERROR [ c.v.v.c.f.p.nsxt.action.NsxtAddClusterHeader] Error occurred while generating input for add hosts in cluster in nsxt environment
java.lang.RuntimeException: Cannot execute request
at com.vmware.vapi.internal.protocol.client.rest.DefaultRequestExecutorFactory$DefaultHttpResponseHandler.onError(DefaultRequestExecutorFactory.java:94)
Caused by: java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2019-03-14T20:36:59.867Z 127.0.0.1 - "GET /ui/api/domainmanager/nsxt/controlcluster HTTP/1.0" 500 2 - 1485.791 ms - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
2019-03-14T21:09:09.563Z 127.0.0.1 - "GET /ui/api/domainmanager/nsxt/controlcluster HTTP/1.0" 500 2 - 505.748 ms - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
2019-03-14 20:36:58.358 [vcf_dm,58af6bd16418dac4,58af6bd16418dac4] [200-exec-8] INFO [ c.v.vcf.domainmanager.rest.DomainManagerAbout] Getting domainmanager service info
2019-03-14 20:36:58.972 [vcf_dm,f1a52c3513e5d811,f1a52c3513e5d811] [200-exec-9] DEBUG [ c.v.e.s.c.s.a.c.CredentialServiceAdapterImpl] Getting API credential for entity 5c3b893d-1014-4332-b3b9-0220d0c4ef82 of entity type NSXT_MANAGER
2019-03-14 20:36:59.856 [vcf_dm,f1a52c3513e5d811,f1a52c3513e5d811] [200-exec-9] ERROR [ c.v.v.v.controller.ViManagerNsxtController] Received an exception while getting NsxtControlCluster for domainId: null
2019-03-14 21:09:08.961 [vcf_dm,d4413bd718eb2bb0,d4413bd718eb2bb0] [200-exec-3] INFO [ c.v.vcf.domainmanager.rest.DomainManagerAbout] Getting domainmanager service info
2019-03-14 21:09:09.393 [vcf_dm,23e582533587b6f2,23e582533587b6f2] [200-exec-1] DEBUG [ c.v.e.s.c.s.a.c.CredentialServiceAdapterImpl] Getting API credential for entity 5c3b893d-1014-4332-b3b9-0220d0c4ef82 of entity type NSXT_MANAGER
2019-03-14 21:09:09.558 [vcf_dm,23e582533587b6f2,23e582533587b6f2] [200-exec-1] ERROR [ c.v.v.v.controller.ViManagerNsxtController] Received an exception while getting NsxtControlCluster for domainId: null
2019-03-14 22:54:24.976 [vcf_om,633f3689af5c123a,633f3689af5c123a] [om-exec-13] ERROR [ c.v.v.r.s.u.vsphere.cache.ClusterMetricsCache] Unable to retrieve vLAN Id from NSX-T manager
java.lang.RuntimeException: Cannot execute request
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
java.lang.RuntimeException: Cannot execute request
at com.vmware.vapi.internal.protocol.client.rest.DefaultRequestExecutorFactory$DefaultHttpResponseHandler.onError(DefaultRequestExecutorFactory.java:94)
Caused by: java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2019-03-14T20:36:59.867Z 127.0.0.1 - "GET /ui/api/domainmanager/nsxt/controlcluster HTTP/1.0" 500 2 - 1485.791 ms - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
2019-03-14T21:09:09.563Z 127.0.0.1 - "GET /ui/api/domainmanager/nsxt/controlcluster HTTP/1.0" 500 2 - 505.748 ms - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
2019-03-14 20:36:58.358 [vcf_dm,58af6bd16418dac4,58af6bd16418dac4] [200-exec-8] INFO [ c.v.vcf.domainmanager.rest.DomainManagerAbout] Getting domainmanager service info
2019-03-14 20:36:58.972 [vcf_dm,f1a52c3513e5d811,f1a52c3513e5d811] [200-exec-9] DEBUG [ c.v.e.s.c.s.a.c.CredentialServiceAdapterImpl] Getting API credential for entity 5c3b893d-1014-4332-b3b9-0220d0c4ef82 of entity type NSXT_MANAGER
2019-03-14 20:36:59.856 [vcf_dm,f1a52c3513e5d811,f1a52c3513e5d811] [200-exec-9] ERROR [ c.v.v.v.controller.ViManagerNsxtController] Received an exception while getting NsxtControlCluster for domainId: null
2019-03-14 21:09:08.961 [vcf_dm,d4413bd718eb2bb0,d4413bd718eb2bb0] [200-exec-3] INFO [ c.v.vcf.domainmanager.rest.DomainManagerAbout] Getting domainmanager service info
2019-03-14 21:09:09.393 [vcf_dm,23e582533587b6f2,23e582533587b6f2] [200-exec-1] DEBUG [ c.v.e.s.c.s.a.c.CredentialServiceAdapterImpl] Getting API credential for entity 5c3b893d-1014-4332-b3b9-0220d0c4ef82 of entity type NSXT_MANAGER
2019-03-14 21:09:09.558 [vcf_dm,23e582533587b6f2,23e582533587b6f2] [200-exec-1] ERROR [ c.v.v.v.controller.ViManagerNsxtController] Received an exception while getting NsxtControlCluster for domainId: null
2019-03-14 22:54:24.976 [vcf_om,633f3689af5c123a,633f3689af5c123a] [om-exec-13] ERROR [ c.v.v.r.s.u.vsphere.cache.ClusterMetricsCache] Unable to retrieve vLAN Id from NSX-T manager
java.lang.RuntimeException: Cannot execute request
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware Cloud Foundation 3.5.x
Resolution
This is a know issue affecting VMware Cloud Foundation 3.5.1. There is currently no resolution.
Workaround:
Use the following steps to workaround this issue:
Note: If you have already replaced the default NSX-T certificate with a CA certificate, do not complete these steps as they will overwrite the CA certificate.
Workaround:
Use the following steps to workaround this issue:
Note: If you have already replaced the default NSX-T certificate with a CA certificate, do not complete these steps as they will overwrite the CA certificate.
- Download the attached 67030_cert-gen.zip file.
- Use a file transfer utility to copy the 67030_cert-gen.zip file to the /home/vcf folder on the SDDC Manager VM.
- ssh to the SDDC Manager VM as the vcf user.
- Issue the following command to extract the contents of the /home/vcf/67030_cert-gen.zip file:
unzip 67030_cert-gen.zip
Note: You will see output similar to the following:
Archive: /tmp/67030_cert-gen.zip
inflating: /tmp/ec-ec.1-upgrade.py
inflating: /tmp/getcertificate.py
inflating: /tmp/setup_logger.py
inflating: /tmp/certificate_util.py
Note: You will see output similar to the following:
Archive: /tmp/67030_cert-gen.zip
inflating: /tmp/ec-ec.1-upgrade.py
inflating: /tmp/getcertificate.py
inflating: /tmp/setup_logger.py
inflating: /tmp/certificate_util.py
- Issue the following command to execute the ec-ec.1-upgrade.py script:
python ec-ec.1-upgrade.py
Note: You will see output similar to the following:
***Starts Execution***
***Ends Execution***
Note: You will see output similar to the following:
***Starts Execution***
***Ends Execution***
- In the SDDC Manager UI, restart the failed task.
Additional Information
To be alerted when this article is updated, click the Subscribe to Article link in the Actions box.
Attachments
Feedback
Yes
No
Powered by
