Search the VMware Knowledge Base (KB)
View by Article ID

After replacing the VMware vCenter Server certificates in VMware vSphere 6.0, the VMware vSphere Auto Deploy solution user fails to log in (2123631)

  • 2 Ratings
Language Editions

Symptoms

After replacing certificates on VMware vCenter Server, you experience these symptoms:
  • In the /var/log/vmware/sca/sca.log or C:\ProgramData\VMware\vCenterServer\logs\sca\sca .log files for the vSphere Auto Deploy service (rbd ), you see entries similar to:
2015-07-01T05:58:17.523-04:00 [pool-5-thread-21 WARN com.vmware.sca.health.HealthStatusRequest] requestHealthStatusFromEndpoint: Failed to request health status (service:'rbd', url:https://vCenter_Server_FQDN:6502/vmw/rbd/healthStatus)
javax.net.ssl.SSLException: hostname in certificate didn't match: <vCenter.vmware.local> != <vpxd-extension>
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:231)
at org.apache.http.conn.ssl.StrictHostnameVerifier.verify(StrictHostnameVerifier.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:152)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:133)
at com.vmware.sca.util.RestClient$HostnameVerifier.verify(RestClient.java:81)
at org.apache.http.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:559)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:534)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:401)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at com.vmware.sca.util.RestClient.sendMessage(RestClient.java:160)
at com.vmware.sca.health.HealthStatusRequest.requestHealthStatusFromEndpoint(HealthStatusRequest.java:243)
at com.vmware.sca.health.HealthStatusRequest.getHealth(HealthStatusRequest.java:150)
at com.vmware.sca.servicecontrol.ServiceControlImplTemplate.getState(ServiceControlImplTemplate.java:318)
at com.vmware.sca.servicecontrol.ServiceControlImplTemplate.access$100(ServiceControlImplTemplate.java:50)
at com.vmware.sca.servicecontrol.ServiceControlImplTemplate$1.run(ServiceControlImplTemplate.java:196)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
  • In the C:\ProgramData\VMware\vCenterServer\logs\autodeploy\autodeploy.log or /var/log/vmware/rbd/rbd-vc-monitor.log file, you see entries similar to:

    2015-05-04T07:59:29.815 [37068]ERROR:rbd_watchdog_windows:caught exception in thread Feedback
    Traceback (most recent call last):
    File "rbd_watchdog_windows.pyc", line 50, in infiniteLoop
    File "rbd_watchdog_windows.pyc", line 64, in feedbackServer
    File "rbd\waiter\feedback.pyc", line 52, in __init__
    File "pyVmomi\VmomiSupport.pyc", line 543, in <lambda>
    File "pyVmomi\VmomiSupport.pyc", line 352, in _InvokeMethod
    File "pyVmomi\SoapAdapter.pyc", line 1270, in InvokeMethod
    vim.fault.InvalidLogin: (vim.fault.InvalidLogin) {
    dynamicType = <unset>,
    dynamicProperty = (vmodl.DynamicProperty) [],
    msg = "Can not make a connection because the username or password is incorrect.",
    faultCause = <unset>,
    faultMessage = (vmodl.LocalizableMessage) []
    }
    2015-05-04T07:59:31.487 [36744]ERROR:rbd_watchdog_windows:caught exception in thread VC-Monitor
    Traceback (most recent call last):
    File "rbd_watchdog_windows.pyc", line 50, in infiniteLoop
    File "rbd_watchdog_windows.pyc", line 58, in vcMonitor
    File "rbd\waiter\vc_monitor.pyc", line 48, in __init__
    File "pyVmomi\VmomiSupport.pyc", line 543, in <lambda>
    File "pyVmomi\VmomiSupport.pyc", line 352, in _InvokeMethod
    File "pyVmomi\SoapAdapter.pyc", line 1270, in InvokeMethod
    vim.fault.InvalidLogin: (vim.fault.InvalidLogin) {
    dynamicType = <unset>,
    dynamicProperty = (vmodl.DynamicProperty) [],
    msg = "Can not make a connection because the username or password is incorrect.",
    faultCause = <unset>,
    faultMessage = (vmodl.LocalizableMessage) []
    }
    2015-05-04T07:59:34.838 [37068]INFO:rbd_watchdog_windows:starting Feedback
    2015-05-04T07:59:34.838 [37068]INFO:vc_servers:client SSL material -- C:\ProgramData\VMware\vCenterServer\data\autodeploy\ssl\waiter.key, C:\ProgramData\VMware\vCenterServer\data\autodeploy\ssl\waiter.crt
    2015-05-04T07:59:36.733 [36744]INFO:rbd_watchdog_windows:starting VC-Monitor
    2015-05-04T07:59:36.733 [36744]INFO:vc_servers:client SSL material -- C:\ProgramData\VMware\vCenterServer\data\autodeploy\ssl\waiter.key, C:\ProgramData\VMware\vCenterServer\data\autodeploy\ssl\waiter.crt
    2015-05-04T07:59:37.862 [37068]ERROR:rbd_watchdog_windows:caught exception in thread Feedback
    Traceback (most recent call last):
    File "rbd_watchdog_windows.pyc", line 50, in infiniteLoop
    File "rbd_watchdog_windows.pyc", line 64, in feedbackServer
    File "rbd\waiter\feedback.pyc", line 52, in __init__
    File "pyVmomi\VmomiSupport.pyc", line 543, in <lambda>
    File "pyVmomi\VmomiSupport.pyc", line 352, in _InvokeMethod
    File "pyVmomi\SoapAdapter.pyc", line 1270, in InvokeMethod
    vim.fault.InvalidLogin: (vim.fault.InvalidLogin) {
    dynamicType = <unset>,
    dynamicProperty = (vmodl.DynamicProperty) [],
    msg = "Can not make a connection because the username or password is incorrect.",
    faultCause = <unset>,
    faultMessage = (vmodl.LocalizableMessage) []
    }
    2015-05-04T07:59:39.763 [36744]ERROR:rbd_watchdog_windows:caught exception in thread VC-Monitor
    Traceback (most recent call last):
    File "rbd_watchdog_windows.pyc", line 50, in infiniteLoop
    File "rbd_watchdog_windows.pyc", line 58, in vcMonitor
    File "rbd\waiter\vc_monitor.pyc", line 48, in __init__
    File "pyVmomi\VmomiSupport.pyc", line 543, in <lambda>
    File "pyVmomi\VmomiSupport.pyc", line 352, in _InvokeMethod
    File "pyVmomi\SoapAdapter.pyc", line 1270, in InvokeMethod
    vim.fault.InvalidLogin: (vim.fault.InvalidLogin) {
    dynamicType = <unset>,
    dynamicProperty = (vmodl.DynamicProperty) [],
    msg = "Can not make a connection because the username or password is incorrect.",
    faultCause = <unset>,
    faultMessage = (vmodl.LocalizableMessage) []
    }
  • In Health Messages under System Configuration > Auto Deploy > Summary > Auto Deploy Summary tab, you see these errors:

    • Failed to request health status from URI https://vCenter_Server_FQDN:6502/vmw/rbd/healthStatus.
    • AutoDeploy Service is not running. Enable AutoDeploy and refresh.

Cause

This issue occurs when the VMware vSphere Auto Deploy (rbd ) service is not aware of the new certificate after replacing the solution user certificates on VMware vCenter Server.

Resolution

This issue is resolved in VMware vCenter Server 6.0 U1b, available at VMware Downloads.

If you do not want to upgrade, you can work around this issue by updating the extension's certificate with vCenter Server.

To update the extension's certificate in vCenter Server for Windows:

  1. Connect to vCenter Server as an administrative user through a console or remote desktop session.
  2. Open an elevated command prompt.
  3. Run this command to retrieve the vpxd-extension  solution user certificate and key:

    "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.crt

    "%VMWARE_CIS_HOME%"\vmafdd\vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output c:\certificates\vpxd-extension.key


  4. Navigate to C:\Program Files\VMware\vCenter Server\vpxd\scripts:

    cd C:\Program Files\VMware\vCenter Server\vpxd\scripts

    Note: The path listed is for a default install of vCenter Server. If you have customized the install location of vCenter Server, change the directory accordingly.

  5. Run this command to update the extension's certificate with vCenter Server:

    "%VMWARE_PYTHON_BIN%" updateExtensionCertInVC.py -e com.vmware.rbd -c C:\Certificates\vpxd-extension.crt -k C:\Certificates\vpxd-extension.key -s vcenter_FQDN -u Administrator@vsphere.local

  6. When prompted, type the administrator@vsphere.local  password.

    Note: If you have customized the vCenter Single Sign-On domain, change the domain name accordingly.
To update the extension's certificate in the vCenter Server Appliance:
  1. Log in to the vCenter Server Appliance as root through SSH.
  2. Run this command to enable access the Bash shell:

    shell.set --enabled true

  3. Type shell  and press Enter.
  4. Run this command to retrieve the vpxd-extension  solution user certificate and key:

    mkdir /certificate

    /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt

    /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key


  5. Run this command to update the extension's certificate with vCenter Server.

    python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.rbd -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s vcsa_FQDN -u Administrator@vsphere.local

  6. When prompted, type the administrator@vsphere.local password.

    Note: If you have customized the vCenter Single Sign-On domain, change the domain name accordingly.

See Also

Update History

01/07/2016 - Added the details of the vCenter Server 6.0 U1b release, which resolves this issue.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 2 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 2 Ratings
Actions
KB: