"Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk" error in the vSphere Web Client
search cancel

"Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk" error in the vSphere Web Client

book

Article ID: 316404

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • The VMware vSphere Web Client displays the error:
     
    • Failed to verify the SSL certificate for one or more vCenter Server Systems: https://vCenterServerFQDN:443/sdk
    • could not connect to one or more vCenter Server Systems:https://vCenterFQDN:443/sdk
       
  • Objects such as host or virtual machines are not displayed in the vSphere Web Client
Failed to verify the SSL certificate for one or more vCenter Server Systems: https://vCenterServerFQDN:443/sdk


Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.0.x
VMware vCenter Server 5.5.x
VMware vCenter Server 6.7.x
VMware vCenter Server 7.0.x
VMware vCenter Server 6.0.x
VMware vCenter Server Appliance 6.7.x

Cause

This issue occurs in these situations when using Enhanced Linked Mode:
  • Another vCenter Server in the Single Sign-On (SSO) domain has restarted or is not fully available following a restart.
  • During the re-installation of vCenter Server, it is possible to have the same vCenter Server registered more than once to Single Sign-On (SSO).
  • With a previous install of vCenter Server, SSL certificates are not overwritten or removed properly during an upgrade or re-installation.

Resolution

Note: This issue may be transient as another vCenter Server in an Enhanced Link Mode domain is restarting. Before continuing with troubleshooting, it is advised to wait 10 minutes, log out and log back in to the vCenter Server. The error may clear on its own. In addition, it is strongly advised to determine if logging into the other vCenter Server identified in the error message directly to determine if vCenter services are up and running before continuing. If all vCenter Servers are up and running and this error persists, continue with this resolution to identify duplicate service registrations or other errors.

This resolution is in multiple sections. Begin with your applicable configuration.

    vSphere 6.x

    Find a duplicate registered vCenter Server instance

    To find a duplicate registered vCenter Server instance follow the steps below:

    Note: Reviewing vsphere-client / vsphere-ui logs are necessary to identify the cause of this error if there are no duplicate registrations and you still observe the same error. In such situation, you may file a Support Request with VMware to fix this issue.

    For Windows:
    1. Log in to the server with the Platform Services Controller installed.
    2. Open a Windows Command Prompt as administrator.
    3. To create a text file with a list of the services registered within the Platform Services Controller, run this command:

      "%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" list --url http://localhost:7080/lookupservice/sdk --type vcenterserver > c:\psc_services.txt
       
    4. Open the generated text file to find a list of services registered to the Platform Services Controller.

      In the text file, you see output similar to:
      Name: AboutInfo.vpx.name
      Description: AboutInfo.vpx.name
      Service Product: com.vmware.cis
      Service Type: vcenterserver
      Service ID: 608AF497-B198-40D1-9855-545533A488AF
      Site ID: home-office
      Node ID: 86ca3bf1-9201-11e3-8f19-000c29562ae2
      Owner ID: [email protected]
      Version: 6.0
      Endpoints:
      Type: com.vmware.cis.workflow
      Protocol: vmomi
      URL: http://vCenter1.domain.local:8088
      SSL trust:


      Name: AboutInfo.vpx.name
      Description: AboutInfo.vpx.name
      Service Product: com.vmware.cis
      Service Type: vcenterserver
      Service ID: 6ae3bf1a-9318-4a33-b2cb-d2eaa7a306c5
      Site ID: home-office
      Node ID: 86ca3bf1-9201-11e3-8f19-000c29562ae2
      Owner ID: [email protected]
      Version: 6.0
      Endpoints:
      Type: com.vmware.cis.workflow
      Protocol: vmomi
      URL: http://vCenter2.domain.local:8088
      SSL trust:

      Name: vCenterService
      Description: vCenter Server
      Service Product: com.vmware.cis
      Service Type: vcenterserver
      Service ID: default-first-site:01c98f18-770a-41c2-a967-b7a4b574cad2
      Site ID: default-first-site
      Owner ID: [email protected]
      Version: 5.5
      Endpoints:
      Type: com.vmware.vim
      Protocol: vmomi
      URL: https://Legacy_vCenter.domain.local:443/sdk

    Unregister a duplicate vCenter Server service:

    1. To unregister the duplicate service endpoint, run this command:
      "%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" unregister --url http://localhost:7080/lookupservice/sdk --id Service_ID from Step 4 --user "[email protected]" --password "administrator_password" --no-check-cert

      Use this as a model:
      "%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" unregister --url http://localhost:7080/lookupservice/sdk --id 608AF497-B198-40D1-9855-545533A488AF --user "[email protected]" --password "VMware123!" --no-check-cert
     

    vCenter Server Appliance


    Connect to the Platform Services Controller using SSH.
    1. Run this command to enable access the Bash shell:
      shell.set --enabled true
       
    2. Type shell and press Enter
    3. To create a text file with a list of the services registered within the Platform Services Controller, run this command:
      /usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost:7080/lookupservice/sdk --type vcenterserver > /tmp/psc_services.txt
      For vCenter 7.0, alter the command as follows:
      
      /usr/lib/vmware-lookupsvc/tools/lstool.py list --url http://localhost:7090/lookupservice/sdk --type vcenterserver > /tmp/psc_services.txt --no-check-cert
    4. Open the generated text file to find a list of services registered to the Platform Services Controller.

      In the text file, you see output similar to:
      Name: AboutInfo.vpx.name
      Description: AboutInfo.vpx.name
      Service Product: com.vmware.cis
      Service Type: vcenterserver
      Service ID: 1dbc3e9f-626d-4314-8731-ca744a0d9f4b
      Site ID: home
      Node ID: d3eba55a-d4df-11e4-b3f7-000c2987c143
      Owner ID: [email protected]
      Version: 6.0
      Endpoints:
      Type: com.vmware.cis.workflow
      Protocol: vmomi
      URL: http://vcsa2.domain.local:8088

      Name: AboutInfo.vpx.name
      Description: AboutInfo.vpx.name
      Service Product: com.vmware.cis
      Service Type: vcenterserver
      Service ID: 6ae3bf1a-9318-4a33-b2cb-d2eaa7a306c5
      Site ID: home
      Node ID: 44b05c52-d4d3-11e4-830b-000c29a0e10e
      Owner ID: [email protected]
      Version: 6.0
      Endpoints:

      Type: com.vmware.cis.workflow
      Protocol: vmomi
      URL: http://vcsa1.domain.local:8088


      Note: To identify a valid registration against stale registration /etc/vmware/install-defaults/vmdir.ldu-guid can be looked to compare Node ID from above out output
    • To unregister the duplicate service endpoint, run this command:
      /usr/lib/vmidentity/tools/scripts/lstool.py unregister --url http://localhost:7080/lookupservice/sdk --id Service_ID from Step 4 --user '[email protected]' --password 'administrator_password' --no-check-cert
    • If you are using vCenter 7.0, alter the command as follows:
    /usr/lib/vmware-lookupsvc/tools/lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id Service_ID from Step 4 --user '[email protected]' --password 'administrator_password' --no-check-cert

    NOTE: To find out which node is the right node mapped with the PSC, we need to view the vmdir.ldu-guid file Steps:
    1. Connect to PSC via SSH 
    2. cd /etc/vmware/install-defaults/ 
    3. cat /etc/vmware/install-defaults/vmdir.ldu-guid


    The output gives the ORIGINAL NODE id now we can remove the stale entry registered for other node id's

    Use this as a model:
    /usr/lib/vmidentity/tools/scripts/lstool.py unregister --url http://localhost:7080/lookupservice/sdk --id 6ae3bf1a-9318-4a33-b2cb-d2eaa7a306c5 --user '[email protected]' --password 'VMware123!' --no-check-cert   

    Or if you are using vCenter 7.0:

    /usr/lib/vmware-lookupsvc/tools/lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id 6ae3bf1a-9318-4a33-b2cb-d2eaa7a306c5 --user '[email protected]' --password 'VMware123!' --no-check-cert


    Note: Same way if you see webclient and log browser endpoint then ensure to delete the same as well.
    The fastest way to identify the bad node in this situation is to look at the last 12 characters of the Node ID, which will be the MAC address of the owning node. Find which node matched.

     

    vSphere 5.x:

    Find a duplicate registered vCenter Server instance:

    To find a duplicate registered vCenter Server instance follow the steps below:
    Note: For a non-linked vCenter Server configuration, ensure there is only one vCenter Server registered with SSO. If a duplicate vCenter Server service is found, unregister the duplicate vCenter Server service by checking the time and date of the ownerId and unregistering the older service.

    You can also identify the current vCenter Server instance by reviewing the vpxd.cfg file located at C:\ProgramData\VMware\VMware VirtualCenter. The current vCenter Server instance ID and name is displayed similar to:

    <lookupService>
    <serviceId>{9300C2AC-4D97-4191-8EB1-387D9823E6E3}:23</serviceId>
    </lookupService>
    <solutionUser>
    <name>vCenterServer_2013.02.28_170324</name>
    </solutionUser>


    To unregister a duplicate vCenter Server service, use the full Service ID found in the sso_services.txt output and unregister the service using the ssolscli unregisterService command.
    1. Log in to the server with vCenter Single Sign-On installed.
    2. Open a Windows Command Prompt as administrator.
    3. Navigate to this directory depending on your vSphere version:

      vCenter Server 5.5 – C:\Program Files\VMware\Infrastructure\VMware\CIS\vmware-sso

    4. Set the JAVA_HOME variable (Windows)
    vCenter Server 5.5 – SET JAVA_HOME=C:\Program Files\Common Files\VMware\VMware vCenter Server - Java Components
    1. To create a text file with a list of the services registered with SSO, run this command:

      ssolscli.cmd listServices https://vCenter_Single_Sign-on_FQDN:7444/lookupservice/sdk > c:\sso_services.txt
    2. Open the generated text file to find a list of services registered to vCenter Single Sign-On.
                 In the text file, you see output similar to:
     
    vSphere 5.5

    Service 1
    -----------
    serviceId=Site Name:02dde295-422a-403e-b32c-1e40c3f188fd
    serviceName=vCenterService
    type=urn:vc
    endpoints={[url=https://FQDN:443/sdk,protocol=vmomi]}
    version=5.1
    description=vCenter Server
    ownerId=vCenterServer_2013.10.10_163108@System-Domain
    productId=
    viSite=Site Name

    Service 2
    -----------
    serviceId=Site Name:811660f9-f110-4ee7-8f9e-dc0dd1d062fe
    serviceName=vCenterService
    type=urn:vc
    endpoints={[url=https://FQDN:443/sdk,protocol=vmomi]}
    version=5.1
    description=vCenter Server
    ownerId=vCenterServer_2013.10.10_163123@System-Domain
    productId=
    viSite=Site Name
     
    1. Create a file called c:\serviceID.txt, which contains only the entire serviceID of the duplicate vCenter Server. For example, to create the file:

      vSphere 5.5:

      Site Name:02dde295-422a-403e-b32c-1e40c3f188fd
       

    Unregister the service by running this command:

    vSphere 5.5:

    ssolscli unregisterService -d https://vCenter_Single_Sign-On_FQDN:7444/lookupservice/sdk -u [email protected] -p SSO_Password -si c:\serviceID.txt

    Note: If the vCenter Single Sign-On service is installed separate from the vCenter Server, use the FQDN of the vCenter Single Sign-On server in the preceding command.
     
    1. Log in to vCenter Server using the vSphere Web Client and a vCenter Single Sign-On administrative account to verify that the issue is resolved.

    Re-install VMware products:

    If the preceding steps do not resolve the issue, the VMware products may need to be re-installed.
    1. Uninstall vCenter Server and its components in this order:
       
      1. vSphere Web Client
      2. vCenter Server
      3. vCenter Inventory Service
      4. vCenter Single Sign-On
    2. Remove the RSA database the RSA_USER and RSA_DBA users.
    3. Rename these folders containing SSL information:
       
      • Rename C:\Program Files\VMware\Infrastructure to InfrastructureOld
      • Rename C:\ProgramData\VMware\Infrastructure to InfrastructureOld
      • Rename C:\ProgramData\VMware\VMware VirtualCenter to vCenterOld
      • Rename C:\ProgramData\VMware\vSphere Web Client to WebClientOld
      • Rename C:\ProgramData\VMware\SSL to SSLOld
    4. Create a new RSA database and users using the provided scripts. For more information, see the vSphere Installation and Setup Guide.

      Create the RSA_USER and RSA_DBA users using the script named rsaIMSLiteMSSQLSetupUsers.sql, which is included on the vCenter Server 5.1 install media.
       
    5. Install vCenter Server and its components in this order:
       
      1. SSO
      2. Inventory Service
      3. vCenter Server
      4. Web Client
    6. Log in to vCenter Server via the Web Client using admin@System-Domain for 5.1 or [email protected] for 5.5.

      Verify that the issue is resolved.


    Additional Information

    VMware Skyline Health Diagnostics for vSphere - FAQ
    Process viewing the List of Services Registered with Single Sign-On
    VMware vSphere Web Client 5.1 reports this SSL warning after an installation or upgrade: Failed to verify the SSL certificate for one or more vCenter Server Systems
    Deploying and using the SSL Certificate Automation Tool 1.0.x
    Deploying and using the SSL Certificate Automation Tool 5.5
    Installing vCenter Single Sign-On 5.5 fails if the password for [email protected] contains certain special character
    “Failed to verify the SSL certificate" after upgrading to vCenter Server 5.5 U1 or later

    Read this article in different languages here:
    vSphere Web Client 显示错误: 无法验证一个或多个 vCenter Server 系统的 SSL 证书
    vSphere Web Client に次のエラーが表示される: 1 つ以上の vCenter Server システムに対して SSL 証明書の検証に失敗しました
    O VMware vSphere Web Client exibe o erro: Failed to verify the SSL certificate for one or more vCenter Server Systems
    Error "Failed to verify the SSL certificate for one or more vCenter Server Systems" en vSphere Web Client
    VMware vSphere Web Client zeigt folgende Fehlermeldung an: „Verifizieren des SSL-Zertifikats für ein oder mehrere vCenter Server-Systeme fehlgeschlagen“