"Error [500] SSO error: com.vmware.vim.vmomi.core.exception...." occurs when VMware VAPI Endpoint service fails to start
search cancel

"Error [500] SSO error: com.vmware.vim.vmomi.core.exception...." occurs when VMware VAPI Endpoint service fails to start

book

Article ID: 322851

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • When trying to connect to vCenter Server using the web client, you may receive a similar error:
[500] SSO error: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint doesn't match
Check the vSphere Web Client server logs for details.
  • When attempting to check the status of services on VC, you may notice that the vapiEndpoint service is not running.
service-control --status --all
Running:
 EsxAgentManager VMWareAfdService VMWareCertificateService VMWareDirectoryService VMwareComponentManager VMwareDNSService VMwareIdentityMgmtService
onapi vmsyslogcollector vmware-cis-config vmware-license vmware-perfcharts vmware-psc-client vmwareServiceControlAgent vpxd vpxd-svcs vsan-health 
Stopped:
 VMWareCAMService content-library mbcs vapiEndpoint vmware-autodeploy-waiter vmware-imagebuilder vmware-network-coredump
  • In the endpoint.log you may see similar entries:
2017-11-25T19:34:28.085-06:00 | ERROR | state-manager1            | DefaultStateManager            | Could not initialize endpoint runtime state.
com.vmware.vapi.endpoint.config.ConfigurationException: Failed to retrieve SSO settings.
    at com.vmware.vapi.endpoint.cis.SsoSettingsBuilder.buildInitial(SsoSettingsBuilder.java:60)
    at com.vmware.vapi.state.impl.DefaultStateManager.build(DefaultStateManager.java:349)
    at com.vmware.vapi.state.impl.DefaultStateManager$1.doInitialConfig(DefaultStateManager.java:176)
    at com.vmware.vapi.state.impl.DefaultStateManager$1.run(DefaultStateManager.java:151)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.vmware.vim.sso.admin.exception.CertificateValidationException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint verification is not configured
virgo.logs
2017-12-15T09:41:20.824-06:00] [INFO ] bio-127.0.0.1-9090-exec-2941 70133207 162400 262092 com.vmware.vise.util.session.SessionUtil                          ClientId 262099 was automatically generated 
[2017-12-15T09:41:20.824-06:00] [WARN ] bio-127.0.0.1-9090-exec-2941 70133207 162400 262092 com.vmware.vise.util.session.SessionUtil                          Just clobbered an existing httpRequest 
[2017-12-15T09:41:20.860-06:00] [INFO ] bio-127.0.0.1-9090-exec-2941 70133216 162408 262099 com.vmware.vise.vim.cm.impl.ComponentManagerServiceImpl           Retry won't be attempted for CM https://AR-FSM-VC.corp.summitutilities.com/cm/sdk?hostid=e2a89e40-47ba-11e7-87f5-44a8422df4d7 for error: com.vmware.vim.vmomi.client.exception.SslException - com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint doesn't match 
[2017-12-15T09:41:20.889-06:00] [WARN ] bio-127.0.0.1-9090-exec-2941 70133216 162408 262099 com.vmware.vapi.provider.local.LocalProvider                      Implementation method reported unexpected exception: com.vmware.vim.vmomi.client.exception.SslException com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint doesn't match
    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:250)
    at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:51)
    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:226)
    at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:110)
 
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server 6.5.x
VMware vCenter Server 6.7.x
VMware vCenter Server Appliance 6.7.x

Resolution

To resolve the issue, follow below steps:
  1. Open Command Prompt as Administrator.
  • vCenter Server installed on Windows : cd C:\Program Files\VMware\vCenter Server\vmafdd
  • vCenter Server Appliance: cd /usr/lib/vmware-vmafd/bin/
  1. Export Trusted Root CRLS to a text file:
  • vCenter Server installed on Windows :
vecs-cli.exe entry list --store TRUSTED_ROOT_CRLS  | findstr Alias > C:\trusted_root_crl.txt
  • vCenter Server Appliance:
./vecs-cli entry list --store TRUSTED_ROOT_CRLS |grep Alias > /tmp/trusted_root_crl.txt
  1. Open the trusted_root_crl.txt and Replace "Alias" with "vecs-cli entry delete --store TRUSTED_ROOT_CRLS -y --alias"
  1. Delete all the Alias in this CRLS store using the command:
  • vCenter Server installed on Windows :

vecs-cli.exe entry delete --store TRUSTED_ROOT_CRLS --Alias <Certificate_Alias> -y  (Here Certificate_Alias is the Alias of each certificate returned by above list command)

  • vCenter Server Appliance:

./vecs-cli entry delete --store TRUSTED_ROOT_CRLS --Alias <Certificate_Alias> -y  

Note: Certificate_Alias is the alias of each certificate returned by the above list command.

Example: ./vecs-cli entry delete --store TRUSTED_ROOT_CRLS -y --alias 76ddcc9450d3ccd5dd475df4e12708bf38c8875c

  1. ​Restart VapiEndpoint Service on vCenter server

service-control --stop vapiEndpoint
service-control --start vapiEndpoint

You can also run the scripts attached under the attachments section in this article for;
  • Windows: remove_clrs
  • Appliance: clr-fix
Note:  If there are expired Certificates within the BACKUP_STORES please refer to  Clearing BACKUP_STORES certificates in the VCSA via shell script.(82560)

Additional Information

How to stop, start, or restart vCenter Server 6.x services
Resolution when entries similar to: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified

Attachments

remove_clrs get_app
crl-fix get_app
Powered by Wolken Software