Symptoms:

After replacing solution user certificates using the certificate manager, you experience these symptoms:

• In the %ALLUSERSPROFILE%\VMWare\vCenterServer\logs\invsvc.log file, you see entries similar to:
  
  2016-02-16T14:24:47.640-06:00 [pool-12-thread-1 INFO com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl$RequestResponseProcessor opId=] Failed trying to retrieve token: ns0:RequestFailed: Error occured looking for solution user :: More than one solution user found
  2016-02-16T14:24:47.640-06:00 [pool-12-thread-1 ERROR com.vmware.vim.vcauthenticate.servlets.AuthenticationHelper opId=] Hit ServiceCommunicationException while fetching admin group for the SSO Admin user : [email protected]
  com.vmware.vim.query.server.ssoauthentication.exception.ServiceCommunicationException: com.vmware.vim.sso.client.exception.InternalError: Failed trying to retrieve token: ns0:RequestFailed: Error occured looking for solution user :: More than one solution user found
   
• In the %ALLUSERSPROFILE%\VMWare\vCenterServer\logs\vsphere-client.log file, you see entries similar to:
  
  [2016-02-16T14:24:43.220-06:00] [INFO ] usage-data-collector-thread com.vmware.vise.vim.security.sso.impl.SsoUtilInternal Preparing the STS configuration for https://psc.domain.com/sts/STSService/vsphere.local
  [2016-02-16T14:24:43.238-06:00] [INFO ] usage-data-collector-thread com.vmware.vise.vim.security.sso.impl.SsoUtilInternal Requesting all STS trusted root certificates from https://psc.domain.com/sso-adminserver/sdk/vsphere.local
  [2016-02-16T14:24:43.376-06:00] [WARN ] usage-data-collector-thread .c.h.i.HttpConfigurationCompilerBase$ConnectionMonitorThreadBase Shutting down the connection monitor.
  [2016-02-16T14:24:43.607-06:00] [ERROR] usage-data-collector-thread com.vmware.vim.sso.client.impl.SoapBindingImpl SOAP fault javax.xml.ws.soap.SOAPFaultException: Error occured looking for solution user :: More than one solution user found
  
  Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
   
• In the /var/log/vmware/vapi/endpoint.log file, you see entries similar to:
  
  com.vmware.vapi.endpoint.config.ConfigurationException: com.vmware.vim.sso.client.exception.InternalError: Failed trying to retrieve token: ns0:RequestFailed: Error occured looking for solution user :: More than one solution user found
  at com.vmware.vapi.endpoint.cis.StsBuilder.createToken(StsBuilder.java:178)
  at com.vmware.vapi.endpoint.cis.StsBuilder.rebuild(StsBuilder.java:73)
  at com.vmware.vapi.endpoint.cis.StsBuilder.buildInitial(StsBuilder.java:52)
  at com.vmware.vapi.state.impl.DefaultStateManager.build(DefaultStateManager.java:349)
  at com.vmware.vapi.state.impl.DefaultStateManager$1.doInitialConfig(DefaultStateManager.java:176)
  at com.vmware.vapi.state.impl.DefaultStateManager$1.run(DefaultStateManager.java:151)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)
  Caused by: com.vmware.vim.sso.client.exception.InternalError: Failed trying to retrieve token: ns0:RequestFailed: Error occured looking for solution user :: More than one solution user found
  Service-control failed. Error Failed to start vmon services.vmon-cli RC=1, stderr=Failed to start vapi-endpoint services. Error: Operation timed out.
   
• The vSphere Web Client fails with the error:
  
  A server error occurred.
  [500] SSO error: null
  Check the vSphere Web Client server logs for details.
   
• Navigating to the https://fqdn/psc/ fails with the error:
  
  HTTP Status 400 - An error occurred while sending an authentication request to the PSC Single Sign-On server - null
  type Status report
  message An error occurred while sending an authentication request to the PSC Single Sign-On server - null
  description The request sent by the client was syntactically incorrect.
  VMware vFabric tc Runtime 2.9.7.RELEASE/7.0.55.A.RELEASE

VMware vCenter Server 6.0.x

This issue is caused by a change in the certificate-manager in vCenter Server Update 1b. New options are present for processing the certool.cfg file correctly, as well as processing config files for each individual solution user. If these config files do not have unique information for each solution user, the generated certificates have the same Subject.

For example, in the C:\ProgramData\VMware\vCenterServer\logs\vmca\certificate-manager.log file, you see entries similar to:

2016-02-16T19:28:59.734Z INFO certificate-manager Selected operation: Replace Solution user certs with VMCA Certificate
2016-02-16T19:28:59.735Z INFO certificate-manager Please configure machine.cfg with proper values before proceeding to next step.
2016-02-16T19:28:59.735Z INFO certificate-manager Press Enter key to skip optional parameters or use Default value.
2016-02-16T19:29:23.529Z INFO certificate-manager machine.cfg file contents.
2016-02-16T19:29:23.530Z INFO certificate-manager Country = US
2016-02-16T19:29:23.530Z INFO certificate-manager Name = vSphere
2016-02-16T19:29:23.530Z INFO certificate-manager Organization = VMware
2016-02-16T19:29:23.530Z INFO certificate-manager OrgUnit = Support
2016-02-16T19:29:23.530Z INFO certificate-manager State = Colorado
2016-02-16T19:29:23.530Z INFO certificate-manager Locality = Denver
2016-02-16T19:29:23.530Z INFO certificate-manager #IPAddress =
2016-02-16T19:29:23.530Z INFO certificate-manager Email = [email protected]
2016-02-16T19:29:23.530Z INFO certificate-manager Hostname = vcsa.domain.com

The same information will be seen for these options in the other config files (vsphere-webclient.cfg, vpxd.cfg, vpxd-extension.cfg) which causes the certificates not to be unique.

This issue is resolved in vCenter Server 6.0 Update 3, available at VMware Downloads.

 

 

Workaround:
To workaround this issue, re-generate new Solution User Certificates, ensuring that each certificate is given a unique subject.
This can typically be achieved by making the Name:value unique for each Solution user.
Using the Certificate Manager > Select Option 6 to re-generate new VMCA issue SOlution User Certificate.

"An error occurred while sending an authentication request to the PSC Single Sign-On server - null" while connecting to PSC Client after upgrading vCenter Server to 6.5
Replacing default certificates with CA signed SSL certificates in vSphere 6.x
vCenter Server または PSC 6.0 Update 1b での Certificate Manager を使用した証明書の更新