Search the VMware Knowledge Base (KB)
View by Article ID

After making a change or restarting vCenter Single Sign-On server system, vCenter Server 5.1.x fails to start (2036170)

  • 68 Ratings

Symptoms

  • vCenter Server 5.1.x fails to start and you are unable to log in to the vSphere Web Client.
  • In the C:\ProgramData\VMware\VMware VirtualCenter\Logs\vpxd.log file, you see entries similar to:

    • <YYYY-MM-DD><TIME> [04584 info 'authvpxdMoSessionManager'] [SSO][SessionManagerMo::Init] Downloading STS Root certificates ...
      <YYYY-MM-DD><TIME> [04584 verbose '[SSO][SsoCertificateManagerImpl]'] [InitConfigManagementService]
      <YYYY-MM-DD><TIME> [04584 verbose '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Connecting to SSO Admin server ...
      <YYYY-MM-DD><TIME> [04584 trivia 'vmomi.soapStub[0]'] Sending soap request to [<cs p:000000000cdeaf40, TCP:vchostname.test.vmware.net:7444>]: retrieveServiceContent {}
      <YYYY-MM-DD><TIME> [04584 trivia 'HttpConnectionPool-000001'] [IncConnectionCount] Number of connections to <cs p:00000000cdeaf40, TCP:vchostname.test.vmware.net:7444> incremented to 1
      <YYYY-MM-DD><TIME> [04584 trivia 'HttpConnectionPool-000001'] [PopPendingConnection] Found pending connection to <cs p:00000000cdeaf40, TCP:vchostname.test.vmware.net:7444>
      <YYYY-MM-DD><TIME> [04584 trivia 'vmomi.soapStub[0]'] Request started [class Vmacore::Http::UserAgentImpl::AsyncSendRequestHelper:000000000DF7FA68]
      <YYYY-MM-DD><TIME> [04280 trivia 'Default'] SSLStreamImpl::DoClientHandshake: verifyPeerName (vchostname.test.vmware.net), peerCertDigest (), unverifiedAction (fail)
      <YYYY-MM-DD><TIME> [06108 info 'Default'] Thread attached
      <YYYY-MM-DD><TIME> [04280 trivia 'vmomi.soapStub[0]'] Request completed [class Vmacore::Http::UserAgentImpl::AsyncSendRequestHelper:000000000DF7FA68]
      <YYYY-MM-DD><TIME> [04584 trivia 'HttpConnectionPool-000001'] [DecConnectionCount] Number of connections to <cs p:00000000cdeaf40, TCP:vchostname.test.vmware.net:7444> decremented to 0
      <YYYY-MM-DD><TIME> [04584 error 'vpxdvpxdMain'] [Vpxd::ServerApp::Init] Init failed: Unexpected exception
      --> Backtrace:
      --> backtrace[00] rip 000000018018977a
      --> backtrace[01] rip 0000000180100c98
      --> backtrace[02] rip 0000000180101fae
      --> backtrace[03] rip 000000018008aeab
      --> backtrace[04] rip 0000000000564971
      --> backtrace[05] rip 0000000000501298
      --> backtrace[06] rip 00000000005016c9
      --> backtrace[07] rip 0000000000470fae
      --> backtrace[08] rip 0000000140d7bfb8
      --> backtrace[09] rip 000000013fc70078
      --> backtrace[10] rip 000000013fc7016a
      --> backtrace[11] rip 000000013fc70279
      --> backtrace[12] rip 000000013fc70609
      --> backtrace[13] rip 000000013ffb2903
      --> backtrace[14] rip 000000014075e4b9
      --> backtrace[15] rip 000000014075835c
      --> backtrace[16] rip 0000000140978a3b
      --> backtrace[17] rip 000007feff4fa82d
      --> backtrace[18] rip 000000007750652d
      --> backtrace[19] rip 000000007788c521

      -->
      <YYYY-MM-DD><TIME> [04584 trivia 'VpxProfiler'] Ctr: TotalTime = 13353 ms

    • <YYYY-MM-DD><TIME> [01892 warning '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Max connection attempts (10) reached. Giving up ...
      <YYYY-MM-DD><TIME> [01892 info '[SSO][CreateSsoFacade]'] [CreateUserDirectory] Probably connection exception: No connection could be made because the target machine actively refused it.
      <YYYY-MM-DD><TIME> [01892 error '[SSO][SsoFactory_CreateFacade]'] Unable to create SSO facade: No connection could be made because the target machine actively refused it. .
      <YYYY-MM-DD><TIME> [01892 warning 'VpxProfiler'] Vpxd::ServerApp::Init [Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)] took 110250 ms
      <YYYY-MM-DD><TIME> [01892 error 'vpxdvpxdMain'] [Vpxd::ServerApp::Init] Init failed: Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)


  • In the C:\Program Files\VMware\Infrastructure\SSOServer\utils\logs\discover-is.log file, you see entries similar to:

    <YYYY-MM-DD> - VCHOSTNAME.test.vmware.net,,,,Executing action: 'discover-is'
    <YYYY-MM-DD> -
    VCHOSTNAME.test.vmware.net,,,,Discovering identity sources
    <YYYY-MM-DD> -
    VCHOSTNAME.test.vmware.net,,,,ERROR: Bean (PrimaryCommandTarget) initialization failure
    com.rsa.ims.security.keymanager.sys.SystemModificationThresholdException: System was modified beyond the allowed threshold, cannot decrypt.
    com.rsa.common.SystemException: Bean (PrimaryCommandTarget) initialization failure
    com.rsa.ims.security.keymanager.sys.SystemModificationThresholdException: System was modified beyond the allowed threshold, cannot decrypt.
    Caused by: com.rsa.ims.components.ComponentFailureException: Unable to load bean named PrimaryCommandTarget


  • In the C:\ProgramFiles\VMware\Infrastructure\SSOServer\utils\logs\imsTrace.log file,  you see the error:

    System was modified beyond the allowed threshold, cannot decrypt.
Notes:
  •  You experience the preceding symptoms when:

    • Changes, such as Windows updates, domain name changes, applying vSphere 5.1 patches, are made to the vCenter Single Sign-On Server,
    • Any modifications, such as dependency changes, are done to vCenter Server.
    • Restating vCenter Single Sign-On server system.

  • The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
  • You can run this command to see if error messages are still present in the discover-is.log file:

    C:\Program Files\VMware\Infrastructure\SSOServer\utils>ssocli.cmd configure-riat -a discover-is -u admin -p masterPassword

Cause

This issue occurs because restarting the machine where SSO is installed may result in changes to the system.

When updates are applied to the operating system, the machine name changes, or the machine is added or removed from an Active Directory domain. These changes prevent the SSO server from starting and, as a result, vCenter Server does not start.

In addition, if you clone or change the parameters of a virtual machine where SSO is installed, such as the amount of RAM, the number of CPUs, or the MAC address, SSO fails to start.

Resolution

To resolve this issue, recover and update the master password.
 
To recover and update the master password:
  1. Click Start, right-click Command Prompt, and click Run as administrator to open a command prompt as an administrator.
  2. Set the Java home path by running this command:

    set JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre

    Note: The default location is C:\Program Files\VMware\Infrastructure\jre. Ensure that the command does not contain quotes around the path.

  3. In the system where SSO is installed, locate and navigate to the SSO server installation directory.The default location of this directory is C:\Program Files\VMware\Infrastructure\SSOServer\Utils.

  4.  Run this command:

    rsautil manage-secrets -a recover -m masterPassword

    Notes:
    • Fill in the master password for the environment in place of masterPassword in the preceding command
    • This command recovers and updates the masterPassword (admin@system-domain)

  5. Click Start > Run and type services.msc and click Enter. Right-click vCenter Single Sign On and select Restart.
  6. Restart the VMware VirtualCenter Server service. For more information, see Stopping, starting, or restarting VMware vCenter Server services (1003895).

Additional Information

To be alerted when this article is updated, click Subscribe to Document in the Actions box.

See Also

Update History

10/05/2012 - Added new step 2 in resolution. 11/15/2012 - Added additional symptom

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 68 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 68 Ratings
Actions
KB: