Troubleshooting VMware Single Sign-On configuration and installation issues in a Windows server
search cancel

Troubleshooting VMware Single Sign-On configuration and installation issues in a Windows server

book

Article ID: 305665

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

This article guides you through the process of troubleshooting issues encountered while configuring or installing VMware Single Sign-On on a Windows server. It helps you eliminate common causes for the problem by verifying the scope of the problem as well as configuration, database, and networking related problems that could cause an issue.


Symptoms:
  • Installer warns that Auto-Discovery failed
  • Cannot install vSphere Single Sign On (SSO)
  • Installation of vSphere SSO fails
  • You see one of these errors:
    • Could not contact the Lookup Service.
    • Connection timed out
    • Connection Refused
    • Unexpected status code: 404
    • Return code is: SSLHandshakeFailed
    • Could not connect to vCenter Single Sign-On
    • Wrong Input - either a command line argument is wrong, a file cannot be found or the spec file doesn't contain the required information, or the clocks on the two systems are not synchronized. Check vm_ssoreg.log in the system temporary folder for details


Environment

VMware vCenter Server 5.1.x

Resolution

Validate that each troubleshooting step below is true for your environment. Each step will provide instructions or a link to a document, in order to eliminate possible causes and take corrective action as necessary. The steps are ordered in the most appropriate sequence to isolate the issue and identify the proper resolution. Do not skip a step when troubleshooting in each subsection of the document.
  1. If you are seeing errors during the installation of SSO and a warning about auto-discovery failing:
     
    1. Validate the configuration of the of the SSO server. For more information, see the Required Information for Installing or Upgrading vCenter Single Sign On, Inventory Service, and vCenter Server section of the vSphere Installation and Setup Guide.
       
    2. Ensure the time difference between the vCenter SSO server and the Active Directory Domain controllers. If the time is off by more than 5 minutes, Kerboros authentication fails and, therefore, automatic discovery fails.
       
    3. Verify that each domain controller has as properly configured PTR records in DNS and ensure that the contents of the PTR record are accurate. To check this from the Windows command line, you can run the nslookup command on both the name and IP:

      For Name:

      nslookup server.domain.com
      Server: DNS Server
      Address: Server IP address

      Name: server.domain.com
      Address: IP address


      For IP address:

      nslookup IP address
      Server: DNS Server
      Address: Server IP address

      Name: server.domain.com
      Address: IP address

       
    4. If SSL is enabled in the domain controllers, verify that the SSL certificate is still valid. By default, SSL is enabled on most Windows Server 2008 machines.

      Note: To determine if SSL is enabled on the domain controller, run ldp.exe and connect to the domain controller on port 636. The output in the right column of the ldp.exe screen indicates if SSL is enabled on the domain controller.

       
    5. Remove and rejoin the vCenter SSO host to the domain. This exposes any connectivity or trust based errors if there are failures during the addition to the domain
       
    6. After the installation completes, review the install.log and imsTrace.log files SSO_Server_Directory\utils\logs\ for errors in the auto discovery process.

      Note: If there have been changes made, you can run this command to observe if there are still any error messages:

      <SSO Server Directory>\utils\ssocli configure-riat -a discover-is -u admin

       
    7. If you see errors during the the SSO installer:
       
      1. Validate the configuration of the of the SSO server. For more information, see the Required Information for Installing or Upgrading vCenter Single Sign On, Inventory Service, and vCenter Server section of the vSphere Installation and Setup Guide.
         
      2. You may see this error while trying to input a vCenter Server administrator user or group while configuring SSO in HA or Multisite mode:

        Wrong Input - either a command line argument is wrong, a file cannot be found or the spec file doesn't contain the required information, or the clocks on the two systems are not synchronized. Check vm_ssoreg.log in the system temporary folder for details

        This error indicates that HA or multisite mode does not recognize the local identity source when specifying a user for the configuration.

        To resolve this issue, qualify the user name with the source you want to use. For example, instead of using Administrators use [email protected].

        For more information on user qualifications, see Understanding and troubleshooting vCenter Single Sign-On users, groups and login qualifications (2033875).
         

      3. For more troubleshooting information, see the SSO_SERVER\utils\logs\imsTrace.log, install.log, and %TEMP%\vminstall.log files.

        Note: Ensure to collect the logs at the time the installation fails. The errors appear as messages similar to ####: Installation failed due to.... Before you click OK, gather a Single Sign On support bundle to assist support in determining the problem. At the command prompt, run this command:

        C:\Windows\System32\cscript.exe "<SSO Server>\scripts\sso-support.wsf" /z

If you experience issues with vCenter SSO during the installation of vCenter Server, Inventory Service or Web client, perform these steps:
 

  1. Verify if there is a time difference between the vCenter SSO server and the Active Directory Domain controllers. If the time is off by more than 5 minutes, Kerboros authentication fails and, therefore. the automatic discovery fails.
  2. Review the logs for the product being installed to determine the cause of the error. The error message will mention the log file which should be reviewed. For example:

    Could not contact Lookup Service. Please check VM_ssoreg.log....

     
  3. Review this table for the various messages and the corresponding cause and solution.

    Note: This table may not include all messages and will be updated as issues surface.

     
    MessageCause and Solution

    java.net.ConnectException: Connection timed out: connect

    Indicates that the IP address is incorrect, a firewall is blocking access to Single Sign On, or Single Sign On is overloaded.

    Ensure that the Single Sign On port (by default 7444) is not blocked by a firewall, and that the machine on which Single Sign On is installed has adequate free CPU, IO, and RAM capacity as noted in Required Information for Installing or Upgrading vCenter Single Sign On, Inventory Service, and vCenter Server section of the
    vSphere Installation and Setup Guide .

    java.net.ConnectException: Connection refused: connect

    Indicates that the provided IP address or FQDN is incorrect, that Single Sign On has not started or has started within the past minute.

    Verify that Single Sign On is working by checking the status of vCenter Single Sign On (Windows) and vmware-sso (Linux). Also, restart the service.

    Unexpected status code: 404. SSO Server failed during initialization

    Indicates that the Single Sign-on service did not initialize properly.

    Try restarting the Single Sign-On. If this fails, review the ssoAdminServer.log and imsTrace.log files for details on SSO startup or review the logs for the application which is trying to connect to SSO.

    Return code: SslHandshakeFailed

    The error shown in the UI begins: Could not connect to vCenter Single Sign-on.

    This is an extremely uncommon error. It indicates that the provided IP address or FQDN which resolves to the Single Sign On host was not the one used when installing Single Sign On.

    In
    %TEMP%\VM_ssoreg.log , locate the line containing the error:

    hostname in certificate didn't match: <install-configured FQDN or IP> != A or B or C

    Where A was the FQDN entered when Single Sign On was installed and B and C are system-generated allowable alternatives.

    Correct the configuration to use the FQDN on the right of the not equal to ( !=) sign. In most cases, use the FQDN specified during Single Sign On installation. If none of these options are possible, recover your Single Sign On SSL configuration.

     

Note: If your problem still exists after trying the steps in this article:



Additional Information

Windows サーバにおける VMware Single Sign-On の構成およびインストールの問題のトラブルシューティング
对 Windows 服务器中的 VMware Single Sign-On 配置和安装问题进行故障排除