Troubleshooting vSphere Auto Deploy
search cancel

Troubleshooting vSphere Auto Deploy

book

Article ID: 313540

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

Symptoms:
This article provides troubleshooting guidance on VMware Auto Deploy 5.x. This article does not apply to vSphere 6.0

Auto Deploy is a new feature of vSphere 5.0 that can be used to quickly provision and configure ESXi hosts.

Note: For more information, see the vSphere Installation and Setup Guide. The guide contains definitive information. If there is a discrepancy between the guide and the article, assume the guide is correct.

Environment

VMware vCenter Server 5.5.x
VMware vCenter Server 5.0.x
VMware vSphere ESXi 5.0
VMware vCenter Server 5.1.x
VMware vSphere ESXi 5.5
VMware vSphere ESXi 5.1

Resolution

VMware Auto Deploy can be installed on a standalone Windows machine. VMware Auto Deploy is also available with the VMware vCenter Appliance.
 

Important files and locations

Note: Some of these paths may be hidden by default.
 
ElementLocation
Default installation paths for the Auto Deploy server
  • 32-bit: C:\Program Files (x86)\VMware\VMware vSphere Auto Deploy
  • 64-bit: C:\Program Files (x86)\VMware\VMware vSphere Auto Deploy
Configuration files for the Auto Deploy server
  • 32-bit: C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy
  • 64-bit: C:\ProgramData\VMware\VMware vSphere Auto Deploy
Configuration files for the vCenter Server Appliance
  • Configuration files: /etc/vmware-rbd
  • Runtime state files: /var/lib/rbd
Database files for the Auto Deploy server
  • 32-bit: C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy\Data\db*
  • 64-bit: C:\ProgramData\VMware\VMware vSphere Auto Deploy\Data\db*
Database files for the vCenter Server Appliance
  • to vCenter 7.0.3.1 /var/lib/rbd/db*
  • from vCenter 7.0.3.2 and above /etc/vmware-rbd/db/db
Cache for the Auto Deploy server
  • C:\Users\All Users\VMware\VMware vSphere Auto Deploy
  • 32-bit: C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy\Data\cache\
  • 64-bit: C:\ProgramData\VMware\VMware vSphere Auto Deploy\Data\cache\
Cache for the vCenter Server Appliance
  • /var/lib/rbd/cache
Main configuration files for the Auto Deploy server
  • 32-bit: C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml
  • 64-bit: C:\ProgramData\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml
 

Logging


Obtaining log files

To obtain log files through vCenter Server:
  1. Log in to vCenter Server with the vSphere Client.
  2. Click Home > Auto Deploy > Download AutoDeploy Log Files.
To manually obtain log files for the Auto Deploy server, go to:

%configuration file location%\Logs

To manually obtain log files for the vCenter Server Appliance, go to one of these locations:
  • /var/log/vmware/rbd
  • /etc/vmware-rbd/httpd/logs
Note: vCenter Server logs do not include Auto Deploy Logs, and must be collected separately as above.


Increasing log file size and rotation

To increase the log file size and rotation:
 
  1. Open the logging.conf file using a text editor. The file is located at:
     
    • 32-bit Windows:

      C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy\logging.conf
       
    • 64-bit Windows:

      C:\ProgramData\VMware\VMware vSphere Auto Deploy\logging.conf
       
    • vCenter Server Appliance:

      /etc/vmware-rbd/logging.conf
       
  2. Change the size value to 1000000 and the backupCount value to 5. For example:

    [autodeploy]
    size=1000000
    backupCount=5


    Note: The size setting is in bytes.
     
  3. Save and close the file.
  4. Restart the Auto Deploy service.
 

Common tasks


Editing the Auto Deploy service configuration

To edit the Auto Deploy service configuration:
 
  1. Open the main Auto Deploy configuration file using a text editor. The file is located at:
     
    • 32-bit Windows:

      C:\Documents and Settings\All Users\Application Data\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml
       
    • 64-bit Windows:

      C:\ProgramData\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml
       
    • vCenter Server Appliance:

      /etc/vmware-rbd/autodeploy-setup.xml
       
  2. Adjust these parameters:
     
    • <serviceAddress>IP_address</serviceAddress>

      Where IP_address is the Auto Deploy IP address.
       
    • <defaultValues>
      <port>port_number</port>
      <maxSize>max_cache_size</maxSize>
      </defaultValues>


      Where port_number is the Auto Deploy port, and max_cache_size is the maximum cache size in GB.
       
    • <vCenterServer>
      IP_address

      <port>port_number</port>
      <user>username</user>
      </vCenterServer>


      Where IP_address is the vCenter Server IP address, port_number is the vCenter Server port, and username is the vCenter Server user name.
       
  3. Save and close the file.
  4. Restart the Auto Deploy service.
Note:
  • This information is also stored in the vCenter Server registry on the Auto Deploy Server located at:

    HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\VMware, Inc.\VMware vSphere Auto Deploy
     
  • Before making any registry modifications, ensure that you have a current and valid backup of the registry and the virtual machine. For more information on backing up and restoring the registry, see the Microsoft Knowledge Base article 136393.
     

    Note: The preceding link was correct as of June 05, 2015. If you find a link is broken, provide feedback and a VMware employee will update the link.


Re-registering the Auto Deploy service

When re-registering the Auto deploy service with vCenter Server, the Auto Deploy rules might need to be rebuilt on vCenter Server. You may also have to re-register the Auto Deploy service if the vCenter Server or Auto Deploy IP addresses change, if the service cannot start or if the SSL certificates change.

Both Windows and the vCenter Server Appliance Auto Deploy register commands use the same syntax and switches.

  • Run the commands from:
     
    • On Windows:

      C:\Program Files (x86)\VMware\VMware vSphere Auto Deploy\autodeploy-register.exe
    • On Windows 2012:

      C:\Program Files (x86)\VMware\vCenter Server\autodeploy\autodeploy-register.exe
    • On the vCenter Server Appliance:

      /usr/bin/autodeploy-register
       
  • To unregister the Auto Deploy service, run the command:

    autodeploy-register -U -a x.x.x.x -u root -w vmware -p 80

    Where x.x.x.x is the vCenter Server IP address, -u root is the user, and -w vmware is the password.
     
  • To register the Auto Deploy service, run the command:

    On Windows:

    autodeploy-register -R -a x.x.x.x -u root -w vmware -p 80 -s "C:\ProgramData\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml"

    On the vCenter Server Appliance:

    autodeploy-register -R -a x.x.x.x -u root -w vmware -p 80 -s /etc/vmware-rbd/autodeploy-setup.xml

    Where x.x.x.x is the vCenter Server IP address, -u root is the user, and -w vmware is the password.
     
  • If the SSL certificates have changed use this command to register the Auto Deploy service:

    Note: Currently, this command is not valid for vCenter Server 5.1. For more information, see vSphere Auto Deploy service fails after a change to vCenter Server SSL Certificates with the error: Server has wrong SHA1 thumbprint (2063528).

    On Windows:

    autodeploy-register.exe -R -a vCenterServerFQDN -u vCenterAdminUsername -w password -s "C:\ProgramData\VMware\VMware vSphere Auto Deploy\vmconfig-autodeploy.xml" -f -T new_vCenter_Server_SSL_Cert_Thumbprint

    On the vCenter Server Appliance:

    autodeploy-register -R -a vCenterServerFQDN -u root -w password -s "/etc/vmware-rbd/autodeploy-setup.xml" -f -T new_vCenter_Server_SSL_Cert_Thumbprint
     
Note: new_vCenter_Server_SSL_Cert_Thumbprint refers to the MACHINE_SSL_CERT and with no colons. 

Troubleshooting


To determine information about registered Auto Deploy ESXi hosts:
 
  1. Access this URL in a web browser:

    https://x.x.x.x:port_number/vmw/rbd/host/

    Where:
    • x.x.x.x is the Auto Deploy server IP address
    • port_number is the Auto Deploy port (6501 by default)
       
  2. Click on each hash link. Each hash represents an ESXi host that has registered with the Auto Deploy service. The page displays information about the ESXi host, including the DHCP/TFTP that was used to boot, the server model, and the MAC address.

    For example:

    Host List:

    5dc289181e9eecc49590d01fa32b0f42

     
    • hostname=
    • ipv4=xxx.xxx.x.xxx
    • mac=xx:xx:xx:xx:xx:xx
    • uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    • vendor=VMware,Inc
       
  3. There are also two links for boot.cfg and get boot.cfg.
     
    • Click the boot.cfg link (or the Get gPXE Configuration link) to view information about the host and the profiles being used.

      For example:

      #!gpxe

      echo
      echo
      echo ******************************************************************
      echo * Booting through VMware Auto Deploy...
      echo *
      echo * Machine attributes:
      echo * . asset=No Asset Tag
      echo * . domain=
      echo * . hostname=
      echo * . ipv4=xxx.xxx.x.xxx
      echo * . mac= xx:xx:xx:xx:xx:xx
      echo * . model=VMware Virtual Platform
      echo * . oemstring=[MS_VM_CERT/SHA1/27d66596a61c48dd3dc7216fd715126e33f59ae7]
      echo * . oemstring=Welcome to the Virtual Machine
      echo * . serial=VMware-xx xx xx xx xx xx xx xx-xx xx xx xx xx xx xx xx
      echo * . uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      echo * . vendor=VMware, Inc.
      echo *
      echo * Host Profile: hostprofile-1
      echo * Image Profile: ip-VMware, Inc.-test1-d5107713e36092ff920705dbf627a092
      echo * VC Host: host-14
      echo *
      echo * Bootloader VIB version: 5.0.0-1.2.381531

      echo ******************************************************************

       
    • Click the Get boot.cfg link to view information about the cached files being used to boot the server.

      For example:

      bootstate=0
      title=Loading VMware ESXi
      kernel=/vmw/cache/a3/36c0980af7357f5d515242e6458be5/tboot.aaef3f985d1dfc669c9490939c82e36f
      kernelopt=BOOTIF=01-00-0c-29-4c-8c-29
      modules=/vmw/cache/72/1776e38e761db08fff0db5edec43af/b.e174d89c00afa21ae697977203c2b9ce --- /vmw/cache/72/1776e38e761db08fff0db5edec43af/useropts.e174d89c00afa21ae697977203c2b9ce --- /vmw/cache/72/1776e38e761db08fff0db5edec43af/k.e174d89c00afa21ae697977203c2b9ce --- /vmw/cache/a3/36c0980af7357f5d515242e6458be5/a.aaef3f985d1dfc669c9490939c82e36f --- /vmw/cache/ff/eae6feb2e63579c776a6041b3de0da/ata-pata.5fa67d0ce923ca8647a45c431c385879
 

Database corruption


vSphere Auto Deploy utilizes a database to store information about hosts. It is possible that the Auto Deploy database may need maintenance. SQLite is the tool of choice for performing these activities. By default, SQLite comes with the vCenter Server Appliance only. However, the SQLite shell is a free application and is available for use with Windows at http://sqlite.org/download.html.
 
  • To connect to the Auto Deploy database, run the command:

    sqlite3 "C:\Users\All Users\VMware\VMware vSphere Auto Deploy\Data\db"
  • To connect to Auto Deploy database on Linux platform, to vCenter version 7.0.3.1, run the command:

sqlite3 /var/lib/rbd/db
  • To connect to Auto Deploy database on Linux platform, from vCenter version 7.0.3.2 and above, run the command:
sqlite3 /etc/vmware-rbd/db/db
  • To verify you are connected to the Auto Deploy database, run the command:

    sqlite> .databases
     
  • To perform an integrity check of the entire database, run the command:

    sqlite> PRAGMA integrity_check;

    If the database is intact, the output from this command is OK.
     
  • To reclaim empty or free space from the database, run the command:

    sqlite> VACUUM;
     
  • To delete and recreate indices from scratch (which may improve performance), run the command:

    sqlite> REINDEX;
     

Verifying Auto Deploy profiles and rulesets


For troubleshooting purposes, it may be necessary to determine which rules are currently applied, and if a particular host meets this compliance. These commands are used with the vSphere 5.0 PowerCLI and the ImageBuilder Snap-in.

To determine currently deployed rules, run the commands:
  • get-deployruleset

    The output displays the active ruleset to be used with Auto Deploy. An active ruleset is a collection of image/host profiles and the patterns with which these rules match (for example, mac address, vendor, IP address range).
     
  • test-deployrulesetcompliance ESXhostname

    The output displays a list of the current profiles applied to the host and the expected profile. This is useful for determining if a host is up to date.
 

gPXE Troubleshooting


During initial boot, an ESXi host attempts to obtain an IP address via DHCP using PXE. In the unlikely event that the server fails to load the PXE boot image, it is possible to boot into a gPXE shell by pressing Ctrl+B. These commands can be useful for troubleshooting PXE networking configuration issues.
  • To attempt to obtain a DHCP address, run one of these commands:
     
    • dhcp net0 (the machine can then be pinged)
       
    • config net0
       
    • set net0/ip x.x.x.x
      set net0/netmask x.x.x.x
      set net0/gateway x.x.x.x
      set net0/dns x.x.x.x

       
  • To verify the routing information, run the command:

    route
     
  • To verify the PXE image loaded, run the command:

    imgstat

    Note: The default PXE image is vmw-hardwired.
     
  • To verify TFTP connectivity, run the command:

    imgfetch tftp://x.x.x.x/tramp


Additional Information

vSphere Auto Deploy service fails after a change to vCenter Server SSL Certificates with the error: Server has wrong SHA1 thumbprint
vSphere 5.x の vSphere Auto Deploy に CA 署名された SSL 証明書を構成する方法
vSphere Auto Deploy のトラブルシューティング
故障排除 vSphere Auto Deploy
Error:"Failed to register Auto Deploy" while Upgrading Windows vCenter Server 6.5 to 6.7 (55076)