Symptoms:

• vCenter converge process fail during VMAFD firstboot
• Running firstboot stage 1 of 4 failed
• In /var/log/firstboot/vmafd-firstboot.py_stdout.log, you see entries similar to:

   

• In /var/log/firstboot/vmafd-firstboot.py_stderr.log, you see entries similar to:

 

There are two DNS RPMs: vmware-dns-server and vmware-dns-client. The client RPM is installed on vCenter with external PSC node, while the server RPM is installed on PSC nodes. When the RPMs are installed, several configuration parameters are added to the "registry”. The client and server RPMs each register a different set of config parameters, however both sets are stored in the same location in the registry, so when the server RPM is installed, it should overwrite the client parameters with the server set.

This issue occurs when the vmware-dns-server RPM is correctly installed on the machine, but the registry values still correspond to the client settings, not the server settings. This causes the VMwareDNSService service fail to start, since the service can't find the server settings that it needs in the registry, causing the vmafd firstboot to fail.

This issue is resolved in vCenter Server 6.7 U3a, available at VMware Downloads.

Workaround:
Note: Take snapshot of all the vCenter Servers and PSCs (which are part of ELM) in a Powered off state before proceeding with below steps:

To workaround this issue, proceed with below steps on the vCenter Server Appliance where Convergence failed :

1. Find the vmware-dns-server rpm:
   • find / -name vmware-dns-server*.rpm  
2. Remove the vmware-dns-server rpm
   • rpm -ev --nodeps vmware-dns-RPMName-FromStep1 (without .rpm extension)

3. Install the vmware-dns-server rpm
   • rpm -ivh --nodeps /root/velma/rpm/<vmware-dns-name-FromStep1.rpm>

4. Stop VMAFD Service
   • service-control --stop vmafdd
5. Recreate VECS DB from convergence process backup
   • mv /storage/db/vmware-vmafd/afd.db /storage/db/vmware-vmafd/afd.db_backup
   • Copy AFD DB from backup
     • ls /root/velma/rollback/ -l (list the files and identify the full name of afd.db backup with the converge date)
     • Copy the backup DB to /storage/db/vmware-vmafd folder with file name 'afd.db' (Example - cp /root/velma/rollback/2019-08-09_05-58-37afd.db /storage/db/vmware-vmafd/afd.db)
6. Start VMAFD Service
   • service-control --start vmafdd
7. Modify file /etc/vmware/install-defaults/deployment.node.type from embedded to management
   • vi /etc/vmware/install-defaults/deployment.node.type
   • Modify the string embedded to management
   • Save and Exit the VI editor (Press Esc Key followed by :wq! and Enter)
8. Modify file /etc/issue from embedded to Exernal
   • vi /etc/issue
   • Modify the string embedded to External
   • Save and Exit the VI editor (Press Esc Key followed by :wq! and Enter)
9. Modify file /etc/vmware/deployment.node.type from embedded to management
   • vi /etc/vmware/deployment.node.type
   • Modify the string embedded to management
   • Save and Exit the VI editor (Press Esc Key followed by :wq! and Enter)
10. Modify file /etc/applmgmt/appliance/appliance.conf from embedded to management
    • vi /etc/applmgmt/appliance/appliance.conf
    • Modify the string embedded to External
    • Save and Exit the VI editor (Press Esc Key followed by :wq! and Enter)
11. Modify DomainState in Likewise Registry to 0x00000002
    • /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\vmafd\Parameters]' DomainState 2
    • Verify the modified value using list_values command - /opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\vmafd\Parameters]'
12. Recreate the Deleted Machine Account by following the Steps in KB Recreate vCenter Server Machine Account in Platform Services Controller after Failed Convergence 
13. Rename the Convergence failed backup folder /root/velma
    • mv /root/velma /root/velma_$(date +%d-%m-%Y_%H_%M_%S)
14. Rename failed Convergence logs under /var/log/vmware/converge/
    • mv /var/log/vmware/converge/converge.log /var/log/vmware/converge/converge_$(date +%d-%m-%Y_%H_%M_%S).log
    • mv /var/log/vmware/converge/converge_status.json /var/log/vmware/converge/converge_status_$(date +%d-%m-%Y_%H_%M_%S).json
    • mv /var/log/vmware/converge/converge_rest.log /var/log/vmware/converge/converge_rest_$(date +%d-%m-%Y_%H_%M_%S).log
15. Start all the services
    • service-control --start --all
16. Retry the Convergence

Deploying or Installing an additional Platform Service Controller 6.0 Update 1b fails during vmafd firstboot
Recreate vCenter Server Machine Account in Platform Services Controller after Failed Convergence