VCF Upgrade fails with message " MGT domain failed upgrade"
search cancel

VCF Upgrade fails with message " MGT domain failed upgrade"

book

Article ID: 318757

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

To resolve the SSH authentication .

Symptoms:
  • VCF Upgrade fails with message "MGT domain failed upgrade. Please resolve the above upgrade failure for this bundle before applying any other available bundle"
  • lcm logs similar to below
2021-09-24 19:33:04.657 DEBUG [,,,] 7233 --- [pool-4-thread-7] com.vmware.vcf.secure.ssh.SshExecuter : Establishing SSH session to host: <FQDN-ESXiHost>
2021-09-24 19:33:04.661 DEBUG [,,,] 7233 --- [pool-4-thread-7] v.s.c.s.SecurityConfigurationServiceImpl : Security config retrieved class SecurityConfig {
    certificateValidationEnabled: true
    fipsMode: false
}
2021-09-24 19:33:04.669 ERROR [,,,] 7233 --- [pool-4-thread-7] com.vmware.evo.sddc.common.util.SshUtil : Unable to create jsch CLI session: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection refused (Connection refused) at com.jcraft.jsch.Util.createSocket(Util.java:394) ~[jsch-0.1.55.jar:na]

2021-09-24 19:33:14.686 ERROR [,,,] 7233 --- [pool-4-thread-7] c.v.e.s.o.model.error.ErrorFactory : [CVASCN] FAILED_TO_UPDATE_LSOM_SETTINGS_ON_HOST Failed to update vSAN LSOM settings on the host <FQDN-ESXiHost>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to update vSAN LSOM settings on the host <FQDN-ESXiHost>
at com.vmware.vcf.migration.actions.UpdateLSOMValuesAction.updateLsomSettings(UpdateLSOMValuesAction.java:87) ~[sddcmanager-migration-app-4.3.1-vcf4310RELEASE.jar:na]

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Environment

Vmware Cloud Foundation 4.5.1
Vmware Cloud Foundation 4.5
VMware Cloud Foundation 5.0

Cause

This occurs due to SSH service  not running on ESXi hosts, and hence SSH connection is refused and  config drift upgrade fails.

Resolution

To resolve this issue turn ON the SSH service on these ESXi  hosts and re-trigger config drift update.

Steps to enable SSH on each host:
1. On vSphere client, navigate to the host(under vCenter -> Datacenter-> cluster)
2. Navigate to Configure tab -> [System] dropdown -> Services -> Select "SSH" from the services list -> click on [start] after [SSH] service is highlighted.
3. Repeat 1,2 for all the hosts which needs SSH service to be enabled

Additional Information

Impact/Risks:
None