'Failed to authentication with guest OS using supplied credentials' error while Deploy, Upgrade, Migrate or Restore of VCSA 6.5 GA or U1

Products

VMware vCenter Server

Issue/Introduction

Symptoms:

vCenter Server Appliance (VCSA) or Platform Services Controller (PSC) deployment fails when you try to deploy, upgrade, migrate, or restore to version 6.5.0GA (4602587) and 6.5.0a (4944578).

Note: This does not affect VCSA/PSC 6.5GA or 6.5.0a deployments prior to October 22, 2017 daily operations. Only restoring to 6.5GA or 6.5.0a will trigger this issue.

When you use the impacted build, you will see the following symptoms:

Deployment hangs at 80% during Stage 1 without error message.
The Installer.log file displays similar to:

2017-10-26T15:15:34.596Z - debug: Failed to get url of file in guest vm:ServerFaultCode: Failed to authenticate with the guest operating system using the supplied credentials.
2017-10-26T15:15:34.596Z - debug: [1]pollRpmInstallProgress:getGuestFileErr:ServerFaultCode: Failed to authenticate with the guest operating system using the supplied credentials.

In the /var/log/messages file of a freshly deployed VCSA, you see similar to:

2017-10-26T15:22:31.720015+00:00 localhost useradd[3196]: pam_unix(useradd:account): expired password for user root (password aged)

2017-10-26T15:22:31.720370+00:00 localhost useradd[3196]: Authentication token is no longer valid; new one required

2017-10-26T15:22:31.720632+00:00 localhost useradd[3196]: failed adding user 'cm', data deleted
2017-10-26T15:22:31.724506+00:00 localhost groupadd[3198]: pam_unix(groupadd:account): expired password for user root (password aged)

2017-10-26T15:22:31.724818+00:00 localhost groupadd[3198]: Authentication token is no longer valid; new one required

2017-10-26T15:22:31.755178+00:00 localhost vaos[1059]: USER PID ACCESS COMMAND

2017-10-26T15:22:31.755413+00:00 localhost vaos[1059]: /var/lib/rpm/Packages:
2017-10-26T15:22:31.762300+00:00 localhost vaos[1059]: 2017-10-26T15:22:31.761Z: RPM Installation failed

In the /var/log/firstboot/cloudvm.log file, you will see similar to:

usermod.bk: group 'cis' does not exist
/usr/sbin/usermod.bk -a -G cis cm
Traceback (most recent call last):
  File "/usr/sbin/usermod", line 19, in <module>
    errcode = subprocess.check_call(sys.argv)
  File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/sbin/usermod.bk', '-a', '-G', 'cis', 'cm']' returned non-zero exit status 6
error: %pre(vmware-cm-6.5.0-4944578.x86_64) scriptlet failed, exit status 1

error: vmware-cm-6.5.0-4944578.x86_64: install failed

Environment

VMware vCenter Server Appliance 6.5.x

Cause

VCSA 6.5 enforces a root password expiration of 365 days from the appliance OS build time.

At this time 6.5GA and 6.5.0a contains an expired root password.

Note: VCSA 6.5U1 will begin experiencing this issue after July 1, 2018.

Resolution

This is a known issue affecting vCenter Server Appliance 6.5.

Note:

This re-released version of vCenter Server Appliance 6.5 has known security issues. These issues are documented in VMware Security Advisories and the release notes for VMware vCenter Server Appliance Photon OS Security Patches. For example the Open Source security issues listed in VMware Security Advisories VMSA-2017-0004 and VMSA-2017-0007 are critical severity and are applicable to vCSA 6.5. VMware recommends adopting the latest version of vCenter Server Appliance 6.5 in order to remediate the known security issues

This table lists the recommended workarounds for VCSA:

VCSA releases	Impacted Build Number	Impact	Recommendation
vCenter Server 6.5 GA	4602587	Cannot be deployed, migrated or restored to.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5a	4944578	Cannot be deployed, migrated or restored to.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5b	5178943	Cannot be deployed, migrated or restored to after 02/03/2018.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5c	5318112	Cannot be deployed, migrated or restored to after 02/03/2018.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5d	5318154	Cannot be deployed, migrated or restored to after 02/03/2018.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5e	5705665	Cannot be deployed, migrated or restored to after 02/03/2018.	Apply patches 6.5.0f or 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server Appliance 6.5.0f and vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5 U1	5973321	Cannot be deployed, migrated or restored to after 07/01/2018.	Apply patches 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5 U1a	6671409	Cannot be deployed, migrated or restored to after 8/14/2018.	Apply patches 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server 6.5 Update 1c Release Notes.
vCenter Server 6.5 U1b	6816762	Cannot be deployed, migrated or restored to after 9/26/2018.	Apply patches 65U1c or later immediately to existing VCSA deployments. For more information, see vCenter Server 6.5 Update 1c Release Notes.

Additional Information

Resolution if entries similar to: debug: initiatefiletransferfromguest error: serverfaultcode: failed to authenticate with the guest operating system using the supplied credentials
vCenter Server Appliance 6.5 GA または U1 リリースへのデプロイ、アップグレード、移行、またはリストアが失敗する (52673)
简体中文：部署、升级、迁移或还原到 vCenter Server Appliance 6.5 GA 或 U1 版本失败