Verify and resolve expired vCenter Server certificates using command line
search cancel

Verify and resolve expired vCenter Server certificates using command line

book

Article ID: 344201

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

This article provides steps to verify certificate expiration dates and resolve expired certificates in the vCenter Server using the command line interface.

Symptoms:
  • You see warnings in the vCenter interface showing certificates are expiring soon.
  • You see the error:
503 service not available...endpoint


Environment

VMware vCenter Server 8.0
VMware vCenter Server 6.x
VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.5.x

Cause

This issue is seen when one or more required certificates are expired or will expire soon.

Resolution

Verify certificate expiration date

  1. Check the Single Sign-on Token Signing (STS) certificate, see Checking Expiration of STS Certificate on vCenter Server.
  2. Run this command to see the status of the environments certificates":
  • Run this command on the vCenter Appliance:

for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

  • Run this command on the Windows vCenter Server:

$VCInstallHome = [System.Environment]::ExpandEnvironmentVariables("%VMWARE_CIS_HOME%");foreach ($STORE in & "$VCInstallHome\vmafdd\vecs-cli" store list){Write-host STORE: $STORE;& "$VCInstallHome\vmafdd\vecs-cli" entry list --store $STORE --text | findstr /C:"Alias" /C:"Not After"}
 

You will see an output similar to:

List of certificates to find expired certificates
  1. Ensure the dates are in the future

Resolving expired certificates

Caution
  • Backup or create a virtual machine snapshot before proceeding.
  • It is recommended power off all linked external Platform Services Controllers/vCenters with embedded PSCs at the same time and to take a snapshot of every linked node VM.


Custom certificates

If you have expired trusted root or SSL certificates it is recommended to get the system working again using the default VMware Certificate Authority certificates, then to re-apply your custom certificate, see Replacing a vSphere 6.x /7.x Machine SSL certificate with a Custom Certificate Authority Signed Certificate


STS certificate

For vCenter with embedded PSC, or external PSCs only, do the following only on one node for each  system of linked nodes: replace the STS certificate per "Signing certificate is not valid" error in VCSA 6.5.x/6.7.x and vCenter Server 7.0.x


Trusted root certificate

  • For vCenter with embedded PSC, or external PSCs only, do the following once in a system of linked nodes: Run certificate-manager per How to use vSphere Certificate Manager to Replace SSL Certificates, and use Option 4 to generate a new root certificate and replace all certificates.
  • On all remaining vCenter and PSCs in the linked system, do the following:
  1. Run certificate-manager option 3 to replace the Machine SSL certificate
  2. Run certificate-manager option 6 to replace the solution user certificates


Machine SSL certificate

On each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 3 to replace the SSL certificate.


Solution user certificates

If one or more of these has expired, On each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 6 to replace the solution users certificates.

Note: If option 3 or 6 of the Certificate manager fails for the VCenter you could try using option 8 to reset all Certificates. 


Additional Information

Manually reviewing certificates in VMware Endpoint Certificate Store for vSphere 6.x
"ERROR certificate-manager 'lstool get' failed: 1" during Certificate Replacement on vCenter Server 6.x
"ERROR certificate-manager 'lstool get-site-id' failed: 1", Certificate Replacement with Custom Certificate Fails on vCenter Server 6.x
Using the 'lsdoctor' Tool
Determining expired SSL certificates in vCenter Server and ESXi 6.x and 7.0.x
VMware Docs: vSphere Security Certificates

Use Custom Certificates with vSphere
"503 Service Unavailable" error on the vSphere Web Client when logging in or accessing the vCenter Server

Impact/Risks:
  • If there are issues with the certificates being replaced, the vCenter Server may stop working.
  • The VMDIR LDAP directory may also fail to update properly, so it may need to be repaired, see Using the 'lsdoctor' Tool
  • If there are expired certificates in trusted roots that are not in use, that will trigger a Certificate status alarm.
  • If there are expired Certificates in the BACKUP_STORES that will trigger a Certificate status alarm.
  • If there are expired certificates such as STS, Machine SSL or any Solution Users, the vCenter will not be able to start services due to the Dependencies for the Services.