Accessing the VAMI returns error "Update installation in progress" after recovering from a failed update
search cancel

Accessing the VAMI returns error "Update installation in progress" after recovering from a failed update

book

Article ID: 320702

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • After logging in to the VAMI UI after recovering from a failed update, instead of the normal Interface, "Update installation in progress" is shown:
  • the log of application management service (applmgmt.log) has the following entries:
    2019-02-18T13:48:29.255 [10787]INFO:vmware.appliance.vapi.auth:Authorization request for service_id: com.vmware.cis.session, operation_id: create
    2019-02-18T13:48:29.259 [10787]DEBUG:vmware.vherd.transport.vapi:Only ['INSTALL_IN_PROGRESS'] block []
    2019-02-18T13:48:29.259 [10787]INFO:vmware.appliance.update.update_state:Found operation in progress /storage/core/software-update/stage_install_operation
    2019-02-18T13:48:29.260 [10787]DEBUG:vmware.vherd.transport.vapi:State 'INSTALL_IN_PROGRESS'
    2019-02-18T13:48:29.263 [10787]INFO:twisted:"127.0.0.1" - - [18/Feb/2019:13:48:28 +0000] "POST /rest/com/vmware/cis/session HTTP/1.0" 200 60 "https://<FQDN>:5480/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0"
    2019-02-18T13:48:29.267 [10787]INFO:vmware.appliance.vapi.auth:Reset pam tally for root
    2019-02-18T13:48:29.267 [10787]DEBUG:vmware.appliance.vapi.auth:stdout: b'Login           Failures Latest failure     From\nroot                0    \n'
    2019-02-18T13:48:29.277 [10787]INFO:vmware.appliance.vapi.auth:Authorization request for service_id: com.vmware.appliance.local_accounts, operation_id: get
    2019-02-18T13:48:29.277 [10787]DEBUG:vmware.vherd.base.authorization_local:Verify privileges user (root) privilege ['ViewConfiguration']
    2019-02-18T13:48:29.277 [10787]DEBUG:root:Validated user privileges in localstore or SSO
    2019-02-18T13:48:29.277 [10787]DEBUG:vmware.vherd.transport.vapi:Only [] block []
    2019-02-18T13:48:29.277 [10787]INFO:vmware.appliance.update.update_state:Found operation in progress /storage/core/software-update/stage_install_operation
    2019-02-18T13:48:29.278 [10787]DEBUG:vmware.vherd.transport.vapi:State 'INSTALL_IN_PROGRESS'
    2019-02-18T13:48:29.279 [10787]INFO:twisted:"127.0.0.1" - - [18/Feb/2019:13:48:28 +0000] "GET /rest/appliance/local-accounts/root HTTP/1.0" 400 136 "https://<FQDN>:5480/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0"
    

 

  • The VAMI log (vami.log) contains the following messages: 
    <response><locale>en-US</locale><requestid>getUpgradeStatus</requestid><value actions="enabled" id="status">firstboot</value><value actions="enabled" id="pr
    ogress">100</value><value actions="enabled" id="details">{
        "error": null,
        "info": [],
        "warning": [],
        "start_time": "2019-11-22T14:04:15.508Z",
        "progress": 100,
        "end_time": "2019-11-22T14:05:15.369Z",
        "progress_message": {
            "args": [
                "VMware Platform Services Controller Health Monitor"
            ],
            "id": "install.ciscommon.component.starting",
            "localized": "Starting VMware Platform Services Controller Health Monitor...",
            "translatable": "Starting %(0)s...


Environment

VMware vCenter Server Appliance 6.7.x
VMware vCenter Server 7.0.x

Cause

The issue is caused post the appliance has recovered from failed update and the update state is "INSTALL_IN_PROGRESS" in "/etc/applmgmt/appliance/software_update_state.conf", as in the example below:
{
    "operation_id": "/storage/core/software-update/stage_install_operation",
    "latest_query_time": "2019-02-13T14:53:00Z",
    "state": "INSTALL_IN_PROGRESS",
    "version": "6.7.0.21000"
}
 
 

Resolution

Note: This is applicable only for the source vCenter which is restored/powered on after an upgrade failure. Do not apply these steps to recover/workaround the upgrade failure on the failed target vCenter

In order to resolve the issue perform the following:
  1. Login to the appliance using ssh
  2. Create a backup copy of the file /etc/applmgmt/appliance/software_update_state.conf in a different location, e.g. in /storage/core:
    # cp /etc/applmgmt/appliance/software_update_state.conf /storage/core/software_update_state.conf
  3. Stop the VAMI service by running the command:
    # service-control --stop applmgmt
  4. Remove the file /etc/applmgmt/appliance/software_update_state.conf :
    # rm -i /etc/applmgmt/appliance/software_update_state.conf
  5. change into the folder /var/log/vmware/upgrade :
    # cd /var/log/vmware/upgrade
  6. verify that the following 3 files exist in this folder:
    prechecks-succeeded
    export-succeeded
    import-succeeded
  7. If these files do not exist, run the following 3 commands to create them:
    # touch prechecks-succeeded
    # touch export-succeeded
    # touch import-succeeded
  8. confirm that the files have been created:
    # ls -l   
    
    -rw-r--r-- 1 root root    0 Mar 11  2019 prechecks-succeeded
    -rw-r--r-- 1 root root    0 Mar 11  2019 import-succeeded
    -rw-r--r-- 1 root root    0 Mar 11  2019 export-succeeded
  9. Start the VAMI service using the following command:
     # service-control --start applmgmt
  10. Login to the VAMI to validate the access
 
If this article didn’t solve your issue:
Ask for help here : Click for VMware Communities


Additional Information