Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Performing a side-by-side upgrade of a VMware vRealize Operations 5.x Appliance to vRealize Operations Manager 6.0.x (2031543)

Purpose

This article provides steps to deploy a new instance of the vRealize Operations Manager vApp (formerly known as vCenter Operations Manager) (vApp), to upgrade or replace an existing instance.

Resolution

Generally speaking, the latest version is deployed, so two instances of vRealize Operations Manager (formerly known as vCenter Operations Manager) exist at the same time and data is migrated from the previous version.

This article uses these terms to describe the vApps involved:
  • New vApp: The vApp that is the target of the upgrade
  • Source vApp: The vApp that is the source of the upgrade and from which data is exported
For more information on storage/CPU/IP prerequisites, see the vApp Deployment and Configuration Guide . For storage, you will require enough space in each virtual machine in the new vApp to hold 110% of the data partition size of the corresponding source virtual machine.

This procedure consists of two stages:
  • Deploy the OVF and transfer data
  • Configure the appliance through the admin web interface

To deploy the OVF and transfer data:

  1. Deploy the latest vRealize Operations Manager vApp (formerly known as vCenter Operations Manager) (vApp) OVF file. This is the new vApp which is used in this procedure.

    Note: During initial configuration choose new IP addresses for the UI and Analytics virtual machines. You can change these later. 

  2. Power on the vRealize Operations Manager vApp (the new vApp) for the first time.

    Note: Do not perform the initial configuration of the application from the admin web interface. Configuration is handled later in this procedure.

  3. Obtain the /data partition size for each virtual machine in the source vApp by running this command on each virtual machine:

    df –k /data

  4. Add additional disk(s) to the virtual machines in the new vApp to ensure that you have at least 110% of the source data storage size available. Be aware that UI virtual machine comes with 120 GB and the Analytics virtual machine comes with 200 GB data disk sizes by default.
  5. Log in to the UI virtual machine on the new vApp console or ssh as admin/admin.
  6. Validate the networking of the UI virtual machine:

    1. Ping the IP addresses of the source UI and Analytics virtual machines.
    2. Ping the IP addresses and host names of the vCenter Servers that will be added to this vApp.

  7. Run this command to transfer all the data, specifying the details of one vCenter Server (preferably the one that holds the license; you will configure the remaining vCenter Servers later) in the command:

    vcops-admin upgrade50 --vc-name vc_name --vc-server vc_server --user vc_username --password vc_password [--collection-user collection_user] [--collection-password collection_password] --uiHost UI_host --uiRootPassword UI_password --analyticsHost analytics_host --analyticsRootPassword analytics_password

    Where:
    • vc_name is the friendly name of one of vCenter Server configured in the source vApp
    • vc_server is the vCenter Server's IP address or hostname related to the vc_name
    • vc_username and vc_password is the vCenter Server username and password with which they are registered
    • UI_host is the source UI virtual machine IP address
    • analytics_host is the source Analytics virtual machine IP address


       Note:

      • You can optionally specify a new collection user and collection password.
      • All special characters for the passwords and usernames needs to be escaped with an \.

        For example:

        --user administrator\@vsphere.local
        --password VMware1010\!


  8. If you see a prompt to trust the certificate of vCenter Server, answer Yes.
  9. The upgrade is complete and one vCenter Server is now registered. If the operation fails, you can check the /var/log/vmware/vcops-admin.cmd.log file for the error details or contact VMware Support. If the operation succeeds, continue to the next step.
  10. Restart the Admin user interface service on the new vApp by running this command, which reloads the newly trusted certificate:

    sudo service vcopsadmin restart
At this point, this content is copied from the source vApp to the new vApp:
  • Data in the CapacityIQ database
  • Data in the Alive database. This includes data for all adapters (both vCenter Server and non-vCenter Server adapters).
  • Data in FSDB.
  • Configuration for CapacityIQ.
  • Configuration for the vCenter Server adapter only. Configuration for non-vCenter Server adapters is not copied.

Note: Do not re-run the preceding commands using the other registered vCenter Servers. This is covered in the next section.


To configure the appliance via the admin web interface:

  1. Navigate to the admin interface at:

    https://new_UI_virtualmachine_IP_or_hostname/admin

  2. Log in as admin/admin.

    Note: You will not be asked to change the passwords for admin and root.

  3. On the Registration tab of admin user interface, reconnect with each vCenter Server by clicking Update for each of them.

    Note: Do not click Update for the vCenter Server specified in the upgrade command in the previous procedure.

  4. Ensure that you accept the certificate for every vCenter Server when you are prompted.

    Note: If the environment is large, the time between each update can take 20 minutes because Analytics takes 20 minutes to initialize RMI on large setups. RMI must be functioning when updating registration later.

  5. Change the root and admin passwords.

This completes the import of all data from the source vApp to the new vApp, and all the original vCenter Servers are registered and connected to the new vApp.

Unless the configuration for vRealize Configuration Manager (formerly known as vCenter Configuration Manager) and vRealize Infrastructure Navigator (formerly known as vCenter Infrastructure Navigator) adapters were customized in the source vApp, they are not required to be copied to the new vApp.

If the source vApp had adapters for third-party software, you must copy the configuration data to the new vApp manually. This may require restarting services after the copy.

Log in to the /vsphere or /custom web page to ensure all data has been migrated. If required, you can change the IP addresses of the new vApp to match the original configuration.

Additional Information

When you upgrade from vRealize Operations Manager (formerly known as vCenter Operations Manager) 5.6 to a newer 5.6 or 5.7 version, you must manually copy these files from the old UI and Analytics VM to the new UI and Analytics VMs:

UI VM:

$ALIVE_BASE/tomcat/webapps/vcops-vsphere/WEB-INF/classes/View_CustomViews.properties
$ALIVE_BASE/tomcat/webapps/vcops-vsphere/WEB-INF/classes/Metric_CustomViews.properties
$ALIVE_BASE/user/conf/analytics/user_ciq_metrics_collected.xml
$ALIVE_BASE/user/conf/analytics/user_group_metrics.xml

Analytics VM:

$ALIVE_BASE/user/conf/analytics/user_ciq_metrics_collected.xml
$ALIVE_BASE/user/conf/analytics/user_group_metrics.xml

In newer versions of vRealize Operations Manager (formerly known as vCenter Operations Manager) (5.8 or greater with the SLES SP2 pack applied), the side by side upgrade might fail with this error seen in the /var/log/vmware/vcops-admin.cmd.log file on the destination UI VM:

<YYYY-MM-DD> <TIME>,832 ERROR [main] com.integrien.alive.vm.manager.SSHClient.executeCommand - Command exited with a non zero exit status: 1
<YYYY-MM-DD> <TIME>,833 ERROR [main] com.integrien.alive.vm.manager.UpgradeManagerImpl.handleUpgradeExceptionNoRethrow - Upgrade validation failed
com.integrien.alive.vm.manager.exception.DatabaseVerificationFailedException: Database state verification failed for vCenter Operations Manager 5.0 Analytics VM at analyticsvm.vcloud.local. R
efer to VMware KB 2010144 for fixing this problem.
at com.integrien.alive.vm.manager.UpgradeManagerImpl.verifyDatabaseState(UpgradeManagerImpl.java:220)
at com.integrien.alive.vm.manager.UpgradeManagerImpl.validateUpgradeInfo(UpgradeManagerImpl.java:285)
at com.integrien.alive.vm.manager.UpgradeManagerImpl.validateInput(UpgradeManagerImpl.java:993)
at com.integrien.alive.vm.manager.UpgradeManagerImpl.upgrade(UpgradeManagerImpl.java:327)
at com.integrien.alive.ui.action.admin.AdminWrapper.upgrade(AdminWrapper.java:750)
at com.integrien.alive.ui.action.admin.AdminWrapper.main(AdminWrapper.java:1742)
Caused by: com.integrien.alive.vm.manager.exception.SSHException: Failed to execute remote command. com.jcraft.jsch.JSchException: channel is not opened.
at com.integrien.alive.vm.manager.SSHClient.executeCommand(SSHClient.java:174)
at com.integrien.alive.vm.manager.SSHClient.executeCommand(SSHClient.java:136)
at com.integrien.alive.vm.manager.UpgradeManagerImpl.verifyDatabaseState(UpgradeManagerImpl.java:198)
... 5 more
Caused by: com.jcraft.jsch.JSchException: channel is not opened.
at com.jcraft.jsch.Channel.connect(Channel.java:197)
at com.jcraft.jsch.Channel.connect(Channel.java:144)
at com.integrien.alive.vm.manager.SSHClient.executeCommand(SSHClient.java:151)
... 7 more


You see an error similar to this in the /var/log/messages file on the source Analytics VM:

<YYYY-MM-DD> <TIME>-06:00 analyticsvm sshd[6445]: error: no more sessions

This is due to a hardening of sshd included in the SLES SP2 update. The line MaxSessions=1 is added to the /etc/ssh/sshd_config file. During the side by side upgrade, more than one session per network connections is established. To work around this, comment out this line and restart the sshd daemon (service sshd restart).

Note: These are few changes that you can make to the /etc/ssh/sshd_config file on all four virtual machines involved to prevent timeouts from interrupting the upgrade:
  • Uncomment out the TCPKeepAlive yes line
  • Uncomment out the ClientAliveInterval 0 line and change 0 to 30
  • Uncomment out the ClientAliveCountMax 3 line and change 3 to 99999
After completing this, restart sshd on all nodes using the service sshd restart command as the root user.

Also, it may be necessary to disable the Dynamic Threshold (DT) on the source and destination Analytics VM so as not to introduce additional load while the upgrade is running. The DT processing runs every night at 1:00AM by default and is incredibly resource intensive with the potential to disrupt the side-by-side migration process. For additional information on steps on disabling Dynamic Threshold (DT), see Changing the calculation time or disabling the dynamic threshold calculation in VMware vRealize Operations Manager 5.x (2040008).

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 5 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 5 Ratings
Actions
KB: