The vCenter proxy service fails to start on Cloud Director cells while Cloud Proxy cells are active
search cancel

The vCenter proxy service fails to start on Cloud Director cells while Cloud Proxy cells are active

book

Article ID: 320531

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

Symptoms:
  • In System > Manage & Monitor > Cloud Cells of the vCloud Director UI, you may see the following message:
None of the cells have a vCenter proxy service running.
  • In System > Manage & Monitor > vCenters of the vCloud Director UI one or more of the resource vCenters show as disconnected.
  • Attempting to reconnect to a disconnected vCenter may fail with the following error:
Unable to communicate with the cell to start the listener.
  • In /opt/vmware/vcloud-director/logs/vcloud-container-debug.log on the vCloud Director cells, you may see the following messages:
2018-03-12 15:26:38,099 | DEBUG    | CoordinatorJob            | VimProxyFailoverService        | Found 1 VC proxies that need to be started on 5 available candidate cells. |
2018-03-12 15:26:38,099 | DEBUG    | CoordinatorJob            | VimProxyFailoverService        | Generated 1 recommendations for VC execution |
2018-03-12 15:26:38,099 | DEBUG    | CoordinatorJob            | VimProxyFailoverService        | Will execute 1 recommendations in this coordinator run. |
2018-03-12 15:26:38,099 | INFO     | CoordinatorJob            | VimProxyFailoverService        | Starting VC Proxy for VC <VC_UUID> on cell <CELL_NAME>|
2018-03-12 15:26:38,100 | DEBUG    | CoordinatorJob            | FailoverMessageServiceImpl     | Published JMS message ID:vcd.example.com-40916-1520593351795-4:2:4:1:16287 |
2018-03-12 15:26:38,100 | DEBUG    | ActiveMQ Session Task-830 | FailoverMessageServiceImpl     | Received JMS message ID:vcd.example.com-40916-1520593351795-4:2:4:1:16287 |
2018-03-12 15:26:38,100 | DEBUG    | CoordinatorJob            | FailoverMessageServiceImpl     | Published request [id = failoverMessage-cid:<CID>-ciid:3141-mid:16287, type = Request, src instance = 141 (id = <INSTANCE_UUID>), target instance = <CELL_NAME>, timestamp = Mon Mar 12 15:26:38 CET 2018] (JMS message ID {1}). |
2018-03-12 15:26:38,100 | DEBUG    | ActiveMQ Session Task-830 | FailoverMessageServiceImpl     | Received message [id = failoverMessage-cid:<CID>-ciid:3141-mid:16287, type = Request, src instance = 141 (id = <INSTANCE_UUID>), target instance = <CELL_NAME>, timestamp = Mon Mar 12 15:26:38 CET 2018] on cell instance 141 |

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware Cloud Director for Service Provider 9.x
VMware Cloud Director for Service Provider 8.x

Cause

  • This issue occurs because the Cloud Proxy cells are being considered as viable cells that the vCenter Proxy can be run on.
  • This can occur when existing vCloud Director cells are cloned and reconfigured to act as a Cloud Proxy.
  • This can occur if the vCloud Director services were started on a Cloud Proxy before its /opt/vmware/vcloud-director/etc/global.properties file was modified to make the cell a dedicated Cloud Proxy.

Resolution

To resolve this issue, problematic Cloud Proxies should be removed from the vCloud Director instance and new Cloud Proxies should be created on new VMs and not cloned from existing vCloud Director cells.

Note: The vCloud Director services should only be started after the cell has been configured as a dedicated Cloud Proxy.

To remove an existing Cloud Proxy from a vCloud Director instance, perform the steps below.
  1. SSH as root to the Cloud Proxy cell.
  2. Stop the vCloud Director services on the Cloud Proxy cell:
service vmware-vcd stop
  1. In the vCloud Director UI navigate to System > Manage & Monitor > Cloud Cells.
  2. Right-click on the stopped Cloud Proxy cell name and click Delete.


Workaround:
To work around this issue, perform the following steps:
  1. SSH to each of the Cloud Proxies.
  2. Stop the vCloud Director services on the Cloud Proxies with the following command:
service vmware-vcd stop
  1. In the vCloud Director GUI navigate to System > Manage & Monitor > vCenters.
  2. Right-click on the vCenter Server that is showing as disconnected and click Reconnect.
  3. After the vCenter has been reconnected start the vCloud Director services on the Cloud Proxies with the following command:
service vmware-vcd start


Powered by Wolken Software