Enabling more than 20 WCP clusters in a VMware Cloud Foundation environment
search cancel

Enabling more than 20 WCP clusters in a VMware Cloud Foundation environment

book

Article ID: 324046

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

Symptoms:
  • For every WCP supervisor cluster, the NSX Container Plugin creates one Load Balancer of medium size in NSXT. NSXT should have enough resources to accommodate new requests to successfully enable workload platform.
  • With the Single Edge cluster, the VMware Cloud Foundation platform can accommodate only 8 edge nodes for eBGP deployment and 10 edge nodes for static route deployment.[Ref: Below note for calculation of LB]
To determine the maximum number of load balancers per Edge Cluster, multiply the maximum number of load balancers for the Edge Node type by the number of Edge Nodes and divide by 2. For example, with 10 Large VM Edge Nodes in an Edge Cluster, you can have up to 200 small load balancer instances (40 x 10/2), or up to 20 medium LB instances (4 x 10/2). See Scaling Load Balancer Resources in the NSX-T Data Center documentation for more information.

With the above calculation:
  • 16 – Max number of WCP cluster enabled with BGP Route Deployment under single edge cluster 
  • 20 – Max number of WCP cluster enabled with Static Route Deployment under single edge cluster
  • When trying to create additional clusters, the cluster will become stuck in a configuring state due to the Kubernetes NCP pod crashing with below error:
Insufficient resources to do auto allocation in edge cluster EdgeCluster/d4fbdc89-1db9-4c18-a475-db08b9ee00e4 of pool type LB_ALLOCATION_POOL because of below reason(s).: [Routing] Node(s) [TransportNode/8aca57c5-e9fb-4acf-8beb-3b76c1c5351d, TransportNode/85613a38-5933-4b6b-9417-eeb91e667759] don't have enough capacity to allocate.


Environment

VMware Cloud Foundation 4.x

Cause

  • Insufficient NSX resources for creating the Load Balancer.
  • The VMware Cloud Foundation design recommendation is only one Edge cluster per Workload Domain.
  • Multiple Tier-0 router on same Transport Zone is not supported by design.

    Note: The "multiple Tier-0 router on same Transport Zone is not supported by design" is only applicable for versions below VMware Cloud Foundation 4.1.

Resolution

Currently there is no resolution, to workaround this be see the section below.

Workaround:
  1. Add a new Edge cluster from the VMware SDDC Manager UI.
  2. Delete the newly created Tier-0 router associated with the Edge cluster.
  3. Start using the newly created the edge cluster for WCP enablement.


Powered by Wolken Software