Kubernetes Policies in VCD 10.2 with vCenter 7.0 and later, Tanzu are non-functional
search cancel

Kubernetes Policies in VCD 10.2 with vCenter 7.0 and later, Tanzu are non-functional

book

Article ID: 321463

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

Symptoms:
Cloud Director unable to reach vSphere for Kubernetes, you see message similar to:

Could not connect to vSphere for Kubernetes infrastructure

Environment

VMware Cloud Director 10.x

Cause

This issues is caused due to the certificate structure of Tanzu Kubernetes in vCenter, The certificate(s) of the Supervisor Cluster is not automatically trusted by VCD. Calls made to the Supervisor Cluster by VCD fail due to SSL errors.

Resolution

To resolve this issue:

Note: This currently applies to VMware Cloud Director 10.2 with vCenter 7.0 and later environments with vSphere with Tanzu feature enabled.

The Supervisor Cluster certificates can be manually trusted and accepted by VCD.
Follow the below steps to manually set CA certificates to trusted are:
  1. Login to the vCenter UI.
  2. Navigate to Menu > Developer Center > API Explorer > GET /api/vcenter/namespace-management/clusters/{cluster}
  3. In the value for the cluster parameter, input the moref of the vCenter Cluster containing the Kubernetes Supervisor Cluster. Note: The moref is located in the URL of the H5 UI when clicking on the cluster. It is in the format: domain-xx
  4. Click execute to execute the API:
Copy the curl command displayed in the VC UI into a linux shell:
Example:

curl -X GET 'https://<URL>/api/vcenter/namespace-management/clusters/domain-c8' -H 'vmware-api-session-id: <API_SESSION_UUID>'
    
Copy and amend this curl command by adding "|jq '.tls_management_endpoint_certificate'|sed 's/\\n/\n/g'" to a linux shell.
Optionally, you may add the curl option "--insecure" if vCenter's endpoint is associated with a self-signed certificate (non production use case).
           
          Output:  
curl -X GET 'https://<URL>/api/vcenter/namespace-management/clusters/domain-c8' -H 'vmware-api-session-id: <API_SESSION_UUID>' | jq '.tls_management_endpoint_certificate'|sed 's/\\n/\n/g'
  
 Executing this command should output something like:
   
   "-----BEGIN CERTIFICATE-----
MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTIxMDMxNzA1NTUxN1oXDTMxMDMxNTA1NTUxN1owFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALKf
xtn8isJaZiYGGPw+2XU4rrNeZe6OeUPFJSHv7AedoEi5ZuVNJCxKZJkWskCwbgED
yfoNu0gu1ReVhrHmzz9K52+fn0rUFaR6K7sTTVZRsVgo5Lhfs7kIYXbEqY+AINjS
N4YecFtzMzrDknhiNWKMox43d9jNSweyGoD1CwaQjD3LAWxqzdAAXXE3WvN7svqn
2oHvn3x8KOce2X7LsIBe478rVq66oZlaICl9ihKg7n4vGQnBWvCcIMsia1HYBclG
fACtCi6gZCEPFTORb0m6N7S6CQzJlzKWeiZzy7FEQlda+oZri9Rm/DsQ05TRMgYs
pBNQ7SJamZrjMbXN65cCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wHQYDVR0OBBYEFET3ffQTvJTxRjUbwK+9H/Hn0wIKMA0GCSqGSIb3
DQEBCwUAA4IBAQAWOL+hZfO2EgIqTOMZRaytTQNhHpFOREqHWEPgyTokXXk861HD
wv/9Z4TvWR9sSUAWy5bXQuXNW3kbQMsgmfZBe2jITEPtKkplEoxh10Ma9FZ+ulpY
8H5/Rzny5OSZIJiBOJe4NhZ3kx5ImaN1/FlvaJeqorx97CKF7TUr4yFkuEyNxg1o
60xyvPVClpdGA7StcpUQw3Cv28JI3oJPXrQESwIlKvyvraefC+fDFade9L8++VJ8
lT07k2b0EVu8uynnhgEXMZcGWWgB9q76qg+0ORGVMrquq9hFe7zC3iHfszruDNFP
5ByXjSEd5ZBC9JkSfSjGbP8TVtLQ9Sal3Bzb
-----END CERTIFICATE-----
"

     Note: You may need to install the following package in your linux distribution: "curl", "sed", "jq" to execute the above command.
  1. Copy this output content, without the double-quotes into a file with suffix ".pem".
  2. Open a browser to vCD provider UI, and navigate to tab "Administration", and select menu "Trusted Certificates"
  3. Click on "IMPORT", then click on "SELECT FILE", and input the file created at step 5 above.
  4. Verify the details displayed and click the "IMPORT" button.
Note: Optionally, If you're using vCenter Server version before 7u1c you'll also need to perform the following additional steps to trust the workload management certificate.
To do so, repeat steps 4 to 6 but adjust the curl command as follow, example:
curl -X GET 'https://<URL>/api/vcenter/namespace-management/clusters/domain-c8' -H 'vmware-api-session-id: <API_SESSION_UUID>' | jq '.tls_endpoint_certificate'|sed 's/\\n/\n/g'
(as you can see this replaces the string "tls_management_endpoint_certificate" with "tls_endpoint_certificate")
  1. In the VCD provider UI, navigate to the Tanzu Kubernetes vCenter.
  2. Go to Infrastructure Resources > vCenter Server Instances > (specific vCenter). Click on the vCenter Server.
  3. Click "REFRESH" to refresh the vCenter Server.
  4. Wait a short period of time (about a minute) and confirm that the policy was generated (or refreshed) by navigating to vCD provider UI "Monitor > Events".
Note: Observe an audit event such as:
  • "Provider VDC Kubernetes Policy '[name-of-PVDC]-[name-of-host-cluster]-KubernetesPolicy' (<POLICY_UUID>) modification"
  • Then navigate to "Provider VDCs", select your PVDC, then select "Policies / Kubernetes", and confirm that the corresponding policy was auto generated.


Powered by Wolken Software