When the "Enable UAA as OIDC provider" checkbox is enabled in Enterprise PKS, the vRealize Operations Solutions Adapter for Containers Monitoring management pack cannot discover and retrieve information from Kubernetes clusters
search cancel

When the "Enable UAA as OIDC provider" checkbox is enabled in Enterprise PKS, the vRealize Operations Solutions Adapter for Containers Monitoring management pack cannot discover and retrieve information from Kubernetes clusters

book

Article ID: 316835

calendar_today

Updated On:

Products

VMware

Issue/Introduction

Symptoms:
  • When the "Enable UAA as OIDC provider" checkbox is enabled in Enterprise PKS, the vRealize Operations Solutions Adapter for Containers Monitoring management pack cannot discover and retrieve information from Kubernetes clusters


Environment

VMware PKS 1.x

Resolution

This is a know issue affecting Enterprise PKS. There is currently no resolution.

Workaround:
Implement the following steps to workaround this issue:

Notes:  
​​​​​​​This workaround does not provide a means for automatic discovery and requires manually adding each existing or newly created Kubernetes cluster.  If you delete a Kubernetes cluster, the VI Admin will need to manually delete the respective adapter configuration from vRealize Operations Manager.”
 
  1. Using a text-editor, create a new file. Paste the following content and modify the ServiceAccount references with a unique value.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: vrealize-k8s01
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: vrealize-k8s01
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: vrealize-k8s01
    namespace: kube-system

Save and close the file (named rbac-svcacct-vrealize.yaml for this example).

  1. Issue a command similar to the following to create the RBAC policy (creates a service account named “vrealize-k8s01” and a corresponding secret):
kubectl apply -f rbac-svcacct-vrealize.yaml -n kube-system
  1. Issue a command similar to the following to collect the token name for the vrealize service account

kubectl describe sa vrealize-k8s01 -n kube-system

Note: You will see output similar to the following:

Name:                vrealize-k8s01
Namespace:           kube-system
Labels:              <none>
Annotations:         kubectl.kubernetes.io/last-applied-configuration:
                       {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"vrealize                                                                                -k8s01","namespace":"kube-system"}}
Image pull secrets:  <none>
Mountable secrets:   vrealize-k8s01-token-r94k4
Tokens:              vrealize-k8s01-token-r94k4
Events:              <none>

 

  1. Issue a command similar to the following to collect the secret token value:

kubectl describe secret vrealize-k8s01-token-r94k4 -n kube-system

Note: You will see output similar to the following:

Name:                vrealize-k8s01
Namespace:           kube-system
Labels:              <none>
Annotations:         kubectl.kubernetes.io/last-applied-configuration:
                       {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"name":"vrealize                                                                                -k8s01","namespace":"kube-system"}}
Image pull secrets:  <none>
Mountable secrets:   vrealize-k8s01-token-r94k4
Tokens:              vrealize-k8s01-token-r94k4
Events:              <none>
ubuntu@cli-vm:~$ kubectl -n kube-system describe secrets vrealize-k8s01-token-r94k4
Name:         vrealize-k8s01-token-r94k4
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: vrealize-k8s01
              kubernetes.io/service-account.uid: d3e78716-f027-11e9-8352-005056a54672

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1094 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJ2cmVhbGl6ZS1rOHMwMS10b2tlbi1yOTRrNCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJ2cmVhbGl6ZS1rOHMwMSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImQzZTc4NzE2LWYwMjctMTFlOS04MzUyLTAwNTA1NmE1NDY3MiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTp2cmVhbGl6ZS1rOHMwMSJ9.m1TCOU9L7s36GX7LvjPsNsXausErHtgVNQJLEg1EtyMIYLp6nyVFJTmH6aA-Fk8qpLV98zuiqXeHYISp6g_cFZEd50_uCmIWtEznSnXBAfeJ5YpnOL3thTPYqTmpe_s5n2gyy3wYqc0P3CCl2YNn-cVTxyCoUuLtiEpvIEpExwe2k8Jw6TW8jjmiFJxDpv3yS4ZBJL2IW0Elpue9JiCkigNaxxipuDuEjG0oRrKeTrs4grPMTCOTtTbDaR9LdC8d9q14eSIoWlRAOCpPLzlh43veKjS57amI5o4PCa2AGEIxJVRuQhMgQLa6AFMd3B1EZaCbCOu9NfQDdkMTC2V3mw

 

  1. Open a web browser and direct it to the vRealize Operations Manager web UI; login with Admin credentials; navigate to Administration > Solutions; select the + icon over the upper table.
  2. When the pop-up window appears, select BROWSE; select the downloaded .pak file; select both checkboxes; select UPLOAD; select NEXT; select the checkbox and NEXT; select FINISH.
  3. In the upper table, highlight the row with the Kubernetes icon then select the gears icon above the table.
  4. Configure the Instance Settings similar to the following:
    1. Display Name: K8S01
    2. Master URL: https://CLUSTER_API_FQDN:8443 
Note: Run kubectl cluster-info to collect the URL for this step.
  1. cAdvisor Service: DaemonSet
  2. cAdvisor Port (DaemonSet): 31194
  3. For the Credential, select +; select Token Auth from the Credential Kind drop-down; enter vrealize-k8s01 as the Credential name; paste the TOKEN_VALUE collected in Step 4 in the Bearer Token field; select OK.
  4. Expand Advanced Settings and complete the PKS Cluster ID and vCenter Server parameter.
Note: Collect the cluster-specific PKS Cluster ID by running the pks clusters command.
  1. Select TEST CONNECTION; select ACCEPT; and OK for any other pop-up notices; select SAVE SETTINGS and YES to the pop-up notice; select OK; select CLOSE.
Note: if the adapter won’t connect to the FQDN, replace the FQDN with the IP of the Kubernetes API virtual server.
  1. Navigate to Dashboards > Kubernetes Overview.
  2. Wait 30 minutes for the initial data collection and the dashboard panels to fully populate then refresh the page.
  3. Repeat the previous steps for each existing and newly created Kubernetes cluster.