Hosts file entries are removed when kubectl -w passes NULL or empty parameters to the sed command preventing kube-api-server endpoint startup
search cancel

Hosts file entries are removed when kubectl -w passes NULL or empty parameters to the sed command preventing kube-api-server endpoint startup

book

Article ID: 318813

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

Symptoms:
  • When attempting to access https://vRAFQDN/csp/gateway/portal a 404 error is seen
  • Log entries within the kubelet.service similar to the below are found:
Oct 20 12:13:08 <applianceFQDN> kubelet[8634]: F1020 12:13:08.726641 8634 docker_service.go:412] Streaming server stopped unexpectedly: listen tcp: lookup localhost on <localIP>: no such host
  • /etc/hosts file appears malformed or empty of the localhost and/or FQDN entries for the appliance.
  • kubectl get nodes generates a NotReady status for any impacted host in the cluster.


Environment

VMware vRealize Automation 8.1.x
VMware vRealize Automation 8.2.x
VMware vRealize Automation 8.0.x

Cause

A Kubernetes pod named update-etc-hosts, under the kube-system namespace, updates the pod name to IP mapping in /etc/hosts every time a new resolvable pod is discovered.  The updates are performed through a watch implemented with kubectl -w.  When this processes an error condition, kubectl -w pipes empty parameters to the 'sed' function removing values from within the /etc/hosts file.

Resolution

This issue is resolved in vRealize Automation 8.2 P1.

Workaround:
  1. Copy the /etc/hosts file entries for a functioning node in the cluster and update the FQDN values for each respective node.
Note:  See the Related Information section for a sample of a /etc/hosts file.
  1. Run 
    systemctl start kubelet
  2. Follow instructions to Start vRealize Automation.​​​​


Additional Information

# Begin /etc/hosts (network card version)

# End /etc/hosts (network card version)
# VAMI_EDIT_BEGIN
# Generated by Studio VAMI service. Do not modify manually.
127.0.0.1  ApplianceFQDN ApplianceShortName localhost
::1  ApplianceFQDN ApplianceShortName localhost ipv6-localhost ipv6-loopback
# VAMI_EDIT_END
127.0.0.1 vra-k8s.local
### k8s service hostnames follow. *.svc.cluster.local are automatically updated ###

Powered by Wolken Software