PKS Container creation fails due to HyperBus interface vmk50 not being present on ESXI. PKS 1.2 NSX-T 2.1
search cancel

PKS Container creation fails due to HyperBus interface vmk50 not being present on ESXI. PKS 1.2 NSX-T 2.1

book

Article ID: 330616

calendar_today

Updated On:

Products

VMware

Issue/Introduction

Purpose is to overcome container creation failure due to HyperBus interface vmk50 not being present on ESXI. 

Symptoms:
Container creation fails. 

Log snippet for reference.

2018-12-05T20:28:46.137Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11471 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] oslo.
privsep.daemon privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/CAP_
DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/none
1 2018-12-05T20:28:46.137Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11471 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] oslo.privsep.daemon privsep daemon running as pid 11471
1 2018-12-05T20:28:46.256Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Created veth pair nsx_agent_outer, nsx_agent_inner
1 2018-12-05T20:28:46.465Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Configured interface nsx_agent_inner with IP 169.254.1.0/31, MAC 00:50:56:a5:87:76
1 2018-12-05T20:28:46.480Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Added OVS port nsx_agent_outer with vlan 4094
1 2018-12-05T20:28:46.520Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.nsxrpc_client Listening NSX RPC connection...
1 2018-12-05T20:28:46.521Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] cli.server.container_cli_server Starting node_agent CLI server
1 2018-12-05T20:28:46.532Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher CNI socket is listening...
1 2018-12-05T20:39:05.068Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be configured
1 2018-12-05T20:39:06.073Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher Failed to find OVS port for container 3497dd0a78a264bf97e4de52a88ce6e57158db92b92107f0abcdfce975c1e407: __init__() takes exactly 1 argument (2 given)
1 2018-12-05T20:44:05.328Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be configured
1 2018-12-05T20:44:06.331Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher Failed to find OVS port for container f2226c12e3786a35e180f669159e1c48dbf44981c7c558b78b8f25da2e8dd74f: __init__() takes exactly 1 argument (2 given)
1 2018-12-05T20:49:05.606Z 37f52426-7afa-4226-b109-60293ad8ce19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be conf

Environment

VMware PKS 1.x

Cause

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.

Resolution

Upgrade to NSX-T 2.3 and post that removed and added back the Fabric nodes or use the below workaround.

Issue 1998217: HyperBus interface vmk50 might be missing on vSphere ESXi causing container creation failure

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.
 

Workaround: Complete the following steps.

  1. Retrieve the vmk50 port ID using CLI on vSphere ESXi
    net-dvs | grep vmk50 -C 10
  2. Create the vmk50 interface on vSphere ESXi.
    esxcli network ip interface add -P <port-id from step-1> -s DvsPortset-0 -i vmk50 -N hyperbus
  3. Assign an IP address to the vmk50 interface.
    esxcfg-vmknic -i 169.254.1.1 -n 255.255.0.0 -s DvsPortset-0 -v <port-id from step-1> -N hyperbus


Workaround:
Issue 1998217: HyperBus interface vmk50 might be missing on vSphere ESXi causing container creation failure

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.
 

Workaround: Complete the following steps.

  1. Retrieve the vmk50 port ID using CLI on vSphere ESXi
    net-dvs | grep vmk50 -C 10
  2. Create the vmk50 interface on vSphere ESXi.
    esxcli network ip interface add -P <port-id from step-1> -s DvsPortset-0 -i vmk50 -N hyperbus
  3. Assign an IP address to the vmk50 interface.
    esxcfg-vmknic -i 169.254.1.1 -n 255.255.0.0 -s DvsPortset-0 -v <port-id from step-1> -N hyperbus


Additional Information

Impact/Risks:
None

Powered by Wolken Software