Search the VMware Knowledge Base (KB)
View by Article ID

Cumulus Linux, Integration with VMWare NSX for vSphere 6.2.4 (2149268)

  • 0 Ratings


This article provides information about the Cumulus Linux Network Operating System solution for NSX Solution.

Disclaimer: The partner product referenced in this article is a software module that is developed and supported by a partner. Use of this product is also governed by the end user license agreement of the partner. You must obtain from the partner the application, support, and licensing for using this product.


Cumulus Linux VTEP solution for VMWare NSX

Supported software

Cumulus Linux is the networking industry's first full-featured Linux operating system. The Debian Jessie-based, networking-focused distribution runs on hardware produced by a broad partner ecosystem, ensuring unmatched customer choice regarding silicon, optics, cables, and systems. Refer to the VMware compatibility Guide for certified Switch software version.

Version of the VMware product

NSX for vSphere 6.2.4

Support information
Cumulus Networks Support Hours of Operation
Support Engineers always available
  • 24 hours a day
  • 7 days a week
  • Every day of the year
How to contact support
  • Web
  • Email
  • Phone : +1 (844) 383-6700 or +1 (650) 383-6700, option 2
Escalation procedures

Contact Cumulus Networks support, reference a previously opened case, and request an escalation to increase priority with appropriate justification.

Additional support information

See User Guide.

Troubleshooting (logs, procedures and techniques)

Enabling Logging for Networking

The /etc/default/networking file contains two settings for logging:
  • To get ifupdown2 logs when the switch boots (stored in syslog)
  • To enable logging when you run systemctl [start|stop|reload] networking.service
This file also contains an option for excluding interfaces when you boot the switch or run systemctl start|stop|reload networking.service.You can exclude any interface specified in /etc/network/interfaces. These interfaces do not come up when you boot the switch or start/stop/reload the networking service.

cumulus@switch:~$ cat /etc/default/networking
# Parameters for the /etc/init.d/networking script
# Change the below to yes if you want verbose logging to be enabled
# Change the below to yes if you want debug logging to be enabled
# Change the below to yes if you want logging to go to syslog
# Exclude interfaces

Using ifquery to Validate and Debug Interface Configurations

You use  ifquery to print parsed interfaces file entries. To use ifquery to pretty print iface entries from the interfaces file, run:

cumulus@switch:~$ sudo ifquery bond0
auto bond0
iface bond0
    address 2001:ded:beef:2::1/64
    bond-slaves swp25 swp26

Use ifquery --check to check the current running state of an interface within the interfaces file. It will return exit code 0 or 1 if the configuration does not match. The line bond-xmit-hash-policy layer3+7 below fails because it should read bond-xmit-hash-policy layer3+4.

cumulus@switch:~$ sudo ifquery --check bond0
iface bond0
    bond-xmit-hash-policy layer3+7  [fail]
    bond-slaves swp25 swp26         [pass]
    address             [pass]
    address 2001:ded:beef:2::1/64   [pass]

ifquery --check is an experimental feature.
Use ifquery --running to print the running state of interfaces in the interfaces file format:

cumulus@switch:~$ sudo ifquery --running bond0
auto bond0
iface bond0
    bond-slaves swp25 swp26
    address 2001:ded:beef:2::1/64

ifquery --syntax-help provides help on all possible attributes supported in the interfaces file. For complete syntax on the interfaces file, see man interfaces and man ifupdown-addons-interfaces.
You can use ifquery --print-savedstate to check the ifupdown2 state database. ifdown works only on interfaces present in this state database.

cumulus@leaf1$ sudo ifquery --print-savedstate eth0
auto eth0
iface eth0 inet dhcp

Debugging mako template errors

An easy way to debug and get details about template errors is to use the mako-render command on your interfaces template file or on /etc/network/interfaces itself.

cumulus@switch:~$ sudo mako-render /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
#auto eth1
#iface eth1 inet dhcp
# Include any platform-specific interface configuration
source /etc/network/interfaces.d/*.if
# ssim2 added
auto swp45
iface swp45
auto swp46
iface swp46
cumulus@switch:~$ sudo mako-render /etc/network/interfaces.d/<interfaces_stub_file>

ifdown cannot find an interface that exists

If you are trying to bring down an interface that you know exists, use ifdown with the --use-current-config option to force ifdown to check the current /etc/network/interfaces file to find the interface. This can solve issues where the ifup command issues for that interface was interrupted before it updated the state database.

For example:

cumulus@switch:~$ sudo ifdown br0
error: cannot find interfaces: br0 (interface was probably never up ?)
cumulus@switch:~$ sudo brctl show
bridge name bridge id       STP enabled interfaces
br0     8000.44383900279f   yes     downlink
cumulus@switch:~$ sudo ifdown br0 --use-current-config

Removing all references to a child interface

If you have a configuration with a child interface, whether it's a VLAN, bond or another physical interface, and you remove that interface from a running configuration, you must remove every reference to it in the configuration. Otherwise, the interface continues to be used by the parent interface.

For example, consider the following configuration:

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto bond1
iface bond1
    bond-slaves swp2 swp1
auto bond3
iface bond3
    bond-slaves swp8 swp6 swp7
auto br0
iface br0
    bridge-ports swp3 swp5 bond1 swp4 bond3
    bridge-pathcosts  swp3=4 swp5=4 swp4=4
    address 2001::10/64

Notice that bond1 is a member of br0. If bond1 is removed, you must remove the reference to it from the br0 configuration. Otherwise, if you reload the configuration with ifreload -a, bond1 is still part of br0.

MTU set on a logical interface fails with error: "Numerical result out of range".

This error occurs when the MTU you are trying to set on an interface is higher than the MTU of the lower interface or dependent interface. Linux expects the upper interface to have an MTU less than or equal to the MTU on the lower interface.
In this example, the swp1.100 VLAN interface is an upper interface to physical interface swp1. If you want to change the MTU to 9000 on the VLAN interface, you must include the new MTU on the lower interface swp1 as well.

auto swp1.100
iface swp1.100
    mtu 9000
auto swp1
iface swp1
    mtu 9000

Interpreting iproute2 batch command failures

ifupdown2 batches iproute2 commands for performance reasons. A batch command contains ip -force -batch - in the error message. The command number that failed is at the end of this line: Command failed -:1.
This is a sample error for the command 1: link set dev host2 master bridge. There was an error adding the bond host2 to the bridge named bridge because host2 did not have a valid address.

error: failed to execute cmd 'ip -force -batch - [link set dev host2 master bridge
addr flush dev host2
link set dev host1 master bridge
addr flush dev host1
]'(RTNETLINK answers: Invalid argument
Command failed -:1)
warning: bridge configuration failed (missing ports)

Understanding the "RTNETLINK answers: Invalid argument" error when adding a port to a bridge.

This error can occur when the bridge port does not have a valid hardware address.
This can typically occur when the interface being added to the bridge is an incomplete bond; a bond without slaves is incomplete and does not have a valid hardware address.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.


  • 0 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)

Please enter the Captcha code before clicking Submit.
  • 0 Ratings