Understanding Internet Control Message Protocol redirects
search cancel

Understanding Internet Control Message Protocol redirects

book

Article ID: 310581

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article provides information about the behavior associated with Internet Control Message Protocol (ICMP) redirects and how to troubleshoot the behavior.

Symptoms:
Certain hosts in your environment are no longer able to communicate with VMware vCenter Server.

Environment

VMware vCenter Server 4.1.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Installable
VMware ESXi 4.1.x Embedded
VMware ESX 4.1.x
VMware ESXi 4.0.x Embedded
VMware vCenter Server 5.1.x
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.5
VMware vCenter Server 4.0.x
VMware ESX 4.0.x
VMware vCenter Server 5.0.x
VMware vCenter Server 5.5.x
VMware vSphere ESXi 5.0

Cause

Network interrupts can cause a network device (usually by a router or firewall) to send an ICMP redirect. ICMP redirects are used to notify the hosts on the data link that a better route is available for a particular destination.

This can be problematic for VMware vSphere environments because it has the ability to interrupt connectivity between the hosts and vCenter Server. Under normal circumstances, you can expect a second ICMP redirect to be sent to the host when the issue has been resolved. However, it has been observed that this does not always happen.

Resolution

To confirm if an ICMP redirect is the cause of your connectivity problems, perform a capture on the affected host and vCenter Server.
If both devices connect through the same router or firewall, their traffic should be destined for the same MAC address.

When you perform a capture, you see that one entity is sending the traffic to a MAC address that is unfamiliar. The unfamiliar MAC address is actually the MAC of the ICMP redirected gateway.
To perform a capture on vCenter Server, you can use a utility such as Wireshark.
Note: VMware does not endorse or recommend any particular third-party utility, nor is the list above meant to be exhaustive.
  • To perform a capture on an ESXi/ESX host, run this command:
tcpdump or tcpdump-uw : tcpdump/tcpdump-uw -e -i vmk# -nn host x.x.x.x
  • Where vmk# is the vmk associated with the management interface of the host and x.x.x.x is the IP of vCenter Server.
  • -i denotes the interface and -e allows tcpdump to display the MAC addresses (by default it does not).

To work around this issue, perform one of these options:
  1. Work with the physical switch, router, or firewall vendor to determine if there is a way to force an ICMP redirect to be sent correcting the communication issue.
  2. Reboot the host to clear the ICMP redirect from the hypervisor.
  3. Delete the affected interface and recreate it.
This issue generally occurs on host Service Console or Management interfaces. The proceeding steps walk you through removing and recreating a service console or management interface. Perform these steps from a console session. When the console session is removed, you lose connectivity to the host until the interface is recreated. The virtual machines remain unaffected during this process.
  1. Find where the vmknic in question is located using these commands:

    1. Run esxcfg-vmknic -l for ESXi hosts
    2. Run esxcfg-vswif -l for ESX hosts

  2. Find the vSwitch or vDSwitch name on which the interface is running. Run the command:

    esxcfg-vswitch -l |less

    Record the name exactly as it appears (it is case sensitive). If using a vDS, record the port number that the interface is associated with.

  3. Remove the offending interface by running one of these commands:
    1. To remove a vmk from a vSwitch, run: esxcfg-vmknic -d PortGroupName
    2. To remove a vmk from a vDSwitch, run: esxcfg-vmknic -d -v port_number -s vDSname
    3. To remove a vswif interface from a vSwitch, run: esxcfg-vswif -d vswif#
    4. To remove a vswif interface from a vDSwitch, run: esxcfg-vswif -d vswif#

  4. Recreate the interface by running one of these commands:
    1. To create a vmk interface on a vSwitch, run: esxcfg-vmknic -a -i x.x.x.x -n x.x.x.x -p PortGroupName vSwitchName
    2. To create a vmk interface on a vDS, run: esxcfg-vmknic -a -i x.x.x.x -n x.x.x.x PortGroupName -s vDSwitchName
    3. To create a vswif interface on a vSwitch, run: esxcfg-vswif -a -i x.x.x.x -n x.x.x.x -p PortGroupName
    4. To create a vswif interface on a vDS, run: esxcfg-vswif -a -i x.x.x.x -n x.x.x.x PortGroupName -s vDSwitchName


Additional Information

For information on ICMP ping response behavior in ESXi 5.1/5.5, see Change to ICMP ping response behavior in ESXi 5.1 and ESXi 5.5 (2042189).Change to ICMP ping response behavior in ESXi 5.1 and ESXi 5.5

Impact/Risks:
Deleting and recreating the Service Console or Management Interface of a host causes the host to drop from vCenter Server. The primary IP of the host is unreachable during this procedure, which may trigger other monitoring alerts in your environment. Use your discretion to determine who you should notify about the changes you are going to make. The entire delete and recreate process takes approximately 5 minutes.

Powered by Wolken Software