Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Cisco Nexus 1000V loses network connectivity with Cisco VSM when a VMware Update Manager baseline includes Critical 1000V fixes for ESXi 5.0 hosts (2036048)

Symptoms

  • After a VMware Update Manager baseline with critical Nexus 1000V VEM (Virtual Ethernet Module) patches is configured for ESXi 5.0 hosts, these hosts will be upgraded to the latest Nexus 1000V patches, whether or not the Cisco VSM (Virtual Supervisor Module) is managing the ESXi hosts, which causes the loss of network connectivity between the VEM and the VSM.

  • You may also see these errors in the vmware-vum-server-log4cpp.log file:

    • <vib-scan-data>
      <id>Cisco_bootbank_cisco-vem-v144-esx_4.2.1.1.5.2.0-3.0.1</id>
      <pkgstate>uninstalled</pkgstate>
      <name>cisco-vem-v144-esx</name>
      <version>4.2.1.1.5.2.0-3.0.1</version>
      <vendor>Cisco</vendor>
      <meetsSystemReq>True</meetsSystemReq>
      <pkgDepsMetByHost>True</pkgDepsMetByHost>
      <requires>
      <requirement>esx-base << 5.1.0</requirement>
      <providedBy>
      <vibID>VMware_bootbank_esx-base_5.0.0-0.4.504890</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-1.18.768111</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-0.3.474610</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-1.13.702118</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-1.11.623860</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-1.17.764879</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-1.16.721882</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-0.10.608089</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-0.7.515841</vibID>
      <vibID>VMware_bootbank_esx-base_5.0.0-0.0.469512</vibID>
      </providedBy>
      <providedByHost>True</providedByHost>
      <hostProvides>
      <package name="esx-base" version="5.0.0-0.7.515841"/>
      </hostProvides>
      </requires>
      <conflicts>
      <conflictsWithHost>False</conflictsWithHost>
      </conflicts>
      <obsoletes>
      <vibID>Cisco_bootbank_cisco-vem-v140-esx_4.2.1.1.5.1.0-3.0.1</vibID>
      <vibID>Cisco_bootbank_cisco-vem-v133-esx_4.2.1.1.4.2.0-3.0.1</vibID>
      <vibID>Cisco_bootbank_cisco-vem-v131-esx_4.2.1.1.4.1.0-3.0.4</vibID>
      <vibID>Cisco_bootbank_cisco-vem-v142-esx_4.2.1.1.5.1a.0-3.0.1</vibID>
      <obsoletesHost>True</obsoletesHost>
      <hostObsoleted>
      <package name="cisco-vem-v140-esx" version="4.2.1.1.5.1.0-3.0.1"/>
      </hostObsoleted>
      <obsoletedByHost>False</obsoletedByHost>
      </obsoletes>
      <systemReqs>
      <swPlatform productLineID="embeddedEsx" version="5.*"/>
      <maintenanceMode>True</maintenanceMode>
      </systemReqs>
      <postInstall>
      <rebootRequired>False</rebootRequired>
      <hostdRestart>False</hostdRestart>
      </postInstall>
      </vib-scan-data>
    • [2012-09-1410:00:14:312 'compatibilityHelper' 10724 INFO] [compatibilityHelper, 85] Result of compatibility check for vib Cisco_bootbank_cisco-vem-v142-esx_4.2.1.1.5.1a.0-3.0.1 against component Cisco Nexus 1000V VSM-4.2(1)SV1(5.1): Failure
      [2012-09-1410:00:14:312 'compatibilityHelper' 10724 INFO] [compatibilityHelper, 85] Result of compatibility check for vib Cisco_bootbank_cisco-vem-v144-esx_4.2.1.1.5.2.0-3.0.1 against component Cisco Nexus 1000V VSM-4.2(1)SV1(5.1): Failure
      [2012-09-1410:00:14:312 'HostUpdateDepotManager' 10724 INFO] [scanHost20, 789] Scan results for DVS patching on host14.17.119.22 (entity: host-448) have been processed.
    • [2012-09-1410:00:15:093 'HostUpdateDepotManager' 7384 INFO] [installController, 270] Install , Cisco_bootbank_cisco-vem-v144-esx_4.2.1.1.5.2.0-3.0.1, VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.500.1.11.623860, VMware_bootbank_esx-base_5.0.0-1.18.768111, VMware_bootbank_misc-drivers_5.0.0-1.18.768111, VMware_bootbank_net-e1000_8.0.3.1-2vmw.500.1.18.768111, VMware_bootbank_net-e1000e_1.1.2-3vmw.500.1.11.623860, VMware_bootbank_net-nx-nic_4.0.557-3vmw.500.1.11.623860, VMware_bootbank_sata-ahci_3.0-6vmw.500.1.11.623860, VMware_bootbank_sata-ata-piix_2.12-4vmw.500.1.11.623860, VMware_bootbank_scsi-aacraid_1.1.5.1-9vmw.500.1.11.623860, VMware_bootbank_scsi-megaraid-sas_5.34-1vmw.500.1.11.623860, VMware_bootbank_scsi-mpt2sas_06.00.00.00-6vmw.500.1.11.623860, VMware_bootbank_scsi-mptsas_4.23.01.00-5vmw.500.1.18.768111, VMware_bootbank_scsi-rste_2.0.2.0088-1vmw.500.1.11.623860, VMware_locker_tools-light_5.0.0-1.18.768111 on host: 14.17.119.22
      [2012-09-1410:00:15:093 'HostUpdateDepotManager' 7384 INFO] [installController, 163] Event: START_INSTALL_UPDATES selected updates, ESXi500-201203203-UG, ESXi500-201203204-UG, ESXi500-201203206-UG, ESXi500-201203207-UG, ESXi500-201203208-UG, ESXi500-201203209-UG, ESXi500-201203210-UG, ESXi500-201203211-UG, ESXi500-201207401-BG, ESXi500-201207402-BG, ESXi500-201207403-BG, ESXi500-201207405-BG, ESXi500-201207406-BG, ESXi500-Update01, VEM500-201208144102-BG on 14.17.119.22
      [2012-09-1410:00:15:093 'HostUpdateDepotManager' 7384 INFO] [installController, 166] Event: START_INSTALL_UPDATES pre-requisites on 14.17.119.22
      [2012-09-1410:00:15:093 'HostUpdateDepotManager' 7384 INFO] [installController, 170] Event: START_INSTALL_UPDATES shared vibs on 14.17.119.22
      [2012-09-1410:00:15:093 'HostUpdateDepotManager' 7384 INFO] [installController, 174] Event: START_INSTALL_UPDATES Obsoleted update, ESXi500-201203101-SG, ESXi500-201203102-SG, ESXi500-201203103-SG, ESXi500-201203201-UG, ESXi500-201203202-UG, ESXi500-201203205-UG, ESXi500-201204401-BG, ESXi500-201205401-SG, ESXi500-201206401-SG, ESXi500-201207101-SG, ESXi500-201207102-SG, VEM500-201204142102-BG on 14.17.119.22
      [2012-09-1410:00:15:109 'HostUpdateDepotManager' 7384 INFO] [installController, 702] Add to install list [Cisco_bootbank_cisco-vem-v144-esx_4.2.1.1.5.2.0-3.0.1] for host:14.17.119.22(Entity: host-448]

Cause

The VEM patches for Nexus 1000V maintenance releases 4.2(1) SV1(5.1a) and 4.2(1) SV1(5.2) are marked as Critical due to Severity 1 and 2 included fixes.

When VMware Update Manager is configured with a baseline of critical Nexus 1000V patches, it upgrades a host's VEM modules even if the host is running 4.2(1) SV1(5.1). When this issue manifests, the VMware Update Manager logs indicate that VMware Update Manager recognizes the new bundle as incompatible.

Resolution

This is not a VMware issue. The 1000V patch is no longer available in VMware's patch portals and should not be included in a baseline for remediation.

For a repository that is already populated with the Nexus 1000V VEM patches, since this patch is classified as critical, VMware Update Manager includes it automatically in dynamic baselines. VMware recommends using static baselines or ignoring this patch if it appears in the VMware Update Manager repository.

To work around this issue:
  1. Before applying a baseline of critical patches, remove all Nexus 1000V VEM modules.

  2. If you have already applied a baseline and have encountered the issue, perform these steps for each affected host:

    1. Place the ESXi host in maintenance mode.
    2. Remove it from the Nexus 1000v VDS.
    3. Find the VEM module (cisco-vem-vXXX-esx) on the ESXi console:

      # esxcli software vib list | grep cisco

    4. Remove that VEM module:

      # esxcli software vib remove –n cisco-vem-vxxx-esx

      Note: If you are running version 5.2 and want to roll back to version 5.1a, you can simple install that version, which automatically removes the old VEM module. You can skip this step and proceed to step e directly.

    5. Apply the VIB for the previous VEM image:

      # esxcli software vib install –v vib-file

      Note: You can obtain the VEM module for previous Nexus 1000v releases can be obtained from Cisco Central. You must log in to obtain the VEM module.

    6. Exit maintenance mode.

  3. VMware worked with Cisco to remove the above two VEM images from these pages:


    However, the VEM images are still available from:

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 2 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 2 Ratings
Actions
KB: