Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

ESX Server 3.0.1, Patch ESX-1002093; Update to VMkernel, Fixes for vmklinux Heap Allocation Failure, Networking Issues, Extraneous Warning Messages

Details

Release Date: 11/15/07
Document Last Updated: 11/15/07

Download Now
Download Size:
32.0 MB
Download Filename:
ESX-1002093.tgz
md5sum:
0bd9a4d952ab7c8c68d8111568d7555d
Product Versions
ESX Server 3.0.1
Patch Classification
Critical
Supersedes
Virtual Machine Migration or Reboot Required
Yes
ESX Server Host Reboot Required
Yes
PRs Fixed
159395, 179129, 182137, 182664
Affected Hardware
N/A
Affected Software
Unknown
RPMs Updated
VMware-esx-vmkernel
Related CVE numbers
N/A

Summary

This patch fixes the following issues:
  • Fixes an issue where a malformed IP packet might cause the ESX Server host to stop responding. This fix checks IP header information, and rejects malformed IP packets.
  • Fixes an issue where the ESX Server host might stop responding due to vmklinux heap allocation failure. vmklinux is a module loaded on top of vmkernel that creates its own heap for memory allocation/free requests from within the module. During module creation, the kernel can be set up for minimum and maximum heap sizes of 256KB and 20MB. Based on the type of heap this module creates (low memory heap), fragmentation of large pages (physical pages of 2MB) might prevent vmkernel from increasing the heap size. This in turn might result in memory allocation requests failing within the vmklinux module. The fix avoids these failures by making vmkernel grow the vmklinux heap to 10MB early on during boot time, when there shouldn't be any large page fragmentation.
  • Fixes an issue where a Reverse Address Resolution Protocol (RARP) broadcast might be sent to multiple vmnics. The ESX Server host normally registers a virtual machine with a switch through a transmission of a RARP broadcast packet that contains the MAC address of the virtual machine. In a topology of NIC teams connected to two or more switches, the virtual machine does not register with the switch. For this reason, the RARP packets are sent through multiple interfaces. This works for a single switch, as a Content-Addressable Memory (CAM) table can only update one entry at a time. However, when ESX Server NIC teams are connected to cascaded switches, a race condition leads to an attempt to update the CAM table of both switches with conflicting entries, with the result that both entries are discarded, leaving the virtual machine unregistered with the switch.
  • Fixes an issue related to a warning message being logged on the service console and in /var/log/vmkwarning . The latest stepping of the Intel Xeon 5300 quad-core processor supports 38-bit physical addresses, while older steppings only support 36-bit physical addresses. When these steppings are mixed in a server specifically certified to support such a configuration, a warning is raised. With this fix, the warning is no longer logged.

Solution

Symptoms

The following symptoms might be seen in systems without this patch: 
  • Faulty information on an IP header might cause checksum calculation operations to falter, resulting in system failure.
  • In a scenario where the ESX Server host becomes unresponsive due to vmklinux heap allocation failure, messages similar to the following might be logged in vmkernel log:

    23:02:36:34.253 cpu6:1037)WARNING: HeapMgr: 715: Could not allocate large pages: Out of memory
    23:02:36:34.253 cpu6:1037)WARNING: HeapMgr: 882: Failed to allocate/xmap large pages.
    23:02:36:34.253 cpu6:1037)WARNING: HeapMgr: 1058: Could not add memory to heap allocator HeapMgrLowMem.
    23:02:36:34.253 cpu6:1037)WARNING: Heap: 1271: Could not allocate 1114112 bytes for dynamic heap vmklinux. Request returned Out of memory
    23:02:36:34.253 cpu6:1037)WARNING: Heap: 1370: Heap_Align(vmklinux, 432/432 bytes, 4 align) failed. caller: 0x7bd7fc
    23:02:36:34.253 cpu6:1037)WARNING: LinSCSI: 273: No memory available! Called from 0x7bd7fc

  • A Reverse Address Resolution Protocol (RARP) broadcast might be sent to multiple vmnics in the following scenario:

    1. Set up VMotion between two ESX 3.0.2 Server hosts.
    2. Both ESX server hosts should have quad port or dual port NICs. Two NICs should be connected to one Cisco switch and rest two to another Cisco switch.
    3. Both Cisco switches should have cascade cable connection.
    4. Ping the VMotion virtual machine continuously and perform VMotion around ten times.
    5. At a certain point during VMotion, the virtual machine becomes unreachable.  The race condition occurs at this point.  Due to the race condition, the virtual machine becomes unreachable for around 40-50 ping requests.  The race is due to the RARP being sent out on the two interfaces.  Because the switches are expandable, it takes time for the switches to consolidate their CAM table.

  • When the ESX Server host boots up with mixed stepping processors (G-stepping processor and B-stepping processor), the following error messages might be displayed on the service console:

    0:00:00:29.763 cpu7:1031)ALERT: MTRR: 103: BSP has 36 physical bits but AP 7 has 38
    0:00:00:29.763 cpu4:1028)ALERT: MTRR: 103: BSP has 36 physical bits but AP 4 has 38
    0:00:00:29.763 cpu6:1030)ALERT: MTRR: 103: BSP has 36 physical bits but AP 6 has 38
    0:00:00:29.763 cpu5:1029)ALERT: MTRR: 103: BSP has 36 physical bits but AP 5 has 38

    These messages might also be logged in /var/log/vmkwarning .

Deployment Considerations

This patch resolves the symptoms discussed above.

Download Instructions

Download and verify the patch bundle as follows:

1.      Download patch ESX-1002093 from http://www.vmware.com/download/vi/vi3_patches.html .

2.      Log in to the ESX Server service console as root.

3.      Create a local depot directory.

# mkdir -p /var/updates

Note: VMware recommends that you use the updates directory.

4.      Change your working directory to /var/updates.

# cd /var/updates

5.      Download the tar file into the /var/updates directory.

6.      Verify the integrity of the downloaded tar file:


# md5sum ESX-1002093.tgz

The md5 checksum output should match the following:

0bd9a4d952ab7c8c68d8111568d7555d ESX-1002093.tgz

7.      Extract the compressed tar archive:

# tar -xvzf ESX-1002093.tgz

8.      Change to the newly created directory, /var/updates/ESX-1002093:

# cd ESX-1002093

Installation Instructions

Note: All virtual machines on the host must be either shut down or migrated using VMotion before applying the patch.  A reboot of the ESX Server host is required after applying the patch.

 

After you download and extract the archive, and if you are in the directory that you previously created, use the following command to install the update:

# esxupdate update

To run esxupdate from a different directory, you must specify the bundle path in the command:

# esxupdate -r file://<directory>/ESX-1002093 update

For example, if the host is called depot:

# esxupdate -r file:///depot/var/updates/ESX-1002093 update

During the update process, logs appear on the terminal. You can specify the verbosity of esxupdate logs by using the -v option as shown below.

# esxupdate -v 10 file://<directory>/ESX-1002093 update

For more information how to use esxupdate, see the Patch Management for ESX Server 3 tech note at http://www.vmware.com/pdf/esx3_esxupdate.pdf .

Keywords

esxpatch;esx301

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
Actions
  • Bookmark Document
  • Email Document
  • Print Document
  • Subscribe to Document
  • Bookmark and Share