Knowledge Base

Due to incompatibilities with some recent chipsets, the VMkernel may disable USB interrupts in some hardware configurations. This happens with servers in which USB controllers share interrupt lines with other controllers, and the IOAPIC is designed to forward masked interrupts. During startup, the VMkernel automatically disables USB so the VMkernel can manage its interrupts more effectively.

VMware has identified the following servers exhibiting this behavior: NEC blades, NEC EXP5800, DL580, Dell 1850, Dell 2850 and Dell 2800. Other models may be affected in addition to these.

If you need to use USB on a server where the VMkernel has disabled it, you need to take these steps to ensure that the USB controller does not interfere with other PCI controllers:

1. Assign all PCI devices to be managed by the VMkernel, not the VMware Service Console.
2. Prevent all USB drivers from loading until after the VMkernel loads.
3. Load the VMkernel with the -M option (if you have ESX Server version 2.1.2).

More details for these steps are given below.

Manage PCI Devices from the VMkernel

Most PCI devices on your server can be assigned one of three modes: dedicated to the VMkernel, dedicated to the VMware Service Console, or shared between them. You can change these assignments at a terminal prompt by running vmkpcidivy -i as root. (See www.vmware.com/support/kb/enduser/std_adp.php?p_faqid=863 for more on using vmkpcidivy.)

Any controller that is dedicated to the service console must be assigned as Shared (type S at the prompt for that device) — with some limitations, described in the next paragraph. Interrupts from shared controllers are managed by the VMkernel, and forwarded to the service console when necessary. Controllers that are dedicated to the VMkernel can be left that way.

For NIC controllers, however, there are limitations. Prior to ESX Server 2.1, vmkpcidivy did not support Shared mode for NICs. A supplemental procedure is required for earlier releases. Furthermore, assigning the service console's primary NIC (eth0) as Shared (in ESX Server 2.1) causes the NIC's MAC address to change when the VMkernel loads. This can be a problem if you are using a DHCP server. In such cases, you should also use the supplemental procedure.

The supplemental procedure is documented in the ESX Server 2.1 Administration Guide, in the section called "Sharing the Service Console's Network Adapter with Virtual Machines," which is found in the chapter on networking. The procedure is summarized here:

1. In the /etc/modules.conf file, comment out the line that includes alias eth0. This prevents eth0 from starting before the VMkernel is loaded.
2. Run vmkpcidivy -i and assign all network controllers to the VMkernel.
3. Make sure you know the VMkernel's device label for each NIC. (See the Administration Guide for help with using findnic, the VMkernel Network Card Locator utility.)
4. Load the vmxnet_console driver to allow the service console to share one or more NICs:
   insmod vmxnet_console devName=""
   where is the VMkernel's device label for eth0, is the VMkernel's device label for eth1, etc. (If you find that the labels are in the wrong order, you can use ifdown eth for each NIC, then use rmmod vmxnet_console and insmod again with the labels in a different order.)
5. Bring the NICs up in the service console manually, with commands like these:
   ifup eth0
   or
   ifconfig eth1 up
6. Once you are confident that the insmod command is correct, you can add it (and the ifup or ifconfig commands) to your /etc/rc.d/rc.local file, where they take effect on each boot of your server.
7. Adjust the DHCP server configuration as needed, to compensate for the changes in the MAC addresses of your NICs. The NICs appear as virtual NICs with MAC addresses generated by ESX Server.

When you reboot the server, the new controller assignments take effect. Note that the NIC or NICs that are now shared with the VMkernel appear to be available for assignment to virtual switches, but they should not be assigned to virtual switches.

Load USB Drivers After the VMkernel

To delay loading USB drivers, use this procedure:

1. Edit the file /etc/modules.conf.
2. Comment out all lines containing usb-ohci, usb-uhci or usb-ehci, by inserting a # character at the start of each line. For example, if you find this line:
   alias usb-controller usb-ohci
   change it to look like this:
   #alias usb-controller usb-ohci
   
3. Exit the editor, saving your changes.
4. Create a text file named /etc/rc3.d/S91usb.
5. For each USB line that you commented out in the /etc/modules.conf file, add a line in the new /etc/rc3.d/S91usb file to load the same module using modprobe. The lines you add should look like one or more of these:
   modprobe usb-ohci
   modprobe usb-uhci
   modprobe usb-ehci
   
6. Exit the editor, saving your changes.

The next time you boot the server, the /etc/rc3.d/S90vmware file executes just before the /etc/rc3.d/S91usb file, ensuring that the VMkernel is prepared by the time the USB drivers load.

Load the VMkernel with the -M Option

If you are running ESX Server version 2.1.2, an additional step is required. To prevent the VMkernel from disabling USB, edit the VMware startup file as follows:

1. Make a backup of /etc/init.d/vmware.

     cp /etc/init.d/vmware /etc/init.d/vmware.backup

2. Edit the file /etc/init.d/vmware.
3. Find the line that looks like this:

     cd "$vmdb_answer_SBINDIR" &&
   "$vmdb_answer_SBINDIR"/"$kernloader" \

4. Insert -M just before the backslash at the end of the line, like this:

     cd "$vmdb_answer_SBINDIR" &&
   "$vmdb_answer_SBINDIR"/"$kernloader" -M \

5. Exit the editor, saving your changes.

The next time you boot the server, the VMkernel leaves USB controllers enabled.

Note: You should add -M only if you are running ESX Server version 2.1.2. For any earlier version, adding -M causes ESX Server to fail when it first loads. For any later version, -M is not needed. To be sure of your ESX Server version, use cat /etc/issue at a terminal prompt, or log on to the VMware Management Interface and look for the version after the VMware logo at the top of the browser display.