Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

ESXi 4.1 virtual machine fails to power on with the error: FoundryVMDirectlyOpenSocketToVMX: Failed to create socket pair (1035564)

Symptoms

When using VMware ESXi 4.1, you may experience one or more of these symptoms:

  • Unable to connect to the MKS
  • Unable to power on virtual machines
  • Powering on virtual machines reports some virtual machines in an invalid state
  • Power state of the virtual machine is reported incorrectly in the vCenter Server/ESX inventory
  • If you restart hostd, virtual machines that were previously not in an invalid state may appear as invalid
  • When a virtual machine is in an invalid state, you see these errors:

    vmx| VmdbPipeStreamsOvlError: write failed, draining reads
    vmx| VmdbPipeStreamsOvlError: Couldn't initiate write
    vmx| Redirecting stdin/stdout/stderr to /dev/null.
    vmx| SOCKET 1 (91) send error 12: Cannot allocate memory
    vmx| Vix: [177936 mainDispatch.c:2472]: VMAutomation: Connection Error (1) on connection 0.
    vmx| SOCKET 1 (91) send error 12: Cannot allocate memory -> not able to create socket,no mem available.
    mks| SOCKET 2 (92) recv error 104: Connection reset by peer
    mks| SOCKET 2 (92) destroying VNC backend on socket error: 1

  • The messages log contains entries similar to:

    • sfcb-CIMXML-Processor[9857708]: SendMsg sending to 7 9857708-9 Bad file descriptor
      sfcb-CIMXML-Processor[9857708]: spSendMsg sending to 7 9857708-9 Bad file descriptor
      sfcb-CIMXML-Processor[9857708]: --- spSendReq/spSendMsg failed to send on 7 (-1)
      root: sfcbd-watchdog:Restarting SFCB! Log a bug!!!
      root: sfcbd-watchdog:stopping sfcbd
      root: sfcbd Stopping sfcbd
      root: sfcbd-watchdog:starting sfcbd
      root: sfcbd Starting sfcbd
      sfcb-sfcb[9849840]: --- Using /etc/sfcb/sfcb.cfg

    • FoundryVMDirectlyOpenSocketToVMX: Failed to create socket pair.

  • The hostd logs may report this error when you try to power on the virtual machine:

    Cannot connect to /var/run/vmware/root_0/1299674323658606_59831734/testAutomation-fd: File not found

  • vMotion fails intermittently at a random percentage
  • Connecting to the remote console via the vSphere Client fails with the error:

    Unable to connect to the MKS: There is no VMware process running for config file

  • Unable to retrieve any files from the datastore via Datastore Browser
  • Copying a file using the datastore browser gives the error:

    Failed to log into NFC server

  • The vm-support diagnostic information gathering utility is unresponsive
  • The Health Status tab may not load correctly
  • Powering on a virtual machine fails at 95% with the error:

    A general system error occurred: Cannot connect to the virtual machine

Cause

This issue occurs due to exhaustion of VMkernel socket resources on ESXi hosts. It occurs more frequently on hosts where OEM CIM providers have been installed, which may be caused by additional CIM Providers loaded under sfcbd. To ensure the stability of the OEM CIM providers, ensure that they are updated to the latest release.
 
Note: This issue only occurs on ESXi. It does not occur on ESX. On ESX, sfcbd runs as a process in the service console rather than from a world under the VMkernel.

Resolution

Resolution

This issue is resolved by VMware ESXi 4.1 Patch ESXi410-201107401-BG. For more information, see VMware ESXi 4.1 Patch ESXi410-201107401-BG: Updates Firmware (2000609).
 
Note: A system which is found to be in a bad state must be rebooted prior to applying the patch.

Workaround

To work around this issue, stop the sfcbd hardware monitoring agent on the ESXi host. When sfcbd is disabled, hardware status information for the ESXi host will be unavailable.
 
To stop sfcbd:
  1. Log into the VMware ESXi host as the root user. For more information, see Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910).

  2. Run the command:

    /etc/init.d/sfcbd-watchdog stop

  3. To make this change persistent on reboot, run these commands:

    chkconfig sfcbd-watchdog off
    chkconfig sfcbd off
If hardware monitoring is an environmental requirement, you can extend the amount of time before the issue re-occurs by changing the configuration of sfcbd:
  1. From the ESXi shell, edit the /etc/sfcb/sfcb.cfg file using a text editor.
  2. Search for the entry provProcs: 16, and change the value from 16 to 12.
  3. Restart sfcbd for the changes to take effect using the command:

    /etc/init.d/sfcbd-watchdog restart

    Note: Depending on the system workload, this change may only temporarily resolve the issue.

If the symptoms persist after applying this patch or workaround, collect logs from the environment and contact VMware Support. For more information, see:

Additional Information

Virtual machines can appear as powered off while they are actually powered on. You can recover these virtual machines without rebooting them. For more information, see Virtual machine appears powered off after restarting the management services on the host (1033591).
 
To be alerted when this article is updated, click Subscribe to Document in the Actions box.

Tags

cannot-power-on-vm cannot-power-on

See Also

Update History

05/11/2011 - Added hot-patch information and how to recover virtual machines. 06/20/2011 - Added information about Health Status tab to the symptoms. 07/20/2012 - Added additional symptom reflecting power on failure at 95%. 07/29/2011 - Added resolution information.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 54 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 54 Ratings
Actions
KB: