Snapshot quiescing fails on Linux guests after upgrading to ESXi and VMware Tools 5.1 and 5.5
search cancel

Snapshot quiescing fails on Linux guests after upgrading to ESXi and VMware Tools 5.1 and 5.5

book

Article ID: 342815

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

Symptoms:
  • Linux guest virtual machines fail to take quiesced snapshots after upgrading ESXi and VMware Tools to 5.1 and 5.5.
  • Using the vSphere Client to take a quiesced snapshot fails immediately with the error:

    The guest OS has reported an error during quiescing. The error code was: 3 The error message was: Error when enabling the sync provider.

  • The hostd.log file, located at /var/log/ of the host, running the virtual machine contains entries similar to:

    [3CB9EB90 verbose 'vm:/vmfs/volumes/<datastore>/<VMNAME>/<VMNAME>.vmx'] Handling message _vmx##: The guestOS has reported an error during quiescing.
    --> The error code was: 3
    --> The error message was: Error when enabling the sync provider.


  • In a Red Hat Enterprise Linux (RHEL) virtual machine, the messages.log file located at /var/log/ contains entries similar to:

    [ debug] [vmbackup] *** VmBackupStart
    [ debug] [vmbackup] Using quiesceApps = 1, quiesceFS = 1, allowHWProvider = 1,execScripts = 1, scriptArg = , timeout = 0
    [ debug] [vmbackup] Quiescing volumes: 6000c296-7562-12a2-a3a5-6547bbe78021
    [ debug] [vmbackup] *** VmBackup_SendEvent
    [ debug] [vmbackup] *** VmBackupStartScripts
    [ debug] [vmbackup] Trying to run scripts from /etc/vmware-tools/backupScripts.d
    [ debug] [vmbackup] *** VmBackupAsyncCallback
    [ debug] [vmbackup] VmBackupAsyncCallback: checking VmBackupOnFreeze
    [ debug] [vmbackup] Async request 'VmBackupOnFreeze' completed
    [ debug] [vmbackup] *** VmBackupEnableSync
    [ info] [guestinfo] Poll loop disabled.
    [ debug] [vmbackup] *** VmBackupSyncDriverStart
    [ debug] [vmsvc] SyncDriver: Freezing using Linux ioctls...
    [ debug] [vmsvc] SyncDriver: failed to open 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx': 2 (No such file or directory)
    [ warning] [vmbackup] Error freezing filesystems

  • With VMwaretools debug logging enabled, you may see messages similar to:

    <YYYY-MM-DD><time>| vcpu-0| I120: Guest: [ debug] [vmsvc:vmsvc] SyncDriver: Freezing using Linux ioctls...</time>

    Or

    <YYYY-MM-DD><time>| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.
    <YYYY-MM-DD><time>| vmx| I120: Tools: Tools heartbeat timeout.
    <YYYY-MM-DD><time>| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.
    <YYYY-MM-DD><time>| vmx| I120: GuestRpc: app toolbox's second ping timeout; assuming app is down
    <YYYY-MM-DD><time>| vmx| I120: ToolsBackup: changing quiesce state: STARTED -> DONE
    <YYYY-MM-DD><time>| vmx| I120: SnapshotVMXTakeSnapshotComplete: done with snapshot 'smvi_ee2e62bb-2eef-4d86-a03d-feb834813702': 0
    <YYYY-MM-DD><time>| vmx| I120: SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine (40).

    </time></time></time></time></time></time></time>
    For more information, see Enabling debug logging for VMware Tools within a guest operating system (1007873)


Environment

VMware vCenter Server 5.1.x
VMware vSphere ESXi 5.1
VMware vCenter Server 5.5.x
VMware vSphere ESXi 5.5

Cause

There can be two possible causes for this issue:
Note: Red Hat has addressed this issue in bug 768304. You need Red Hat customer login credentials to view the bug details.

Resolution

  • You need to check and possibly update the kernel of your Linux guest virtual machine. If your Linux is RHEL, contact Red Hat for clarification about the release which can reliably support the FIFREEZE/FITHAW ioctl feature. For other vendor's Linux releases, you may have to contact your vendor to determine the Linux kernel used. For more information, see the Additional Information section.

  • Alternatively, you can downgrade from ESXi 5.1 or 5.5 tools to ESXi 5.0 tools on the guest OS. This will force the use of the VMware vmsync driver to do the quiesce snapshot, and bypass the issue.


Additional Information

Some vendors may not have a Linux release from kernel version 2.6.35-22. For instance, Ubuntu does, and RHEL does not. If your vendor does not have this particular or a later 2.6.35 release available, contact them for clarification. To see the Linux kernel versions associated with each RedHat version, see the document Red Hat Enterprise Linux Release Dates documentation.

For more information, see the Linux kernel releases section in the Linux Kernel 2.6 Release notes.

Note: The preceding links were correct as of May 22, 2015. If you find the link is broken, please provide feedback and a VMware employee will update the link.

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box

Enabling debug logging for VMware Tools within a guest operating system
Snapshot quiescing fails in Linux guests when disk.enableUUID is set to true in the vmx file
ESXi および VMware Tools 5.1 へのアップグレード後に Linux ゲストでのスナップショット静止が失敗する
升级到 ESXi 和 VMware Tools 5.1 和 5.5 后 Linux 客户机上的快照静默失败