Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Commands to monitor snapshot deletion in ESX 2.5/3.x/4.x and ESXi 3.x/4.x/5.x (1007566)

Symptoms

  • Cannot observe the progress when deleting a snapshot.
  • It is difficult to determine status of snapshot deletion.
  • Snapshot removal task stops at 95% and does not appear to proceed.
  • The hostd process does not respond.
  • Storage vMotion operation stays at 18% for a long time.

Purpose

This article provides information on monitoring directories using the watch command and waiting for the snapshot deletion operations to complete in ESX and ESXi.

For more information on how snapshots work, see Understanding virtual machine snapshots in VMware ESXi and ESX (1015180).

Resolution

Note: Time involved to commit snapshots is environmental and subjective.

ESX 3.5/4.x and ESXi 4.1/5.x

To monitor directories during snapshot deletion in ESX 3.5/4.x and ESXi 4.1/5.x: 
  1. Log in as root to the ESX host using SSH. For more information, see Connecting to an ESX host using a SSH client (1019852) or Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910)
  2. Navigate to the virtual machine directory containing vmdk virtual disk files.
  3. List files in the directory by executing:
     
    # ls -al
     
  4. Determine any VM_NAME-00000#.vmdk or VM_NAME-00000#-delta.vmdk snapshot files. Look for numbered files following the hyphen(-) in the name.
  5. To monitor the VMDK snapshot and base disks which are currently being updated use the following watch command:

    # watch -d 'ls -luth | grep -E "delta|flat"'

    where:

    -d  highlights the differences between successive updates
    t   sorts by modification time
    l   shows a long listing which displays additional file information  
    sorts by and shows access time
    h   prints sizes in a readable format like 1K 234M 2G

    You can also run the following command to monitor the time stamp update of the base disks to confirm if the process is working or not.

    ls -lrt |grep -E "flat|delta"
     
    This command monitors the contents of a directory and displays files by their modification date.

    Note: In ESX 3.5 and 4.0 (pre Update 2) the snapshot delta files will be written to the previous snapshot delta file and so on and are finally written to the base disk (flat). In ESX/ESXi 4.0 Update 2 and later the process works differently in that the data in snapshots (deltas) are written directly to the base disk (flat). For more information on the snapshot process, see Understanding virtual machine snapshots in VMware ESXi and ESX (1015180) and Consolidating snapshots (1007849).
     
    If there are more than 10 snapshots, use this command to monitor the snapshot commit process and to prevent the screen from filling with too many files:

    # while true;do date;ls -lht *vmdk|head -10;echo ________;sleep 3;done

    Note: You can quit the consolidation process monitoring by pressing Ctrl + C. 
     

ESXi 3.5/4.0

To monitor directories during snapshot deletion in ESXi 3.5/4.0:

  1. Log in as root to the ESXi host using the Tech Support mode. For more information, see Tech Support Mode for Emergency Support (1003677).
  2. Change to the virtual machine directory /vmfs/volumes/datastore/VM.
  3. Open a text editor such as vi or nano and create the file snapmon. For more/related information, see Editing files on an ESX host using vi or nano (1020302).
  4. Add this script the the file:

    snapmon
     clear
     while [ 1 ]
      do
      date;ls -luth *.vmdk
      sleep 2
      clear 
     done


  5. Click Save and close the file. 
  6. Run this command to make this file executable:

     # chmod ug+x snapmon

  7. Commit the snapshot from the Snapshot Manager.
  8. Execute the snapmon script to monitor the snapshot:

     # ./snapmon
Note: This process can be sent to background and brought to foreground using the fg command.
 
You can locate most recent pid (/bin/sh) using the ps ax command. 

To kill a running snapmon, run the command:
# kill <pid>


Additional Information

Note: The VMware Virtual Machine File System (VMware VMFS) is not like a normal, local file system. Do not treat /vmfs like a general-purpose file system. For example, do not store temporary or data files under /vmfs. Operations such as chmod, chown, and any commands that use the open() function or that otherwise modify file properties that are very fast on normal non-distributed file systems are significantly more costly on distributed file systems like VMware VMFS.

Avoid these operations unless absolutely necessary. For example, as a result of the high expense of open() calls, tail -f file is relatively safe, but tail file can cause problems.

For more information, see VMware ESX Server: Third-Party Software in the Service Console.

The Delete all snapshots process has significantly improved in these ESX patches. VMware recommends that you install them for performance improvements:

See Also

This Article Replaces

1035220

Update History

03/30/2010 - Provided shorter watch command 04/28/2010 - Indicated that this article does not apply to ESXi 05/24/2011 - Added symptom 12/16/2011 - Added command to monitor the snapshot commit process when there are more than 10 snapshots 10/26/2012 - Added ESXi 5.1 to Products 11/29/2012 - Added ESXi 3.x and 4.x to Products and added additional symptom 11/30/2012 - Added vCenter Server 5.1 to Products

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 66 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 66 Ratings
Actions
KB: