Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x hosts (2004605)

Purpose

This article provides steps to unmount a LUN from an ESXi 5.x host, which includes unmounting the file system and detaching the device. These steps must be performed for each ESXi host.

Note: The documented method using the vSphere Client is limited to detaching the device on a per-host basis. To detach a storage device from multiple hosts at the same time, see Automating detaching datastores using PowerCLI and the vSphere SDK for Perl in the Resolution section of this article.

For information on unmounting a datastore in ESXi/ESX 4.x, see Removing a LUN containing a datastore from VMware ESXi 4x and ESX 4.x (1029786).


Resolution

Removing a LUN in ESXi/ESX 4.x is complex. Improvements have been made in ESXi 5.x to streamline the procedure to make it easier for Administrators to remove LUNs.

Unmounting a LUN checklist

Before unmounting a LUN, ensure that:
  • If the LUN is being used as a VMFS datastore, all objects (such as virtual machines, snapshots, and templates) stored on the VMFS datastore are unregistered or moved to another datastore.

    Note: All CD/DVD images located on the VMFS datastore must also be unregistered from the virtual machines.

  • The datastore is not used for vSphere HA heartbeat.
  • The datastore is not part of a datastore cluster. For more information on datastore clusters, see the vSphere 5.1 Resource Management Guide or vSphere 5.0 Resource Management Guide.
  • The datastore is not managed by Storage DRS. For more information on Storage DRS, see the vSphere 5.1 Resource Management Guide or vSphere 5.0 Resource Management Guide.
  • The datastore is not configured as a diagnostic coredump partition. For more information, see Configuring a diagnostic coredump partition on an ESXi 5.x host (2004299).
  • Storage I/O Control is disabled for the datastore. For more information, see Managing Storage I/O Resources in the vSphere 5.1 Resource Management Guide or vSphere 5.0 Resource Management Guide.
  • No third-party scripts or utilities running on the ESXi host can access the LUN in question. If the LUN is being used as a datastore, unregister all objects (such as virtual machines and templates) stored on the datastore.
  • If the LUN is being used as an RDM, remove the RDM from the virtual machine. Click Edit Settings, highlight the RDM hard disk, and click Remove. Select Delete from disk if it is not already selected, and click OK.

    Note: This destroys the mapping file, but not the LUN content.

  • Check if the LUN/Datastore is used as the persistent scratch location for the host. For more information on persistent scratch, see Creating a persistent scratch location for ESXi 4.x and 5.x (1033696).

    This PowerCLI script can be used to check the current scratch location:

    $vcServer = "vCenter01"
    $cluster = "CL01"
    $esxCred = Get-Credential
    Connect-VIServer $vcServer | Out-Null
    #Connect to ESX hosts in cluster
    foreach ($esx in Get-Cluster $cluster | Get-VMHost) {
    Connect-VIServer $esx -Credential $esxCred | Out-Null
    Get-VMHostAdvancedConfiguration -Name "ScratchConfig.ConfiguredScratchLocation"
    }
Note: When using the vSphere Web Client with vSphere 5.1 and 5.5, only these checks are performed during the datastore unmount operation:
  • Host should not have any virtual machines residing on this datastore
  • Host should not use the datastore for HA heartbeats

Obtaining the NAA ID of the LUN to be removed

From the vSphere Client, this information is visible in the Properties window of the datastore.

From the ESXi host, run the command:

# esxcli storage vmfs extent list

The output is similar to:

Volume Name VMFS UUID                           Extent Number Device Name                           Partition
----------- ----------------------------------- ------------- ------------------------------------  ---------
datastore1  4de4cb24-4cff750f-85f5-0019b9f1ecf6             0  naa.6001c230d8abfe000ff76c198ddbc13e        3
Storage2    4c5fbff6-f4069088-af4f-0019b9f1ecf4             0  naa.6001c230d8abfe000ff76c2e7384fc9a        1
Storage4    4c5fc023-ea0d4203-8517-0019b9f1ecf4             0  naa.6001c230d8abfe000ff76c51486715db        1
LUN01       4e414917-a8d75514-6bae-0019b9f1ecf4             0  naa.60a98000572d54724a34655733506751        1


Make a note of the NAA ID of the datastore to use this information later in this procedure.

Note: Alternatively, you can run the esxcli storage filesystem list command, which lists all file systems recognized by the ESXi host.

Unmounting a LUN using the vSphere Client

To detach a storage device using the vSphere Client, see Detach Storage Devices in the vSphere 5 Documentation Center.

Caution: You must successfully unmount the datastore using the documented method prior to completing the steps described below. For more information, see Unmount VMFS or NFS Datastores in the vSphere 5 Documentation Center.

To unmount a LUN from an ESXi 5.0 host using the vSphere Client:
  1. If the LUN is an RDM, skip to step 2. Otherwise, in the Configuration tab of the ESXi host, click Storage. Right-click the datastore being removed, and click Unmount.

    A Confirm Datastore Unmount window appears. When the prerequisite criteria have been passed, click OK.

    Note: To unmount a datastore from multiple hosts in the vSphere Client, click Hosts and Clusters > Datastores and Datastore Clusters view (Ctrl+Shift+D). Perform the unmount task and select the appropriate hosts that should no longer access the datastore to be unmounted.

  2. Click the Devices view (under Configuration > Storage):



  3. Right-click the NAA ID of the LUN (as noted above) and click Detach. A Confirm Device Unmount window is displayed. When the prerequisite criteria have been passed, click OK. Under the Operational State of the Device, the LUN is listed as Unmounted.

    Note: The Detach function must be performed on a per-host basis and does not propagate to other hosts in vCenter Server. If a LUN is presented to an initiator group or storage group on the SAN, the Detach function must be performed on every host in that initiator group before unmapping the LUN from the group on the SAN. Failing to follow this step results in an all-paths-down (APD) state for those hosts in the storage group on which Detach was not performed for the LUN being unmapped.

  4. Confirm the LUN has been successfully detached. The LUN can then be safely unpresented from the SAN. For more information, contact your storage array vendor.

  5. Perform a rescan on all ESXi hosts which had visibility to the LUN. The device is automatically removed from the Storage Adapters.
When the device is detached, it stays in an unmounted state even if the device is re-presented (that is, the detached state is persistent). To bring the device back online, the device must be attached.

If the device is to be permanently decommissioned from an ESXi host, you may have to manually remove the NAA entries from the host configuration:
  1. To list the permanently detached devices, run the command:

    # esxcli storage core device detached list

    The output is similar to:

    Device UID                            State
    ------------------------------------  -----
    naa.50060160c46036df50060160c46036df  off
    naa.6006016094602800c8e3e1c5d3c8e011  off 


  2. To permanently remove the device configuration information from the system, run the command:

    # esxcli storage core device detached remove -d NAA_ID

    For example:

    # esxcli storage core device detached remove -d naa.50060160c46036df50060160c46036df

Unmounting a LUN using the command line

To unmount a LUN from an ESXi 5.x host using the command line:
  1. If the LUN is an RDM, skip to step 4. Otherwise, to obtain a list of all datastores mounted to an ESXi host, run the command:

    # esxcli storage filesystem list

    The output, which lists all VMFS datastores, is similar to:

    Mount Point                                       Volume Name UUID                                Mounted Type         Size         Free
    ------------------------------------------------- ----------- ----------------------------------- ------- ------ -----------  -----------
    /vmfs/volumes/4de4cb24-4cff750f-85f5-0019b9f1ecf6 datastore1  4de4cb24-4cff750f-85f5-0019b9f1ecf6    true  VMFS-5   140660178944   94577360896
    /vmfs/volumes/4c5fbff6-f4069088-af4f-0019b9f1ecf4 Storage2    4c5fbff6-f4069088-af4f-0019b9f1ecf4    true  VMFS-3   146028888064    7968129024
    /vmfs/volumes/4c5fc023-ea0d4203-8517-0019b9f1ecf4 Storage4    4c5fc023-ea0d4203-8517-0019b9f1ecf4    true  VMFS-3   146028888064  121057050624
    /vmfs/volumes/4e414917-a8d75514-6bae-0019b9f1ecf4 LUN01      4e414917-a8d75514-6bae-0019b9f1ecf4    true  VMFS-5  146028888064   4266131456


  2. To find the unique identifier of the LUN housing the datastore to be removed, run this command:

    # esxcfg-scsidevs -m

    This command generates a list of VMFS datastore volumes and their related unique identifiers. Make a note of the unique identifier (NAA_ID) for the datastore you want to unmount as this will be used later on.

    For more information on the esxcfg-scsidevs command, see Identifying disks when working with VMware ESX/ESXi (1014953).

  3. Unmount the datastore by running the command:

    # esxcli storage filesystem unmount [-u UUID | -l label | -p path ]

    For example, use one of these commands to unmount the LUN01 datastore:

    # esxcli storage filesystem unmount -l LUN01
    # esxcli storage filesystem unmount -u 4e414917-a8d75514-6bae-0019b9f1ecf4
    # esxcli storage filesystem unmount -p /vmfs/volumes/4e414917-a8d75514-6bae-0019b9f1ecf4


    Note: If the VMFS filesystem you are attempting to unmount has active I/O or has not fulfilled the prerequisites to unmount the VMFS datastore, you see an error in the VMkernel logs similar to:

    WARNING: VC: 637: unmounting opened volume ('4e414917-a8d75514-6bae-0019b9f1ecf4' 'LUN01') is not allowed.
    VC: 802: Unmount VMFS volume f530 28 2 4e414917a8d7551419006bae  f4ecf19b 4 1 0 0 0 0 0 : Busy


  4. To verify that the datastore has been unmounted, run the command:

    # esxcli storage filesystem list

    The output is similar to:

    Mount Point                                       Volume Name UUID                                Mounted Type         Size         Free
    ------------------------------------------------- ----------- ----------------------------------- ------- ------ -----------  -----------
    /vmfs/volumes/4de4cb24-4cff750f-85f5-0019b9f1ecf6 datastore1  4de4cb24-4cff750f-85f5-0019b9f1ecf6    true  VMFS-5   140660178944   94577360896
    /vmfs/volumes/4c5fbff6-f4069088-af4f-0019b9f1ecf4 Storage2    4c5fbff6-f4069088-af4f-0019b9f1ecf4    true  VMFS-3   146028888064    7968129024
    /vmfs/volumes/4c5fc023-ea0d4203-8517-0019b9f1ecf4 Storage4    4c5fc023-ea0d4203-8517-0019b9f1ecf4    true  VMFS-3   146028888064  121057050624
                                                     LUN01      4e414917-a8d75514-6bae-0019b9f1ecf4   false  VMFS-unknown version    0     0


    Note that the Mounted field is set to false, the Type field is set to VMFS-unknown version, and that no Mount Point exists.

    Note: The unmounted state of the VMFS datastore persists across reboots. This is the default behavior. If you need to unmount a datastore temporarily, you can do so by appending the --no-persist flag to the unmount command.

  5. To detach the device/LUN, run the command:

    # esxcli storage core device set --state=off -d NAA_ID

  6. To verify that the device is offline, run the command:

    # esxcli storage core device list -d NAA_ID

    The output, which shows that the Status of the disk is off, is similar to:

    naa.60a98000572d54724a34655733506751
       Display Name: NETAPP Fibre Channel Disk (naa.60a98000572d54724a34655733506751)
       Has Settable Display Name: true
       Size: 1048593
       Device Type: Direct-Access
       Multipath Plugin: NMP
       Devfs Path: /vmfs/devices/disks/naa.60a98000572d54724a34655733506751
       Vendor: NETAPP
       Model: LUN
       Revision: 7330
       SCSI Level: 4
       Is Pseudo: false
       Status: off
       Is RDM Capable: true
       Is Local: false
       Is Removable: false
       Is SSD: false
       Is Offline: false
       Is Perennially Reserved: false
       Thin Provisioning Status: yes
       Attached Filters:
       VAAI Status: unknown
       Other UIDs: vml.020000000060a98000572d54724a346557335067514c554e202020


    Running the partedUtil getptbl command on the device shows that the device is not found.

    For example:

    # partedUtil getptbl /vmfs/devices/disks/naa.60a98000572d54724a34655733506751

    Error: Could not stat device /vmfs/devices/disks/naa.60a98000572d54724a34655733506751 - No such file or directory.
    Unable to get device /vmfs/devices/disks/naa.60a98000572d54724a34655733506751


  7. If the device is to be permanently decommissioned, it is now possible to unpresent the LUN from the SAN. For more information, contact your storage team, storage administrator, or storage array vendor.

  8. To rescan all devices on the ESXi host, run the command:

    # esxcli storage core adapter rescan [ -A vmhba# | --all ]

    The devices are automatically removed from the Storage Adapters.

    Notes:
    • A rescan must be run on all hosts that had visibility to the removed LUN.
    • When the device is detached, it stays in an unmounted state even if the device is re-presented (that is, the detached state is persistent). To bring the device back online, the device must be attached. To do this via the command line, run the command:

      # esxcli storage core device set --state=on -d NAA_ID

  9. If the device is to be permanently decommissioned from an ESXi host, (that is, the LUN has been or will be destroyed), remove the NAA entries from the host configuration by running these commands:

    1. To list the permanently detached devices:

      # esxcli storage core device detached list

      The output is similar to:

      Device UID                            State
      ----------------------------  -----
      naa.50060160c46036df50060160c46036df  off
      naa.6006016094602800c8e3e1c5d3c8e011  off


    2. To permanently remove the device configuration information from the system:

      # esxcli storage core device detached remove -d NAA_ID

      For example:

      # esxcli storage core device detached remove -d naa.50060160c46036df50060160c46036df

    The reference to the device configuration is permanently removed from the ESXi host's configuration.

    Note: If the device is detached but still presented (step 7 was skipped), the command above fails to permanently remove the device from the system, and the device is automatically re-attached. You must complete step 7 for the device to be permanently removed.

Automating detaching datastores using PowerCLI and the vSphere SDK for Perl

It is possible to automate the process of detaching datastores from multiple hosts using PowerCLI scripts.

Using the PowerCLI

To detach a storage device using PowerCLI:
  1. Review the VMware Contributed Sample Code disclaimer.

  2. Download the PowerCLI script available at Automating Datastore/Storage Device Detachment in vSphere 5.

    Note: This PowerCLI script is provided "as-is" and is accordingly community supported. If you experience issues with this PowerCLI script, seek assistance from the VMware Communities forums.

  3. Import the script using the command:

    Import-Module path_to_script

  4. Ensure that you have already unmounted the target datastore. For more information, see Unmount VMFS or NFS Datastores in the vSphere 5 Documentation Center.

  5. List all datastores and their attached hosts by running the command:

    Get-Datastore | Get-DatastoreMountInfo | Sort Datastore, VMHost | FT –AutoSize

    You see output similar to:

    Datastore                VMHost              Lun                                                                        Mounted State  
    ---------                ------              ---                                                                        ------- -----  
    IX2ISCSI01               esx01.vmw.local     naa.5000144f52145699                                                         False Attached
    IX2ISCSI01               esx02.vmw.local     naa.5000144f52145699                                                         False Attached
    IX2ISCSI01               esx03.vmw.local     naa.5000144f52145699                                                         False Attached
    LocalDatastore           esx01.vmw.local     mpx.vmhba1:C0:T0:L0                                                           True Attached
    LocalDatastore           esx02.vmw.local     mpx.vmhba1:C0:T0:L0                                                           True Attached
    esx04-Internal-150GB     esx04.vmw.local     t10.ATA_____GB0160EAPRR_____________________________WCAT25563003________      True Attached
    esx04-Internal-500GB    esx04.vmw.local     t10.ATA_____WDC_WD5000AAKS2D00V1A0________________________WD2DWMAWF0069467    True Attached
    esx03-Internal-150GB    esx03.vmw.local     t10.ATA_____GB0160EAPRR_____________________________WCAT25704089________      True Attached
    esx03-Internal-500GB    esx03.vmw.local     t10.ATA_____WDC_WD5000AAKS2D00YGA0________________________WD2DWCAS85034601    True Attached


  6. Select the appropriate datastore and record the name beneath the Datastore column, and confirm that the Mounted column contains the value False for all hosts.

  7. Detach the devices from all hosts by running the command:

    Get-Datastore datastore_name| Detach-Datastore

    Where datastore_name is the name of the datastore recorded in step 3.

    You see output similar to:

    Detaching LUN naa.5000144f52145699 from host esx01.vmw.local...
    Detaching LUN naa.5000144f52145699 from host esx02.vmw.local...
    Detaching LUN naa.5000144f52145699 from host esx03.vmw.local...

Using the vSphere SDK for Perl

To detach a storage device using Perl:
  1. Review the VMware Contributed Sample Code disclaimer.

  2. Deploy the community-supported Perl script available in the VMware vSphere Blog, Automating Datastore/Storage Device Detachment in vSphere 5.

    Caution: Before proceeding, ensure you have already unmounted the target datastore. For more information, see Unmount VMFS or NFS Datastores in the vSphere 5 Documentation Center.

  3. List all datastores and their attached hosts by running the command:

    ./lunManagement.pl --server vcenter_ip --username user --operation list

    Where vcenter_ip is the IP address of the vCenter Server managing your hosts and user is a user with administrative privileges.

  4. You are prompted for a password for the user account used in step 3. If the correct password is entered, the script generates output similar to:

    Datastore: esx01-local-storage-1 LUN: mpx.vmhba1:C0:T0:L0
            esx01.vmw.local  MOUNTED   ATTACHED
    Datastore: esx02-local-storage-1 LUN: mpx.vmhba1:C0:T0:L0
            esx02.vmw.local  MOUNTED   ATTACHED
    Datastore: iSCSI-1 LUN: naa.600144f0a33bc20000004e9772510001
            esx01.vmw.local  UNMOUNTED   ATTACHED
            esx02.vmw.local  UNMOUNTED   ATTACHED
    Datastore: iSCSI-2 LUN: naa.600144f0a33bc20000004e9772ee0002
            esx01.vmw.local  MOUNTED   ATTACHED
            esx02.vmw.local  MOUNTED   ATTACHED
    Datastore: iSCSI-3 LUN: naa.600144f0a33bc20000004e9773560003
            esx01.vmw.local  MOUNTED   ATTACHED
            esx02.vmw.local  MOUNTED   ATTACHED
    Datastore: iSCSI-4 LUN: naa.600144f0a33bc20000004e9773560004
            esx01.vmw.local  MOUNTED   ATTACHED
            esx02.vmw.local  MOUNTED   ATTACHED
    Datastore: iSCSI-5 LUN: naa.600144f0a33bc20000004e9773570005
            esx01.vmw.local  MOUNTED   ATTACHED
            esx02.vmw.local  MOUNTED   ATTACHED


  5. Confirm that the datastore that you want to detach has been unmounted by checking the UNMOUNTED keyword beneath the applicable datastore name and NAA value.

  6. Detach the device across multiple hosts by running the command:

    ./lunManagement.pl --server vcenter_ip --username user --operation detach --datastore datastore

    Where vcenter_ip is the IP address for the vCenter Server, user is a user with administrative privileges, and datastore is the name of the datastore identified in step 4.

  7. You are prompted for a password and confirmation that you wish to do the operation. After providing the correct password and acknowledging the warning, the tool generates output similar to:

    Detaching LUN "0200000000600144f0a33bc20000004e9772510001434f4d535441" from Host "esx01.vmw.local" ...
            Successfully detached LUN!
    Detaching LUN "0200000000600144f0a33bc20000004e9772510001434f4d535441" from Host "esx02.vmw.local" ...
            Successfully detached LUN!


Additional Information

To delete a datastore instead of unpresenting it, see the relevant documentation:

For related information, see the VMware vSphere Blog post, Best Practice: How to correctly remove a LUN from an ESX host.

For translated versions of this article, see:

See Also

This Article Replaces

2011506

Update History

07/31/2012 - Added link to article 2011506 08/14/2012 - Added link to article 2008507

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 60 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 60 Ratings
Actions
KB: