Search the VMware Knowledge Base (KB)
View by Article ID

Persistent mount of a snapshot LUN may not persist across reboots in VMware ESXi 5.0/5.1/5.5 (2065248)

  • 3 Ratings

Symptoms

  • After mounting a snapshot LUN using the Persistent Mount option:
    • The snapshot LUN is not mounted following an ESXi host reboot.
    • The snapshot LUN mount fails to persist.
    • The forced LUN mount is successful and remains mounted until a reboot occurs.
  • After rebooting the host, the LUN is not mounted.

    Note: It does not matter what method you use to mount the snapshot LUN. If you use the vSphere client or either of the available command line options listed below to mount the volume, the result is the same, failure.

    • esxcli storage vmfs snapshot mount -u <VMFS UUID>
    • vicfg-volume --persistent-mount <VMFS UUID>
For more information on persistent mounts and snapshot LUN handling in ESXi, see:

Cause

When the system parses the list of snapshot LUNs, it checks if the LUNs are Read-Only. If there is a read-only LUN in the list, the parsing stops and any LUNs after the read-only LUN are ignored and remain unmounted.

Resolution

This is a known issue affecting VMware ESXi 5.0, 5.1 and 5.5 hosts.

This issue is resolved in ESXi 5.5 Patch Release ESXi550-201505002. For more information, see VMware ESXi 5.5, Patch Release ESXi550-201505002 (2114245).

To work around this issue, apply one of these two methods:

Method 1

Ensure there are no read-only LUNs presented to the ESXi host.

Note: If two-way replication is in use, this may not be possible.

Method 2

Warning: The steps outlined here are potentially hazardous for your environment. If you are not able to complete the resolution yourself, contact VMware Technical Support.

You must acquire the VMFS UUID of the problematic read-only LUN and filter it out by removing the related entries from ESXi configuration file esx.conf
  1. In the vmkernel.log file, located at /var/log look for an entry similar to:

    2013-11-27T10:19:11.954Z cpu9:16854)LVM: 11710: Failed to open device naa.60060e80167b770000017b1234567890:1

  2. Use vsish to identify the VMFS UUID of the problematic read only LUN:

    Warning: The vsish command set is not supported for customer use, it is undocumented and may produce undesirable results. The below vsish command is used only to identify the snapshot LUN(s) that is causing the problem, there is no equivalent esxcli or esxcfg command. Use only the exact commands listed below.

  3. Open the vsish command prompt using this command:

    # vsish

  4. Change directory to vmkModules/lvmdriver/unresolved/devices/ and run the ls command to show a list of unresolved devices:

    /> cd vmkModules/lvmdriver/unresolved/devices/
    /vmkModules/lvmdriver/unresolved/devices/> ls

  5. Find the the entries that correspond to the NAA ID identified in the log in step 1 and cat the file.

    For example:

    /vmkModules/lvmdriver/unresolved/devices/> cat /vmkModules/lvmdriver/unresolved/devices/0#naa.60060e80167b770000017b1234567890:1/properties

    Unresolved device information {
    VMK name:naa.60060e80167b770000017b1234567890:1
    LV name:5177c75f-8fc1d038-3c1a-d8465649cf66
    LV State:1
    VMFS3 UUID (First extent only):5177c761-5a47bc68-cc73-d8465649cf66
    VMFS3 label (First extent only):MY-DATASTORE-SAN001
    Reason: Snapshot detected; further actions required
    Extent address start (MB):0
    Extent address end (MB):524031
    Volume total size (MB):524032
    }

  6. Open /etc/vmware/esx.conf using a plain text editor.
  7. Find and remove the 3 lines that reference the UUID identified in step 5.

    /fs/vmfs[5177c761-5a47bc68-cc73-d8465649cf66]/forceMountedLvm/readOnly = "false"
    /fs/vmfs[5177c761-5a47bc68-cc73-d8465649cf66]/forceMountedLvm/lvmName = "5177c75f-8fc1d038-3c1a-d8465649cf66"
    /fs/vmfs[5177c761-5a47bc68-cc73-d8465649cf66]/forceMountedLvm/forceMount = "true"

    Note: There may be more than one read-only LUN presented, so check the logs for similar messages after you have applied this workaround.
    Note: A reboot of the host is required after editing the /etc/vmware/esx.conf

Impact/Risks

THE CONTENT OF THIS ARTICLE IS PROVIDED "AS-IS," AND TO THE MAXIMUM 
EXTENT PERMITTED BY APPLICABLE LAW, VMWARE DISCLAIMS ALL OTHER 
REPRESENTATIONS AND WARRANTIES, EXPRESS OR IMPLIED, REGARDING THIS 
CONTENT, INCLUDING THEIR FITNESS FOR A PARTICULAR PURPOSE, THEIR 
MERCHANTABILITY, OR THEIR NONINFRINGEMENT.  VMWARE SHALL NOT BE LIABLE 
FOR ANY DAMAGES ARISING OUT OF OR IN CONNECTION WITH THE USE OF THIS 
CONTENT, INCLUDING DIRECT, INDIRECT, CONSEQUENTIAL DAMAGES, LOSS OF 
BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF VMWARE HAS BEEN ADVISED 
OF THE POSSIBILITY OF SUCH DAMAGES.

Additional Information

To be alerted when this article is updated, click Subscribe to Document in the Actions box.

For information on the modification of esx.conf, see Corrupted esx.conf file prevents network settings from displaying (1004451).

See Also

Update History

05/18/2016 - Updated the changes in Resolution Section related to fix

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 3 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 3 Ratings
Actions
KB: