Search the VMware Knowledge Base (KB)
View by Article ID

Unable to access certain files on a VMFS datastore (1012036)

  • 25 Ratings

Symptoms

  • You have problems accessing certain files on a VMFS datastore.
  • You cannot modify or erase files on a VMFS datastore.
  • Attempting to read files on a VMFS datastore may fail with the error:

    invalid argument

  • You see entries in the /var/log/vmkernel file similar to:

    • vmkernel: 25:21:39:57.861 cpu15:1047)FS3: 130: <START termserv2-5160fe37.vswp>
      vmkernel: 25:21:39:57.861 cpu15:1047)Lock [type 10c00001 offset 52076544 v 69, hb offset 4017152
      vmkernel: gen 109, mode 1, owner 4a15b3a2-fd2f4020-3625-001a64353e5c mtime 3420]
      vmkernel: 25:21:39:57.861 cpu15:1047)Addr <4, 1011, 10>, gen 36, links 1, type reg, flags 0x0, uid 0, gid 0, mode 600
      vmkernel: 25:21:39:57.861 cpu15:1047)len 3221225472, nb 3072 tbz 0, zla 3, bs 1048576
      vmkernel: 25:21:39:57.861 cpu15:1047)FS3: 132: <END termserv2-5160fe37.vswp>


    • vmkernel: 0:00:20:51.964 cpu3:1085)WARNING: Swap: vm 1086: 2268: Failed to open swap file '/volumes/4730e995-faa64138-6e6f-001a640a8998/mule/mule-560e1410.vswp': Invalid metadata
      vmkernel: 0:00:20:51.964 cpu3:1085)WARNING: Swap: vm 1086: 3586: Failed to initialize swap file '/volumes/4730e995-faa64138-6e6f-001a640a8998/mule/mule-560e1410.vswp': Invalid metadata

    • cpu11:268057)WARNING: HBX: 599: Volume 50fd60a3-3aae1ae2-3347-0017a4770402 ("<Datastore_name>") may be damaged on disk. Corrupt heartbeat detected at offset 3305472: [HB state 0 offset 6052837899185946624 gen 15439450 stampUS 5 $

Resolution

A VMFS datastore has a region designated for heartbeat types of operations to ensure that distributed access to the volume occurs safely. When files are being updated, the heartbeat region for those files is locked by the host until the update is complete. The Invalid metadata status indicates that the content of the heartbeat region is not correct. The subset of files managed by that region are likely to have been locked by a single host that crashed at some point (although other reasons are possible).

If a corrupt heartbeat region is at fault, an ESXi/ESX host that tries to access those files receives the same symptoms and the same hb offset value.

vmkfstools -D filename generates the same output.

To resolve this issue, file a support request with VMware Support and note this Knowledge Base article ID (1012036) in the problem description. For more information on filing a support request, see How to Submit a Support Request.

Additional Information

For translated versions of this article, see:

To be alerted when this article is updated, click Subscribe to Document in the Actions box.

Tags

cannot-access-datastore-files

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 25 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 25 Ratings
Actions
KB: