Search the VMware Knowledge Base (KB)
View by Article ID

"The ramdisk "vsantraces" is full" error reports in vSAN logging (2150320)

  • 0 Ratings

Symptoms

  • Logging in to vSAN host through Log Insight or through the vSphere Web Client fails.
  • In the vmkernel.log and vobd.log files, you see this error:

    The ramdisk "vsantraces" is full

Resolution

It is possible in vSAN, especially when ESXi is installed to a USB or SD device, that the ramdisk for vSAN trace information may grow and fill the allotted ramdisk. We can redirect this logging to another location, depending on what is available:

vSAN traces can grow to any size you select, but VMware recommend around 600 MB if available.

Note: 300 MB ramdisk is the default maximum size when installing to USB/SD.

If we have persistent logging somewhere external the vSAN, we can redirect using these steps. If no other storage exists, skip to the end of this section to limit vsantraces to 200 MB.

If external storage exists
  1. Connect to each ESXi server in the cluster using SSH.
  2. To set Configure VSAN trace, run this command:

    [root@localhost:~] esxcli vsan trace
    Usage: esxcli vsan trace set [cmd options]

    Note: This command is not thread safe.

    Cmd options:
    • -f|--numfiles: Log file rotation for vSAN trace files.
    • -p|--path: Path to store vSAN trace files.
    • -r|--reset: When set to true, reset defaults for VSAN trace files.
    • -s|--size: Maximum size of vSAN trace files in MB.
If the Scratch folder has already been redirected, we can also redirect vsantraces into the same folder. There may be unique folders to each host that already exist on your storage for this if previously configured. In this case, create a folder called vsantraces so these can be separated out per-Host. If one folder per Host is not there, create one folder for each. To create one folder for each host, for the corresponding vSphere version, see Creating a persistent scratch location for ESXi 4.x/5.x/6.x.
 
Log-bundles for vSAN becomes very large, therefore, the host-folders must be separated. We collect the entire vsantraces folder. If all Hosts are in the same folder, the log collection grabs every file, every time, duplicating the entire folder per host-bundle.
After we have folder for each Host and the logging is separated, we can redirect traces using these commands:
  • mkdir /vsantraces
  • esxcli vsan trace set -f 10 -s 60 -p /vsantraces

For example:
 
If files are located at /vmfs/volumes/Datastore/scratch/localhost/, run these commands:
  • mkdir /vmfs/volumes/Datastore/scratch/localhost/vsantraces/
  • esxcli vsan trace set -f 10 -s 60 -p /vmfs/volumes/Datastore/scratch/localhost/vsantraces/
Note: The preceding commands are just an example. The location may vary based on environment.

This automatically begins sending trace files to the new location, no further configuration is needed.
 
If No External Storage is Available
 
If no additional storage is available, limit the size of vsantraces to 200MB. This allows a bit of overhead for additional files and ensures that we do not fill the ramdisk.
 
You can do this by running this command on every vSAN host:

esxcli vsan trace set -f 10 -s 20

These changes are immediate and older files will be rolled off as smaller ones take their places over time.

No services require restart to enable this change.

Tags

vSAN Logging, vSAN, ramdisk vsantraces full

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 0 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 0 Ratings
Actions
KB: