Search the VMware Knowledge Base (KB)
View by Article ID

Unmanaged workload is detected on datastore running SIOC (1020651)

  • 34 Ratings

Details

This alarm may appear in the vCenter vSphere Client:

Non-VI workload detected on the datastore

A warning message similar to one of these may also appear in the vCenter vSphere Client:

An external I/O activity is detected on datastore “datastore”, this is an unsupported configuration. Consult the Resource Management Guide or follow the Ask VMware Link for more information
An external I/O workload is detected on datastore “datastore
 
This informational event alerts the user of a potential misconfiguration or I/O performance issue caused by a non-ESX workload. It is triggered when Storage I/O Control (SIOC) detects that a workload that is not managed by SIOC is contributing to I/O congestion on a datastore that is managed by SIOC. (Congestion is defined as a datastore's response time being above the SIOC threshold.) Specific situations that can trigger this event include:
  • The storage array is performing a system operation such as replication or RAID reconstruction.
  • VMware Consolidated Backup or vStorage APIs for Data Protection are accessing a snapshot on the datastore for backup purposes.
  • The storage media (spindles, SSD) on which this datastore is located is shared with volumes used by non-vSphere workloads
  • The host is running in an unsupported configuration.
SIOC continues to work during these situations. This event can be ignored in many cases and you can disable the associated alarm once you have verified that none of the potential misconfigurations or serious performance issues are present in your environment. As explained in detail below, SIOC ensures that the ESX workloads it manages are able to compete for I/O resources on equal footing with external workloads. This event notifies the user of what is happening, provides the user with the opportunity to better understand what is going on, and highlights a potential opportunity to correct or optimize the infrastructure configuration.
 
Note: Starting with vSphere 5.0, SIOC is supported with NFS storage. However, Raw Device Mapping (RDM) virtual disks are not supported. This includes both Physical Mode and Virtual Mode RDMs as well as RDMs used for Microsoft Cluster Server (MSCS). Datastores with multiple extents are also not supported with SIOC. This alarm could be triggered if these storage objects are configured. 

This article helps you understand what is causing the event, and what, if any, action is required to address the cause.

Example

Scenario 1

A /vmfs/volumes/shared-LUN datastore is accessible across multiple hosts. Some hosts are running ESX version 4.1 or later and others are either running an older version or are outside the control domain of vCenter Server.
 
Scenario 2

The array being used for vSphere is also being used for non-vSphere workloads. The non-vSphere workloads are accessing a storage volume that is on the same disk spindles as the affected datastore.

Impact

When SIOC detects that datastore response time has exceeded the threshold, it typically throttles the ESX workloads accessing the datastore to ensure that the workloads with the highest shares get preference for I/O access to the datastore and lower I/O response time. However, such throttling is not appropriate when workloads not managed by SIOC are accessing the same storage media. Throttling in this case would result in the external workload getting more and more bandwidth, while the vSphere workloads get less and less. Therefore, SIOC detects the presence of such external workloads, and as long as they are present while the threshold is being exceeded, SIOC competes with the interfering workload by reducing its usual throttling activity.

SIOC automatically detects when the interference goes away and resumes its normal behavior. In this way, SIOC is able to operate correctly even in the presence of interference. The vCenter Server event is notifying the user that SIOC has noticed and handled the interference from external workloads.

Note: When an external workload is acting to drive the datastore response time above the SIOC threshold, the external workload might cause I/O performance issues for vSphere workloads. In most cases, SIOC can automatically and safely manage this situation. However, there may be an opportunity to improve performance by changing some aspects of your configuration. The next section provides guidance on this.


Solution

These unsupported configurations can result in the triggering of the alarms or the Warning message as seen above:

  • One or more hosts accessing the datastore are running an ESX version older than 4.1.
  • One or more hosts accessing the datastore are not managed by vCenter Server.
  • Not all of the hosts accessing the datastore are managed by the same vCenter Server.
  • The storage media (spindles, SSD) where this datastore is located are shared with other datastores that are not SIOC enabled.
  • Datastores in the configuration have multiple extents.
Ensure that you are not running the above configurations when using SIOC for Congestion Management:
 
Note: SIOC can be used even when External Workloads are using the same underlying storage array as vSphere. This could be detrimental to the I/O performance of the virtual machines. To function in presence of external workloads, SIOC needs to throttle virtual workloads; around 10% average if the external workload interference is continuous. If this is not acceptable, currently, disabling SIOC is the only solution. But, if the external workload is temporal, for example, for a few hours and there is sufficient capacity on the array, enabling SIOC acts as an insurance against IO performance problems that may happen due to mis-configurations in a shared storage virtual environment. 
  • Can you disable and successfully re-enable congestion management for the affected datastore?
Disable and attempt to re-enable congestion management for the affected datastore. If the event occurred because the configuration includes hosts that are running an older version of ESX and the hosts are managed by the same vCenter Server, vCenter Server detects the problem and does not allow you to re-enable congestion management. When the older hosts are updated to ESX 4.1 or later, or the hosts are disconnected from the affected datastore, you can enable congestion management.
  • Are hosts that are not managed by this vCenter Server accessing the affected datastore?
If disabling and re-enabling congestion management for the affected datastore does not solve the problem, other hosts that are not managed by this vCenter Server might be accessing the datastore.

Verify that the datastore is shared across hosts that are managed by different vCenter Server systems or are not managed hosts. If so, perform one of these actions:
    • Add all the hosts that access the affected datastore to the same vCenter Server or
    • Migrate the datastore's workloads to hosts that are managed by the same vCenter Server.

  • Do all datastores in the configuration that share the same physical storage media (spindles, SSD) have the same SIOC configuration?

    All datastores that share physical storage media must share the same SIOC configuration — all enabled or all disabled. In addition, if you have modified the default congestion threshold setting, all datastores that share storage media must have the same setting.

  • Are any SIOC-enabled datastores in the configuration backed up by multiple extents?

    SIOC-enabled datastores must not be backed up by multiple extents.

If none of the above scenarios apply to your configuration and you have determined that you are running a supported configuration, but are still seeing this event, investigate possible I/O throttling by the storage array.

If an environment is known to have shared access to datastores or performance constraints, it may be preferable to disable the Alarm in vCenter Server. For more information, see Working with Alarms in the vSphere 4.1 Datacenter Administration Guide.
 
This flowchart describes the troubleshooting process for VMware Storage I/O Control:


To disable the Alarm:

  1. Log in to vCenter Server using the vSphere Client.
  2. Click the vCenter Server in the Inventory.
  3. Click the Alarms tab.
  4. Click the Definitions sub-tab. A list of defined alarms is displayed.
  5. Right-click the Unmanaged workload is detected on the datastore alarm and click Edit Settings.
  6. Deselect Enable this alarm.
"The /var/log/vobd.log file in the ESXi host may contain 

< Detected a non vi workload on datastore > followed by the friendly name of the affected VMFS datastore."

Additional Information

For translated versions of this article, see:

Tags

Unmanaged workload is detected on the datastore

Keywords

SIOC, congestion, I/O shares, storage management, I/O control, external

This Article Replaces

1031116

Update History

07/10/2012- Added information that RDM virtual disks, including those used for MSCS, are not supported.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 34 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 34 Ratings
Actions
KB: