This issue is resolved
in VMware ESX and ESXi 4.1 Update 3. For more information, see the VMware ESX and ESXi 4.1 Update 3 Release Notes.
Invalid Opcodes
are usually logged by the SCSI (ESX host storage) or VSCSI (virtual machine storage) modules in one of these conditions:
- A SAN target does not support a command or operation issued by an ESX Server:
- This may be as part of regular operation, an agent's request, or by a virtual machine.
- Required commands should not return with
Invalid Opcode
. In such cases, review the SAN configuration for error and compatibility with ESX, and if necessary involve the storage vendor for further assistance.
- The host has received undefined or unsupported storage commands from a virtual machine guest operating system, dispatched by the virtual machine's virtual SCSI adapter:
- These messages are usually the result of running hardware monitoring agents in a guest virtual machine.
- This can result in instability of the virtual machine and its guest operating system, in rare cases. Disabling and/or removal of these agents should be regarded as a priority troubleshooting measure.
- A combination of these two conditions is also possible. This occurs when a virtual machine guest has issued commands or operations through ESX and toward the SAN storage target, but the operation is not recognized or supported by the target.
Note: To understand Opcode logging in greater detail, see the Identification section in this article.
Steps
- If the virtual machine has been converted from physical host hardware, verify that the hardware monitoring agents have been disabled and/or removed prior to the conversion.
- Check the deployed virtual machines for residual agents or processes that were originally intended for use on physical host hardware. While virtualized, hardware management agents are usually unable to provide the full extent of their intended functionality and may instead result in errors or further problems.
- Due to the potential stability issues which may arise from running system/platform-specific hardware agents on virtual hardware, VMware recommends that you disable and/or remove them if present, unless explicitly designed and supported for use in a virtual machine by the respective hardware or application vendor.
Identification
Example: See this vmkernel log output of an ESX server. This message was generated because the SAN target did not support the attempted command or operation.
itvmw15 vmkernel: 0:23:58:15.668 cpu1: 1062)SCSI: 215: Invalid Opcode (0x4d)
In these error messages:
- The involved virtual machine can be identified by examining the CPU value. This number identifies a World ID for a virtual machine.
- The specific illegal SCSI operation or command can be identified by the value specified after the Invalid Opcode message, above. This hexadecimal value would usually conform to SCSI specification. For more information about these values, see SCSI Command Operation Codes at http://www.t10.org/lists/2op.htm.
Note: The preceding link was correct as of July 31, 2010. If you find the link is broken, provide feedback and a VMware employee will update the link.
To correlate this World ID with a specific virtual machine, use one of the methods described in Mapping a virtual machine world number to a virtual machine name (1001101).
Summary of Example
- Hexadecimal value
0x4d
translates as LOG SENSE
. This command is used to obtain current data from the log pages of a SCSI target, for diagnostic purposes. When additional feedback (supported log pages) is provided in response to processing this command (at the SAN target), error counters
can be retrieved by the initiator. This ultimately provides diagnostic metrics for monitoring software. - In this context, the SAN target does not support this command.
- In this example, the guest operating system ran storage monitoring agents and attempted to query these log pages via a
0x4d (LOG SENSE)
command, above, on a Raw Device Mapping (RDM) virtual disk. - To resolve this issue, the guest operating system should be reviewed, by disabling or removing of any hardware or storage monitoring agents responsible.