Search the VMware Knowledge Base (KB)
View by Article ID

Guest Introspection memory usage spikes to 90+% or you see the error: "Lost communication with ESX module" in NSX-V 6.3 (2151235)

  • 1 Ratings

Symptoms

In a VMware NSX for vSphere 6.3.x environment where Guest Introspection is installed, you see these symptoms:

  • Intermittent network I/O disruption on virtual machines protected by the Guest Introspection (GI) agentless antivirus protection.
  • Memory usage on the Guest Introspection USVM spikes to nearly 100% at the same time impact to virtual machines is observed.
  • In the Tasks and Events tab for the ESXi host on which the affected virtual machines are running, you see the error similar to:

    Lost communication with ESX module.


Note:

For this article to match your issue, all of the following must be true:

  • You have NSX for vSphere 6.3.x installed.
  • Guest Introspection is installed in the cluster(s) where this issue is observed.
  • Both the Guest Introspection (vsepflt.sys) and the Network Introspection driver (vnetflt.sys) are installed in the affected guest virtual machines.

Resolution

This is a known issue affecting Guest Introspection in the NSX for vSphere 6.3.x releases.

Currently, there is no resolution.

To temporarily work around this issue, do one of the following:
  • Reboot the Guest Introspection USVM on the ESXi host.
  • Place the ESXi host on maintenance mode. Once all virtual machines have been evacuated, reboot the ESXi host.

Note: This issue could re-occur unless a more persistent workaround is applied.

For a more persistent workaround, remove the network introspection driver (vnetflt.sys) from the affected guest virtual machines.

To un-install vnetflt.sys, you can use one of the following three methods:

  • Re-run the VMware Tools installer in the affected guest virtual machine with the following additional switch: /s /v"/qn REMOVE=NetworkIntrospection"
  • Alternatively, you can also manually run the VMware Tools installer inside your guest virtual machine and remove the Network Introspection driver. To do this, select "Custom" installation, expand the VMCI driver section, and under vShield Drivers, click on NSX Network Introspection driver and select "Entire feature will be unavailable". Subsequently, you will need to reboot the guest virtual machine for the changes to take effect.
  • There is also automation available to assist with vnetflt removal from guest virtual machines. To get assistance with automating this task, file a support request with VMware Support and quote this Knowledge Base article ID (2151235) in the problem description. For more information, see Filing a Support Request in My VMware (2006985) or How to Submit a Support Request.
Notes:
  • Currently known workarounds will disable Activity Monitoring and Identity Firewall functionality. If you are using either of these features and are experiencing this issue, please clearly indicate this in your support request.
  • Other workarounds have been identified and are available to use, if they apply to your scenario. Please open a support ticket with VMware Technical Support if you believe you are experiencing this issue

Additional Information

To be alerted when this article is updated, click Subscribe to Document in the Actions box.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 1 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 1 Ratings
Actions
KB: