Monitoring VCSA using SNMP
search cancel

Monitoring VCSA using SNMP

book

Article ID: 345645

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

This article provides an overview of SNMP monitoring of the VMware vCenter Server Appliance (VCSA) using the built-in SNMPv1/v2c/v3 agent.
This article helps users who wish to move from a Windows based Virtual Center Server to a VCSA appliance deployment and need to update their monitoring infrastructure to work with the Appliance.


Symptoms:
 
 


Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.0.x

Cause

Virtual Center Server on Windows is a traditional application that requires operators to watch processes, memory, cpu and filesystems.
The Virtual Center Server Appliance moves to an embedded system for which the Operating System becomes opaque to the operator, and specific APIs are made available for these purposes.

Resolution

To use the SNMP agent in VCSA, the experience is the same as it is in ESXi. The ESXi and SNMP agents share similar configuration and provide consistent monitoring of memory/process, cpu, filesystems, and networking.

For more information about SNMP and ESXi, see Understanding the SNMP Layer 2 networking in the VMware ESXi (2118059).
 
The VCSA basic SNMPv3 configuration used as an example in this KB is configured with securityLevel noAuthNoPriv. In the steps that follow you will check the current configuration, understand the variables that are applicable and explore the variables you can monitor and configure, based on your environment type.
 
Note: VMware recommends using SNMPv3 with security level authPriv for all production environments.
 
 
The sections that follow explain how to check the current SNMP configuration, and the key parts of the agent to poll for using a command line prompt or using a MIB browser.
 

Checking the current SNMP configuration

To enable remote polling of the VCSA SNMP agent from the ssh command line:
  1. Open an shell session to the Virtual Center Server Appliance and run these commands:

    Command> snmp.set --users test/-/-/none
    snmp.enable

    For more information about accessing the VCSA shell, see the Using the Appliance Shell to Configure the vCenter Server Appliance section of the vCenter Server Appliance 6.0 Configuration guide.
     
  2. Run the snmp.get command to check the current SNMP configuration:

    For example in a production deployment, here is a configured user named 'test' with securityLevel authPriv:

    Command> snmp.get
    Config:
    Enable: True
    Users:
    Configuration:
    Username: test
    Priv_key: bf91b2fb426a97121be51dd52eec494c83495254
    Sec_level: priv
    Auth_key: e03858abcfd4ad5fe99d52ce02d72174d6469190
    Notraps:
    Privacy: AES128
    Loglevel: warning
    V3targets:
    Configuration:
    Port: 1162
    Ip: 192.0.2.151
    Type: trap
    User: dq
    Sec_level: priv
    Pid: 13391
    Syslocation: PaloAlto
    Targets:
    Remoteusers:
    Authentication: SHA1
    Communities:
    Engineid: 766d77617265
    Port: 161
    Syscontact: [email protected]
 

Which agent is this?

In a distributed system where common mgmt APIs exist, the first task is to validate the remote system.
An agent will provide two variables or "managed objects" for this task, one is for humans and one for programs:
  • Human readable objects are called sysDescr:

    .iso.org.dod.internet.mgmt.mib-2.system.sysDescr.0 = STRING: VMware-vCenter-Server-Appliance 6.0.0.5110 embedded build 2656759 VMware, Inc x86_64
     
  • Programs use sysObjectID:

    .iso.org.dod.internet.mgmt.mib-2.system.sysObjectID.0 = OID: .iso.org.dod.internet.private.enterprises.vmware.vmwProductSpecific.vmwEmbeddedVirtualCenterAppliance
    which translates to: .1.3.6.1.4.1.6876.4.7
 
The translation between the OID .1.3.6.1.4.1.6876.4.7 and the name vmwEmbeddedVirtualCenterAppliance is maintained in the VMWARE-PRODUCTS-MIB.mib file. All VMware SNMP agents are uniquely identified and will be found in this MIB module.

These two objects should be polled to maintain the assertion that the device being polled is what central management thinks it should be. SNMP is widely used so these same objects work for any number of systems in the IT enterprise.
 

Identification of VMware products

If the VCSA is deployed standalone, then the OID reports vmwEmbeddedVirtualCenterAppliance.
 
If VCSA is configured or deployed in two different Appliances, the OID will report each uniquely:
  • The SSO/Identity appliance reports as:

    SNMPv2-MIB::sysDescr.0 = STRING: VMware-vCenter-Server-Appliance 6.0.0.5110 infrastructure build 2656759 VMware, Inc x86_64
    .iso.org.dod.internet.mgmt.mib-2.system.sysObjectID.0 = OID: .iso.org.dod.internet.private.enterprises.vmware.vmwProductSpecific.vmwInfrastructureAppliance

    Or if the MIB modules are not loaded in your management system it will display as:

    .1.3.6.1.4.1.6876.4.8
     
  • The Virtual Center function (the vpxd process) reports as:

    SNMPv2-MIB::sysDescr.0 = STRING: VMware-vCenter-Server-Appliance 6.0.0.5110 management build 2656759 VMware, Inc x86_64
    .iso.org.dod.internet.mgmt.mib-2.system.sysObjectID.0 = OID: .iso.org.dod.internet.private.enterprises.vmware.vmwProductSpecific.vmwManagementAppliance

    Or if the MIB modules are not loaded in your management system it will display as:

    .1.3.6.1.4.1.6876.4.9

    To download VMware MIB modules, see SNMP MIB module file download (1013445).
These VMware specific variables are used across VMware products to uniquely identify the "product":
 
VMWARE-SYSTEM-MIB::vmwProdName.0 = STRING: VMware-vCenter-Server-Appliance
VMWARE-SYSTEM-MIB::vmwProdVersion.0 = STRING: 6.0.0.5110
VMWARE-SYSTEM-MIB::vmwProdBuild.0 = STRING: 2656759
 

Firmware version information for BIOS and NIC

Firmware version information for BIOS and NIC can be found in the RFC 2790 hrDeviceTable and RFC 4113 entPhysicalTable.
For example:
 
BIOS version is reported by the variable:

ENTITY-MIB::entPhysicalFirmwareRev.1 = STRING: 6.00
 
Network Adapter (NIC) driver and version is reported in the hrDeviceTable's hrDeviceDescr variable:
 
HOST-RESOURCES-MIB::hrDeviceDescr.3 = STRING: NIC eth0, driver vmxnet3, version: 1.4.2.0-k-NAPI, 0000:03:00.0
 
The same format is also found on ESXi, for example:
 
HOST-RESOURCES-MIB::hrDeviceDescr.2 = STRING: NIC vmnic0, driver tg3, version 3.131d.v60.4, 0000:001:00.0
 

Locating a given system's unique Identifier (UUID)

To uniquely identify a given appliance the ENTITY-MIB (RFC 4113) provides the UUID and manufacturer identity:
 
ENTITY-MIB::entPhysicalName.1 = STRING: HostnameSetByOperator
ENTITY-MIB::entPhysicalSerialNum.1 = STRING: VMware-42 03 ac e7 2e c6 5f 8c-e
ENTITY-MIB::entPhysicalModelName.1 = STRING: VMware Virtual Platform
ENTITY-MIB::entPhysicalMfgDate.1 = STRING: 2014-9-30,0:0:0.0
ENTITY-MIB::entPhysicalUris.1 = STRING: "urn:uuid:4203ace72ec65f8ce9187e868c568fdc"
 
The variable entPhysicalName reports the plain hostname, for the fully qualified domain name (FQDN) see sysName:
 
SNMPv2-MIB::sysName.0 = STRING: HostnameSetByOperator3.example.com
 

Where are the agents located/situated?

An agent can be found relative to either an operator defined locator string known as sysLocation
For example:
 
SNMPv2-MIB::sysLocation.0 = STRING: PaloAlto
 
This operator is set by the CLI command:
 
snmp.set --syslocation PaloAlto
 
OR by relation to another system in ENTITY-MIB entLogicalTable
 
For example, all ESXi system SNMP agents report an entry in the entLogicalTable containing the Virtual Center system that controls it or by its networking IP/MAC address:
 
IP-MIB::ipNetToPhysicalPhysAddress.2.ipv4."10.160.95.253" = STRING: 0:0:c:9f:fa:2a
 
Some platforms may report the hardware vendor's SNMP agent running in the BMC. The entLogicalTable provides a way to discover topology of a distributed set of software using a unique OID/identifier to define the relationship. For example, when VCSA is deployed with the Platform Service Controller (PSC) and vCenter Server (VC) in separate appliances, the VC (management) node will point to the PSC (infrastructure) node in the entLogicalTable.
For example:
 
ENTITY-MIB::entLogicalDescr.0 = STRING: Platform Services Controller, infrastructure node
ENTITY-MIB::entLogicalType.0 = OID: VMWARE-PRODUCTS-MIB::vmwInfrastructureAppliance
ENTITY-MIB::entLogicalTAddress.0 = STRING: "10.160.52.14"
ENTITY-MIB::entLogicalTDomain.0 = OID: TRANSPORT-ADDRESS-MIB::transportDomainUdpIpv4
ENTITY-MIB::entLogicalContextEngineID.0 = ""
ENTITY-MIB::entLogicalContextName.0 = STRING:
ENTITY-MIB::entLastChangeTime.0 = Timeticks: (1137) 0:00:11.37
 
Lastly, the software inventory hrSWInstalledTable provides basic identity for each software component in the system:
 
HOST-RESOURCES-MIB::hrSWInstalledName.382 = STRING: "VMware, Inc.,VMware-vpxd,6.0.0,2656761"
HOST-RESOURCES-MIB::hrSWInstalledDate.382 = STRING: 2015-7-27,21:6:17.0
HOST-RESOURCES-MIB::hrSWInstalledName.364 = STRING: "VMware, Inc,vmware-snmp,1.0.0,1"
HOST-RESOURCES-MIB::hrSWInstalledDate.364 = STRING: 2015-7-27,21:5:53.0
 
In the example entries, you see this appliance has packaged the Virtual Center Server and SNMP agent, and lists their installation date and time.
 

Monitoring a system using the SNMP agent

System time

Querying the agents status and health is often answered using multiple questions.
 
The first, most common question is how long has the system been up? RFC 2780 HOST-RESOURCES-MIB variable hrSystemUptime provides this information:
 
HOST-RESOURCES-MIB::hrSystemUptime.0 = Timeticks: (6566100) 18:14:21.00
 
This should not be confused with:
 
SNMPv2-MIB::sysUpTime = Timeticks: (5642377) 15:40:23.77
 
This variable tracks the time the snmpd program has been running for and resets each time the snmpd agent is restarted.
When snmpd is restarted, there are two related events, coldStart and warmStart. The coldStart event is sent when the agent is started for the first time, to distinguish from the warmStart event, which is sent on subsequent starts of the agent where the appliance is not restarted.
 
SNMP will report time relative to when the agent started, eg sysUptime in most variables that deal with time.
To get the actual wall clock time, the variable hrSystemDate is available:
 
HOST-RESOURCES-MIB::hrSystemDate.0 = STRING: 2015-7-28,15:19:37.0

System boot

Configuration information on how the system was started and where it booted from is provided in the hrSystemInitialLoadParameters variable:
 
HOST-RESOURCES-MIB::hrSystemInitialLoadParameters.0 = STRING: "root=/dev/sda3 append resume=/dev/sda2 splash=silent crashkernel= noexec=on nousb audit=1 vga=0x311
 
What device it booted from. Here you see that device number 4 was the boot device:
 
HOST-RESOURCES-MIB::hrSystemInitialLoadDevice.0 = INTEGER: 4

What was the the attached virtual disk:
 
HOST-RESOURCES-MIB::hrDeviceDescr.4 = STRING: LUN VMware Virtual_disk 1.0 sda
 
Finally here we show s how to see memory/processes, cpu and disk and networking. These are resources which can become exhausted. HOST-RESOURCES-MIB provides the bulk of the variables to look at:
 

Memory and processes

Memory and processes are reported mainly in hrStorageTable, hrSWRunTable and their related sub-tables.
For example:
 
HOST-RESOURCES-MIB::hrMemorySize.0 = INTEGER: 8194012 KBytes
 
The available memory is being consumed by 179 processes:
 
HOST-RESOURCES-MIB::hrSystemProcesses.0 = Gauge32: 179
HOST-RESOURCES-MIB::hrStorageDescr.14 = STRING: Real Memory
HOST-RESOURCES-MIB::hrStorageAllocationUnits.14 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageSize.14 = INTEGER: 8194012
HOST-RESOURCES-MIB::hrStorageUsed.14 = INTEGER: 41140
HOST-RESOURCES-MIB::hrStorageAllocationFailures.14 = Counter32: 0
 
Each process in the system can be tracked for CPU and memory use. Starting with the process list in hrSWRunTable, hrSWRunName provides the process name and process id.
For example:
 
HOST-RESOURCES-MIB::hrSWRunPath.27982 = STRING: "[vpxd]"
HOST-RESOURCES-MIB::hrSWRunPath.30696 = STRING: "/bin/snmpd"
 
You see that vpxd is process id 27982 and the snmpd process id is 30696. These do not change frequently. If you see these process IDs changing frequently on the order of minutes to hours, it is recommended to collect a support bundle and submit a support request with this KB article ID in the SR description for review by VMware Support. To upload a bundle see the links in the Additional Information section.

The vpxd daemon may be restarted by operator commands and by watchdog restart in the case of failure. To understand watchdog, see Supported vCenter Server high availability options (1024051).

 
Now, let's find more details on vpxd process, is it running or runnable? How much CPU and memory is in use?
 
HOST-RESOURCES-MIB::hrSWRunStatus.27982 = INTEGER: runnable(2)
HOST-RESOURCES-MIB::hrSWRunPerfCPU.27982 = 14178
HOST-RESOURCES-MIB::hrSWRunPerfMem.27982 = INTEGER: 34037760 KBytes
 
These values are cached over 5 seconds.
From the running process we can locate the software package using the variable hrSWRunName which should be the same value as hrSWInstalledName to locate the software package version data for a given running program:
 
HOST-RESOURCES-MIB::hrSWRunName.27982 = STRING: "VMware, Inc.,VMware-vpxd,6.1.0,3719558"
 
Next, let's see what ports are open by the vpxd process using the snmpwalk command:
 
snmpwalk -v2c -c public 10.160.3.123 tcpConnectionProcess | grep 27982
TCP-MIB::tcpConnectionProcess.ipv4."10.160.3.123".54978.ipv4."10.160.3.123".443 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".8085.ipv4."127.0.0.1".59734 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".8089.ipv4."127.0.0.1".52134 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".8089.ipv4."127.0.0.1".53526 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".8089.ipv4."127.0.0.1".53654 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".8089.ipv4."127.0.0.1".53848 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".57124.ipv4."127.0.0.1".10080 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58166.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58168.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58170.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58172.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58174.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58176.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58178.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58180.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58182.ipv4."127.0.0.1".5432 = Gauge32: 27982
TCP-MIB::tcpConnectionProcess.ipv4."127.0.0.1".58184.ipv4."127.0.0.1".5432 = Gauge32: 27982
 
Note: All ports vpxd opens are on the loopback (127.0.0.1) interface, nothing is directly accessible from the ethernet NIC.

How then does remote traffic get to vpxd? Let's see which port has tcp/443 port then, dumping all entries for tcpListenerProcess
 
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".443 = Gauge32: 9073
 
Checking this SWRunName, it resolves to rhttpproxy:
 
HOST-RESOURCES-MIB::hrSWRunName.9073 = STRING: "VMware, Inc.,VMware-rhttpproxy,6.1.0,3719558"
 
The rhttpproxy takes input from the external network and sends it over loopback interface to vpxd for processing and visa versa.
So for monitoring, we can see that vpxd has a direct dependency on the rhttp proxy.
 
The vpxd daemon also opens udp ports, so checking that protocol applies as well using the variable udpEndpointProcess.
 

CPU utilization

The agent reports a one minute load average per processor in the range 0-100 where 100 is full utilization.
In this example, you see two cpus in this virtual appliance using less than 15-20% of each CPU:
 
HOST-RESOURCES-MIB::hrProcessorLoad.1 = INTEGER: 15
HOST-RESOURCES-MIB::hrProcessorLoad.2 = INTEGER: 20
 
More details of the processors are available in hrDeviceTable:
 
HOST-RESOURCES-MIB::hrDeviceDescr.2 = STRING: CPU Pkg/ID/Node: 0/0/0 Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz
HOST-RESOURCES-MIB::hrDeviceDescr.2 = STRING: CPU Pkg/ID/Node: 0/1/0 Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz
 

Filesystem usage

Tracking resource utilization is a primary operational concern. Filesystems may be monitored using the HOST-RESOURCES-MIB across all VMware appliances that support SNMP, offering consistent API. Filesystems are mapped to their underlying devices, either local or remote storage and to the partitions they belong to.
 
The VCSA VMware SNMP agent reports real time disk, partitions and filesystem configuration. Of most immediate attention will be the filesystem usage and cache results retrieved from the underlying operating systems at 5 second cache rates. The variable hrStorageDescr, like for memory also contains the filesystem mount points/paths and their use.
 
HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: /dev
HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: /dev/shm
HOST-RESOURCES-MIB::hrStorageDescr.4 = STRING: /run
HOST-RESOURCES-MIB::hrStorageDescr.5 = STRING: /sys/fs/cgroup
HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.7 = STRING: /boot
HOST-RESOURCES-MIB::hrStorageDescr.8 = STRING: /storage/core
HOST-RESOURCES-MIB::hrStorageDescr.9 = STRING: /storage/log
HOST-RESOURCES-MIB::hrStorageDescr.10 = STRING: /storage/db
HOST-RESOURCES-MIB::hrStorageDescr.11 = STRING: /storage/dblog
HOST-RESOURCES-MIB::hrStorageDescr.12 = STRING: /storage/seat
HOST-RESOURCES-MIB::hrStorageDescr.13 = STRING: /storage/netdump
HOST-RESOURCES-MIB::hrStorageDescr.14 = STRING: /storage/autodeploy
HOST-RESOURCES-MIB::hrStorageDescr.15 = STRING: /storage/imagebuilder
HOST-RESOURCES-MIB::hrStorageDescr.16 = STRING: /storage/invsvc
HOST-RESOURCES-MIB::hrStorageDescr.17 = STRING: /var/spool/snmp
 
For the root filesystem, the index is 6 and its usage is found in:
 
HOST-RESOURCES-MIB::hrStorageSize.6 = INTEGER: 2771006
HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 1184911
HOST-RESOURCES-MIB::hrStorageAllocationUnits.6 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationFailures.6 = Counter32: 0
HOST-RESOURCES-MIB::hrStorageType.6 = OID: HOST-RESOURCES-MIB::hrStorageType.hrStorageFixedDisk
 
It can further provide details about network disks (hrStorageNetworkDisk) or RAM disks (hrStorageRam) and if a filesystem is read only or readwrite.
Partition names are found in:
 
HOST-RESOURCES-MIB::hrPartitionLabel.4.1 = STRING: "sda1"
HOST-RESOURCES-MIB::hrPartitionLabel.4.2 = STRING: "sda2"
HOST-RESOURCES-MIB::hrPartitionLabel.4.3 = STRING: "sda3"
HOST-RESOURCES-MIB::hrPartitionLabel.4.4 = STRING: "sda1"
HOST-RESOURCES-MIB::hrPartitionLabel.4.5 = STRING: "sda2"
 
The LUN/Disk devices are reported in hrDeviceTable:
 
HOST-RESOURCES-MIB::hrDeviceDescr.4 = STRING: LUN VMware Virtual_disk 1.0 sda
HOST-RESOURCES-MIB::hrDeviceDescr.5 = STRING: LUN VMware Virtual_disk 1.0 sdb
HOST-RESOURCES-MIB::hrDeviceDescr.6 = STRING: LUN VMware Virtual_disk 1.0 sdc
HOST-RESOURCES-MIB::hrDeviceDescr.7 = STRING: LUN VMware Virtual_disk 1.0 sdd
HOST-RESOURCES-MIB::hrDeviceDescr.8 = STRING: LUN VMware Virtual_disk 1.0 sde
HOST-RESOURCES-MIB::hrDeviceDescr.9 = STRING: LUN VMware Virtual_disk 1.0 sdf
HOST-RESOURCES-MIB::hrDeviceDescr.10 = STRING: LUN VMware Virtual_disk 1.0 sdg
HOST-RESOURCES-MIB::hrDeviceDescr.11 = STRING: LUN VMware Virtual_disk 1.0 sdh
HOST-RESOURCES-MIB::hrDeviceDescr.12 = STRING: LUN VMware Virtual_disk 1.0 sdi
HOST-RESOURCES-MIB::hrDeviceDescr.13 = STRING: LUN VMware Virtual_disk 1.0 sdj
HOST-RESOURCES-MIB::hrDeviceDescr.14 = STRING: LUN VMware Virtual_disk 1.0 sdk
HOST-RESOURCES-MIB::hrDeviceDescr.15 = STRING: LUN VMware Virtual_disk 1.0 sdl
 

Networking

A common issue often in operational sense is connectivity. Basic connectivity starts with applications and ports. Services must open TCP and UDP (protocol) ports to provide service. Besides monitoring processes, checking for open ports helps maintain an operational system. Ports for services are provided in Required ports for vCenter Server Appliance 5.x (2012773) and Network port diagram for vSphere 5.x (2054806).
 
Open TCP and UDP ports can be found easily and consistently across the VMware vSphere stack using the following tables from HOST-RESOURCES-MIB. The table tcpListenerTable provides open TCP connections and as shown earlier in the Memory/Processes section, you can find out which applications have what ports open by using the related process id.
 
snmpwalk -v3 -u test 192.0.2.1 tcpListenerProcess
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".22 = Gauge32: 2554
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".80 = Gauge32: 2773
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".443 = Gauge32: 2773
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".902 = Gauge32: 2554
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".1134 = Gauge32: 2851
TCP-MIB::tcpListenerProcess.ipv4."0.0.0.0".8300 = Gauge32: 2500
TCP-MIB::tcpListenerProcess.ipv4."127.0.0.1".8307 = Gauge32: 2851
TCP-MIB::tcpListenerProcess.ipv4."127.0.0.1".8309 = Gauge32: 2851
TCP-MIB::tcpListenerProcess.ipv4."127.0.0.1".8889 = Gauge32: 3259
TCP-MIB::tcpListenerProcess.ipv4."127.0.0.1".12001 = Gauge32: 2851
 
For UDP, the udpConnectionTable report on both all interfaces (0.0.0.0) and internal/loop interface (127.0.0.1). Port 161 is used by snmpd since it points to process id 30696, port 514 is the rsyslogd service the same as if you ran netstat –nap | grep 514, and port 902 is vpxd for example when polling the variable udpEndpointProcess.
 
snmpwalk -v2c -c public 192.0.2.1 udpEndpointProcess
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".53.ipv4."0.0.0.0".0.1 = Gauge32: 492547
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".68.ipv4."0.0.0.0".0.1 = Gauge32: 489043
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".88.ipv4."0.0.0.0".0.1 = Gauge32: 534292
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".135.ipv4."0.0.0.0".0.1 = Gauge32: 489592
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".161.ipv4."0.0.0.0".0.1 = Gauge32: 30696
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".514.ipv4."0.0.0.0".0.1 = Gauge32: 1143449
UDP-MIB::udpEndpointProcess.ipv4."0.0.0.0".902.ipv4."0.0.0.0".0.1 = Gauge32: 27982
UDP-MIB::udpEndpointProcess.ipv4."127.0.0.1".57359.ipv4."127.0.0.1".57359.1 = Gauge32: 1185092
UDP-MIB::udpEndpointProcess.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".53.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".0.1 = Gauge32: 492549
UDP-MIB::udpEndpointProcess.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".161.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".0.1 = Gauge32: 137990276
UDP-MIB::udpEndpointProcess.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".514.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".0.1 = Gauge32: 1143552
UDP-MIB::udpEndpointProcess.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".41978.ipv6."00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00".0.1 = Gauge32: 1640484
 
Networking details to monitor network traffic are found in the IF-MIB.
 
HOST-RESOURCES-MIB::hrNetworkIfIndex.3 = INTEGER: 2
 
IF-MIB::ifDescr.2 = STRING: Device eth0 at 03:00.0 VMware - VMXNET3 Ethernet Controller
IF-MIB::ifType.2 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifPhysAddress.2 = STRING: 2:0:16:db:47:c9
IF-MIB::ifAdminStatus.2 = INTEGER: up(1)
IF-MIB::ifOperStatus.2 = INTEGER: up(1)
IF-MIB::ifLastChange.2 = Timeticks: (0) 0:00:00.00
With statistics at 5 second cache rate are:
IF-MIB::ifInOctets.2 = Counter32: 2284246
IF-MIB::ifInUcastPkts.2 = Counter32: 18624
IF-MIB::ifInNUcastPkts.2 = Counter32: 3157
IF-MIB::ifInDiscards.2 = Counter32: 0
IF-MIB::ifInErrors.2 = Counter32: 0
IF-MIB::ifInUnknownProtos.2 = Counter32: 0
IF-MIB::ifOutOctets.2 = Counter32: 1892021
IF-MIB::ifOutUcastPkts.2 = Counter32: 17033
IF-MIB::ifOutNUcastPkts.2 = Counter32: 30
IF-MIB::ifOutDiscards.2 = Counter32: 0
IF-MIB::ifOutQLen.2 = Gauge32: 1000
 
The IPv4 and IPv6 addresses assigned, their state (preferred, invalid) and how the were configured (dhcp, manual) are found in:
 
IP-MIB::ipAddressIfIndex.ipv4."10.160.95.253" = INTEGER: 2
IP-MIB::ipAddressStatus.ipv4."10.160.6.247" = INTEGER: preferred(1)
IP-MIB::ipAddressOrigin.ipv4."10.160.6.247" = INTEGER: manual(2)
 
Routing configuration information is provided by
  • interface routes:
    IP-FORWARD-MIB::inetCidrRouteIfIndex.ipv4."10.160.31.255".32.2.0.0.ipv4."0.0.0.0" = INTEGER: 2

    and
  • default route, where the value 0.0.0.0 means any dest will go to 10.160.31.253:
    IP-FORWARD-MIB::inetCidrRouteIfIndex.ipv4."0.0.0.0".0.2.0.0.ipv4."10.160.31.253" = INTEGER: 2
 
How to convert IP address to Ethernet MAC address for local systems connected by NIC for both IPv4 (ARP) and IPv6(ND) are available in the IP-MIB ipNetToPhysicalPhysAddress:
IP-MIB::ipNetToPhysicalPhysAddress.2.ipv4."10.160.31.253" = STRING: 0:0:c:9f:fa:28
which translates to 2=IfIndex as shown in the above example, the ethernet port knows that IP address 10.160.31.253 is MAC address 0:0:c:9f:fa:28.
 
Full statistics for protocols ICMP, ICMP6, IPv4, IPv6 are also available:
 
IP-MIB::icmpStatsInMsgs.ipv4 = Counter32: 46
IP-MIB::icmpStatsInMsgs.ipv6 = Counter32: 29
IP-MIB::icmpStatsInErrors.ipv4 = Counter32: 0
IP-MIB::icmpStatsInErrors.ipv6 = Counter32: 0
 
General counters for UDP:
 
UDP-MIB::udpInDatagrams.0 = Counter32: 94922
UDP-MIB::udpNoPorts.0 = Counter32: 38
UDP-MIB::udpInErrors.0 = Counter32: 6
UDP-MIB::udpOutDatagrams.0 = Counter32: 94967
 

Conclusion

You can now conclude that Memory/Processes, CPU, Filesystems and Networking can all be examined with queries to identify a VCSA system uniquely, to see elements of configuration, and verify much of what is typically examined using an SSH command line session, to provide details of a Virtual Center Server appliance (VCSA).


Additional Information

For related information, see:
 
Collecting diagnostic information for VMware products
Uploading diagnostic information for VMware
SNMP MIB module file download
Special characters in the community string for SNMP traps are not retained
Supported vCenter Server High Availability Options
Restoring and copying vCenter Server alarms
Required ports for vCenter Server Appliance 5.x
SNMP Trap is repeated every 5 minutes
ESXi SNMP agent log entries for informs
Debugging with the esxcli system snmp test command
Determining the MIB module listing, name, and type of an SNMP OID
Network port diagram for vSphere 5.x
Understanding the SNMP Layer 2 networking in the VMware ESXi
使用 SNMP 监控 VCSA

Impact/Risks:
VMware SNMP agents are designed to be small, memory and CPU efficient. Further SNMP agents are designed to avoid the same system/software dependencies of the other management APIs found coresident in VMware appliances. The goal being that partial failures in a given appliance do not create an all-or-nothing situation in terms of reporting management status.


Powered by Wolken Software