Investigating the health of a vCenter Server database (1003979)
Validate that each troubleshooting step below is true for your environment. The steps and sections are ordered in the most appropriate sequence to isolate the issue and identify the proper resolution. Do not skip a step or a section.
The article is split up into several sections to allow for targeted troubleshooting depending on the symptoms. The sections are:
- General Troubleshooting
- Troubleshooting growth of the vCenter Server database
- Troubleshooting the database when vCenter Server is slow
- Reinitializing the vCenter Server database
VMware recommends that you start in the General Troubleshooting section to verify common configuration problems which might impact the other sections of the article.
Note: If you perform a corrective action in any of these steps, restart the VMware VirtualCenter Server service to validate the change. For more information, see Stopping, starting or restarting the vCenter server service (1003895).
To validate the basic configuration of the vCenter Server database:
- Verify that there is adequate disk space available on the volume that is storing the database files to ensure correct operation of the database. If there is not adequate space available on the physical volume that stores the database files, free up disk space.
- If you are using Microsoft SQL Server, verify that the transaction logs for the vCenter Server database are healthy and are backed up on a regular basis. For more information, see Troubleshooting transaction logs on a Microsoft SQL database server (1003980).
- If you are using Oracle, verify that there is adequate space available in the tablespace for growth of the database. For more information, see Extending the tablespace on an Oracle database (1003982).
- Validate the authentication to the vCenter Server database. The vCenter Server service may not be able to authenticate with the database if:
- There are permission issues with the database when importing from one instance to another. For more information, see the Microsoft Knowledge Base article 246133.
- The password on the account you are using to authenticate to the database has changed but the password in the registry has not changed as well. For more information, see Changing the vCenter Server database user ID and password (1006482).
- The vCenter Server database user is not granted correct permissions. For more information, see Required credentials for Oracle and SQL Server Databases when installing or upgrading vCenter server (1003052).
- You may not be able to connect to the database repository through SQL Server or Oracle. If vCenter Server disconnects from the SQL or Oracle database, the database server can maintain a lock on the vCenter Server database. You may need to restart the SQL Service or Oracle instance in order to free the lock.
There are many circumstances where the vCenter Server database grows excessively. Growth of the vCenter Server database is due to data being collected and stored in the database. This data falls into three categories:
- Performance data
- Log of tasks that were performed
- Log of events that occurred
Performance data, in most cases, causes the most growth of the database. When troubleshooting excessive growth of the database, start by examining where the growth is occurring. From here, you can determine how to troubleshoot the issue.
It may be necessary to truncate the information in the database. However, this should never be the first step. Diagnose the issue completely, first, if at all possible. In many cases, the amount of data is too large to be able to process in the given amount of time. For example, if there is more data that is being collected than can be processed in the same period of time, or if it takes several months to process the data, it may be better to simply truncate the data. In these cases, it is dependent on the business requirements as well as the cause of the growth.
To validate where the growth of the database is occurring, see Determining where growth is occurring in the vCenter Server database (1028356).
When the culprit tables have been determined, see Purging old data from the database used by vCenter Server (1025914)
If it is the transaction log that grew, see Troubleshooting transaction logs on a Microsoft SQL database Server (1003980).
vCenter Server is a database application and is therefore subject to the same problems as any other application. These are common issues that may occur:
- Slow response when retrieving performance data, tasks, events or inventory information.
- Only real time performance data is available.
- You see the error:
Performance data is currently not available for this entity
To troubleshoot these issues:
- If you are running a Microsoft SQL database, perform a defragmentation of the performance data indexes on the vCenter Server database. This ensures that the data can be accessed as quickly as possible. For more information, see Performing a defragmentation of the vCenter performance data indexes on a Microsoft SQL database (1003990).
- Validate whether the vCenter Server database server has adequate resources available. Consult the vSphere Installation Guide for your version of vCenter Server, along with the vendor documentation for the database. There is no one answer for this, depending on the amount of load and the database being used.
It may be beneficial to separate vCenter Server and the database, if they are running on the same server and if there is a lot of activity in the environment. Resource contention can cause a noticeable decrease in vCenter Server performance.
In addition, these steps may help to increase the performance of the database:
- Separate the database and transaction / redo logs on to separate physical drives. This helps to avoid contention on the disks because it increases the number of write operations that can be done at the same time.
- Avoid the use of statistics collection levels above level 2, unless troubleshooting a specific problem. The amount of data collected in the higher levels can cause both slow performance of the database and excessive growth.
- Avoid running other applications on the server which is hosting vCenter Server. This can appear to be a database problem. However, in reality, due to contention on the vCenter Server system, it can be a factor in slow performance of the database.
- Do not use SQL Express. It is only designed to run a limited configuration and quickly hits the 4 GB database size limit. This causes a decrease in performance, and then failure of the service.
For more information on performance recommendations, see the VMware vCenter Server Performance and Best Practices document.
A reinitialization of the vCenter Server database resets it to the default configuration, as if the vCenter Server was newly installed. These are a few situations which can warrant resetting the database:
- It is required to rebuild vCenter Server
- Data corruption is suspected
- VMware Support requests it
To reset the database configuration:
Warning: These steps permanently erase all data in the database. Ensure a proper backup of the database is taken prior to following these steps, if there is data that needs to be protected.
- Stop the VMware VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter Services (1003895) .
- Go to Start > Run type cmd, and click OK.
- Navigate to the vCenter Server program files directory. By default, this directory is C:\Program Files\VMware\Infrastructure\VirtualCenter\.
- Run the command:
- Start the VMware VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter Services (1003895).
If your problem still exists after trying the steps in this article:
- Gather the VMware Support Script Data. For more information, see Collecting diagnostic information for VMware products (1008524).
- File a support request with VMware Support and note this KB Article ID (1003979) in the problem description. For more information, see How to Submit a Support Request.
Note: If you encounter this error in the
vpxd.log when trying to connect the SQL database server
ODBC error: (08S01) - [Microsoft][SQL Native Client]Named Pipes Provider: No process is on the other end of the pipe
and if a corresponding Event ID 17187 is recorded in the Windows Event log on the SQL Server at approximately the same time, this Event Log code indicates
SQL Server is not ready to accept new client connections; the connection has been closed
This suggests a possible delay is needed between the time the SQL service starts, and the time that VirtualCenter Server services is attempted to be started.
Before performing additional troubleshooting, restart the SQL Server service, wait approximately 60 seconds, and then start the VirtualCenter Server service. If this starts successfully, start the VMware Management Webservices service. For further steps on preventing a race condition between VirtualCenter Server service starting before the SQL database service see VMware VirtualCenter Server service does not start automatically after reboot (1007669).
- Troubleshooting a failed vCenter Server installation
- Purging old data from the database used by vCenter Server
- Determining where growth is occurring in the vCenter Server database
- Verifying jobs and stored procedures in VMware vCenter Server 4.1.x
- Investigando el estado del servidor de bases de datos de VirtualCenter
- Verifying vCenter Server 4.0 jobs and stored procedures
- Verifying vCenter Server 2.5 jobs and stored procedures
- vCenter Server データベースのヘルス調査について