NSX-T Manager, Edge and Intelligence node log rotate has stopped
search cancel

NSX-T Manager, Edge and Intelligence node log rotate has stopped

book

Article ID: 318774

calendar_today

Updated On:

Products

VMware NSX Networking

Issue/Introduction

Symptoms:
In NSX-T Manager or Edge node or NSX Intelligence appliance:
  • The /var/log/ partition grows in size.
  • If you log in as root and list the the files in /var/log/, you noticed that all log files are growing quite large. For example: 1 or 2 GB.

    ls -lh /var/log
     
  • The root user password on the NSX-T Manager or Edge node or NSX Intelligence appliance has expired. This can be seen when you log into the manager as root (or even admin), you are prompted to change the password.
  • Routing issue may be reported on the Edge node affected by the /var/log partition full.


Environment

VMware NSX-T Data Center
VMware NSX-T Data Center 2.x
VMware NSX-T Data Center 3.x
VMware NSX-T Data Center 2.5.x

Cause

This issue occurs due to a known issue affecting NSX-T, which causes the log rotate to stop once the root user password has expired.

Resolution

This issue is resolved in:
  • VMware NSX-T Data Center 2.5.1, available at VMware Downloads.
  • VMware NSX-T Data Center 3.0.1, available at VMware Downloads. 
  • Currently there is no resolution for NSX Intelligence.


Workaround:
Instead of a full NSX-T upgrade, the following workaround can be applied. The root user password for the NSX-T manager appliances or Edge nodes must be reset. If the password has already expired, log into either of the appliances via console to reset the password. Below is the procedure to reset the password:
  • Under normal conditions the following can be used to reset the password by running this command:
    set user <username> [password <password> [old-password <old-password>]]
 
  • The password expiration time can be configured with the following command:
    set user <username> password-expiration <password-expiration>

Note:
Once the password is reset, clean up the log files in the /var/log which are larger than 31M by either removing them or moving them to /image.  This clean up process is not service impacting to NSX T.
Check the output of the command: “df -kh” and confirm that /var/log reflects the reduced usage. 
If “/var/log” shows 100% utilization even after moving/deleting the files, restarting the syslog service will resolve this issue. The syslog service can be restarted via the following command:
 
# /etc/init.d/rsyslog restart

Note: Above command ran at root level access of NSX T Manager. 

 


Powered by Wolken Software