Changing VMware vCloud Director Java heap size to prevent java.lang.OutOfMemoryError messages
search cancel

Changing VMware vCloud Director Java heap size to prevent java.lang.OutOfMemoryError messages

book

Article ID: 339716

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

  • When vCloud Director logs messages of the form java.lang.OutOfMemoryError, it often indicates that the default JVM heap size is inadequate for the load being placed on a vCloud Director cell. You can reconfigure the default heap size if necessary.
  • A java dump file, java_pid#.hpof is generated in the vCloud Director logs directory, located at /opt/vmware/vcloud-director/logs/.
  • The cell.log or the cell.log.last logs contain entries similar to:

    Successfully posted pending audit events: com/vmware/vcloud/event/cell/start
    Successfully started remote JMX connector on port 8999
    java.lang.OutOfMemoryError: GC overhead limit exceeded
    Dumping heap to /opt/vmware/vcloud-director/logs/java_pid32277.hprof ...
    Uncaught Exception. Originating thread: Thread[170684272@pool-jetty-14,5,main]. Message: GC overhead limit exceeded
    Application shutdown event: Beginning
    Dump file is incomplete: file size limit
    log4j:ERROR Failed to flush writer,
    java.io.IOException: No space left on device
    at java.io.FileOutputStream.writeBytes(Native Method)
 
Note: This error may also be reported in the vcloud-container-debug.log file.
 

INFO |vmware-vcd-watchdog started
  • The vmware-vcd-watchdog.log file may contain entries similar to:
ERROR | vmware-vcd-cell requires additional disk space to start, will not try to restart
INFO | vmware-vcd-watchdog exiting


Note: For more information on vCloud Director logs see: Viewing the vCloud Director Logs


    Environment

    VMware Cloud Director 5.5.x
    VMware Cloud Director for Service Provider 5.6.x
    VMware Cloud Director for Service Provider 8.0.x
    VMware Cloud Director for Service Provider 8.10.x
    VMware Cloud Director 5.1.x
    VMware Cloud Director for Service Provider 9.x
    VMware Cloud Director 1.5.x

    Resolution

    To modify the default values for vCloud Director Java heap size:
    1. Log in to a vCloud Director server host as root.
    2. Open the $VCLOUD_HOME/bin/vmware-vcd-cell file using a text editor.

      Note: For VMware vCloud Director for Service Providers 9.x and higher, the file to edit is $VCLOUD_HOME/bin/vmware-vcd-cell-common
       
    3. Search for an entry like this one within the file (values shown for -Xms, -Xmx, and -XX:MaxPermSize might be different on your system):

      JAVA_OPTS="\${JAVA_OPTS:--Xms512M -Xmx1024M -XX:MaxPermSize=256m
      -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${VCLOUD_HOME}/logs} \

      ${ESB_JAVA_OPTS} \

      ${VCLOUD_JAVA_OPTS}"

    4. Modify the entry and increase the values for these parameters:
      • -Xms – Specifies the initial heap size
      • -Xmx – Specifies the maximum heap size
      • -XX:MaxPermSize – Specifies the size of the Permanent Generation

        Note: Do not modify any other parameters in this entry.
    5. Stop and restart the vCloud Director services on the host.
    6. Repeat this procedure for all other hosts in the vCloud Director cluster.
    Powered by Wolken Software