Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Changing the default password hashing algorithm from md5 to sha512 in ESXi / ESX 4.x (1032666)

Purpose

This article provides information on changing the default password hashing algorithm from md5 to sha512 on ESX 4.x hosts.

Resolution

Setting the ESXi host to use SHA512 encryption algorithm

  1. Log in to the ESXi host using the Local Tech Support Mode or SSH.

    Note: Ensure that the appropriate support mode is enabled in Direct Console > Customize System (F2) > Troubleshooting Options.

  2. Back up the /etc/shadow and /etc/pam.d/system-auth files.
  3. Run these commands to assign edit permission to the system-auth file.

    #cd /etc/pam.d
    #chmod 644 system-auth
  4. Open the system-auth file using a text editor.
  5. Search for the password sufficient entry in the file, similar to:

    password sufficient /lib/security/$ISA/pam_unix.so use_authtok nullok shadow
  6. Replace the existing hash key ( md5, des, or sha256) with sha512 or append sha512, if there is no existing key. For example:

    password sufficient /lib/security/$ISA/pam_unix.so use_authtok nullok shadow sha512
  7. Run this command to restore the system-auth file permissions:

    #chmod 444 pam.d/system-auth

  8. If you want all passwords to be in the new hash format, you must renew the passwords for all users.

    For example, to change the hash format for the root password and to verify if the encryption algorithm is updated:

    1. Run this command to change the root password:

      #passwd root

    2. Open the /etc/shadow file and verify the if the encryption algorithm is updated.

      To verify the encryption algorithm, run this command:

      #cat /etc/shadow

      You see an output similar to:

      root:$6$BpPe7UrJ$oLexOiRNefHHDsfVETKTL0:13358:0:99999:7:::

      The tag $6$ in the output indicates that a sha512 encryption algorithm is used.
Note: In ESXi, changes made to the /etc/pam.d/system-auth file to edit password settings do not persist across system reboots.
         
This issue is fixed in ESXi 5.0.
 
 

Setting the ESX host to use SHA512 encryption algorithm

  1. Log in to the ESX host using the console or SSH.
  2. Back up these files:

    • /etc/pam.d/system-auth-*
    • /etc/libuser.conf
    • /etc/login.defs
    • /etc/shadow

  3. Run these commands to assign edit permissions to the system-auth, libuser.conf, and login.defs files:

    #cd /etc
    #chmod 644 pam.d/system-auth-*
    #chmod 644 libuser.conf
    #chmod 644 login.defs
  4. Open the /etc/pam.d/system-auth-generic file using a text editor.
  5. Search for the password sufficient entry in the file, similar to:

    password sufficient /lib/security/$ISA/pam_unix.so use_authtok nullok shadow

  6. Replace the existing hash key ( md5, des, or sha256) with sha512 or append sha512, if there is no existing key. For example:

    password sufficient /lib/security/$ISA/pam_unix.so use_authtok nullok shadow sha512

  7. Open the /etc/pam.d/system-auth-local file using a text editor.
  8. Repeat Steps 5 and 6 for the /etc/pam.d/system-auth-local file.
  9. Open the /etc/libuser.conf file using a text editor.
  10. Change this entry:

    crypt_style = md5

    to

    crypt_style = sha512

  11. Open the /etc/login.defs file using a text editor.
  12. Change this entry:

    MD5_CRYPT_ENAB yes

    to

    MD5_CRYPT_ENAB no

  13. Change this entry:

    ENCRYPT_METHOD DES

    to

    ENCRYPT_METHOD SHA512

    Note: If the MD5_CRYPT_ENAB and ENCRYPT_METHOD entries are not available in the /etc/login.defs file, add them at the end of the file:

    MD5_CRYPT_ENAB no
    ENCRYPT_METHOD SHA512

  14. Run these commands to restore the permissions on the system-auth, libuser.conf, and login.defs files:

    #chmod 444 pam.d/system-auth-*
    #chmod 444 libuser.conf
    #chmod 444 login.defs

  15. If you want all passwords to be in the new hash format, you must renew the passwords for all users.

    For example, to change the hash format for the root password and to verify if the encryption algorithm is updated:

    1. Run this command to change the root password:

      #passwd root

    2. Open the /etc/shadow file and verify the if the encryption algorithm is updated.

      To verify the encryption algorithm, run this command:

      #cat /etc/shadow

      You see an output similar to:

      root:$6$zR1rwYqO$aP/hDfnYQQ1oJVigYbG3O0:14833:0::7:::

      The tag $6$ in the output indicates that a sha512 encryption algorithm is used.

Note: Changing the hash method does not rehash existing passwords. It only impacts the format in which future passwords are stored when new accounts are created and passwords of existing accounts are modified.

Additional Information

There are two other weaker password hash methods: DES and SHA256. To use these hash methods, you must update the hash methods to des or sha256 in the appropriate files. Ensure to use DES and SHA256 (upper case) in the login.defs file.
 
Note: VMware does not recommend using these weaker password has methods. 
 
The following table lists the various encryption methods and their tags as seen in the /etc/shadow file:
 
Encryption Method Tag as seen in the /etc/shadow file
DES 13-char
MD5 $1$
SHA256 $5$
SHA512 $6$

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 2 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 2 Ratings
Actions
KB: