The following steps can be followed when migrating vSAN clusters from one vCenter to another vCenter and when the original vCenter is offline permanently or unable to be restored from backup.
Notes:
Base Steps:
1. Build a new vCenter server with a build version equal or greater than the version of the ESXi Hosts being migrated.
Version differences may affect the Skyline vSAN Health services in rare cases.
2. Prior to migrating the cluster, please be sure to test the vSAN VMkernel, Host Management, vMotion, and any additional dependent network routing schemes.
3. Determine if the host networking is configured with vDS, vSS, or NSX-T NVDS virtual switches.
- When using vSphere Standard Virtual Switch no additional steps are required other than ensuring the physical networking routes are working as designed.
- When using vSphere Distributed Switch, please reference further material/instruction KB links below to prepare the vDS for migration.
Note: While importing vDS settings, do not select preserve original distributed switch port group identifiers as this could negatively impact the migration.
- When using NSX-T NVDS with standalone host configuration, no additional steps are required other than ensuring the physical networking routes are working as designed.
Note: If using Transport Node Profile for your NSX-T NVDS, then you need to migrate to standalone host configuration.
For more information on the migration of Hosts, see Moving an ESX/ESXi host with vDS from one vCenter Server to another (1029498)
For more information on migration of vDS see Exporting/importing/restoring Distributed Switch configs using vSphere Web Client (2034602)
Backing Up and Restoring a vSphere Distributed Switch Configuration
For more information on vSphere see https://docs.vmware.com/
- Once the cluster networking configuration requirements are satisfied you may proceed with the next steps of the migration.
4. Determine if vSAN Data at Rest Encryption is in use and make certain that keys and the KMS configurations are well known. See the guide for further guidance in preparing the cluster for migration.
Also, See further details in this instruction on step 7a.
Document: Using Encryption on a vSAN Cluster
5. Create a new cluster with DRS, vSphere HA, and EVC Disabled. These options should only be enabled after the cluster is moved and stable on the new vCenter.
6. Create a new cluster Inventory object.
7a. Enable vSAN along with required services matching the original cluster parameters such as Encryption and or Deduplication and Compression.
- If the original vCenter is not available to compare configurations, the following command can be used to list the vSAN services in use.
localcli vsan storage list |grep -i 'Device\|Is ssd\|Is Capacity Tier\|DEDUPLICATION\|COMPRESSION\|In CMMDS\|ENCRYPTION:' |sed 'N;N;N;N;;N;N;s/\n//g' | sort -k9;
# localcli vsan storage list |grep -i 'Device\|Is ssd\|Is Capacity Tier\|DEDUPLICATION\|COMPRESSION\|In CMMDS\|ENCRYPTION:' |sed 'N;N;N;N;;N;N;s/\n//g' | sort -k9; Device: mpx.vmhba0:C0:T1:L0 Is SSD: true In CMMDS: true Deduplication: false Compression: false Is Capacity Tier: false Encryption: true Device: mpx.vmhba0:C0:T2:L0 Is SSD: true In CMMDS: true Deduplication: false Compression: false Is Capacity Tier: true Encryption: true
7b. When KMS Encryption service is in use -
Make certain to add the KMS servers to the newly built vCenter server with the same KMS-Cluster names and complete all required steps to add the KMS servers into the new vCenter server.
Example :
# esxcli vsan encryption kms list Cluster Name KMS Address Proxy Address Proxy Port Username -------------------- ----- ----------------- ------------- ---------- -------- Hy-Trust-KMS-Cluster KMS-1 10.109.44.40:5696 N/A N/A N/A # esxcli vsan encryption info get Attribute Value ------------------- ----- kekId bfdfad26-ba41-4bec-8593-d16014858278 dekGenerationId 1 enabled True hostKeyId 835cc701-1734-4411-bf2c-7df3ec524536 eraseDisksBeforeUse False changing False
8. Create the Virtual Machine vSAN storage policies that match the vSAN policies from the old cluster. Ensure that rulesets have matching policies. If the VMs are migrated without matching policies, vSAN may need to perform a resync to ensure the VMs and objects are compliant.
If the policy names and policy attributes were unknown due to old vCenter unavailability, the following commands can be used to obtain the policy settings:
"esxcli vsan debug object list | grep spbmProfileId | sort | uniq" or "esxcli vsan debug object list | grep spbmProfileName | sort | uniq"
Review the attributes for each object using the policies by running "esxcli vsan debug object list |less" and search by the policy name (found above).
Example :
# esxcli vsan debug object list | grep spbmProfileId | sort | uniq
spbmProfileId: 3128b75d-6790-4f7f-a25d-4d7889850940
spbmProfileId: aa6d5a82-1c88-45da-85d3-3d74b91a5bad
# esxcli vsan debug object list | grep spbmProfileName | sort | uniq
spbmProfileName: VM Storage Policy-DN
spbmProfileName: vSAN Default Storage Policy
An example is provided below of the policy attributes for the spbmProfileName: vSAN Default Storage Policy
Run Command : esxcli vsan debug object list | less
Example :
Object UUID: b5bf275e-5ba7-5d01-be96-ecf4bbec6050
Version: 10
Health: healthy
Owner: ESXI1.lab.com
Size: 15.00 GB
Used: 0.89 GB
Policy: ** CONTENT BELOW INDICATES POLICY ATTRIBUTES**
cacheReservation: 0
forceProvisioning: 0
spbmProfileId: aa6d5a82-1c88-45da-85d3-3d74b91a5bad
proportionalCapacity: 0
spbmProfileGenerationNumber: 0
stripeWidth: 1
spbmProfileName: vSAN Default Storage Policy
CSN: 16
hostFailuresToTolerate: 1
Configuration:
RAID_1
Component: b5bf275e-6125-aa03-efcc-ecf4bbec6050
Component State: ACTIVE, Address Space(B): 16106127360 (15.00GB), Disk UUID: 52208d4c-7f55-578a-cf89-21b65b58e3a3, Disk Name: naa.5002538c4044d87f:2
Votes: 1, Capacity Used(B): 486539264 (0.45GB), Physical Capacity Used(B): 478150656 (0.45GB), Host Name: ESXI1.lab.com
Component: b5bf275e-8389-ab03-58a0-ecf4bbec6050
Component State: ACTIVE, Address Space(B): 16106127360 (15.00GB), Disk UUID: 5225900e-2911-3548-1c57-3fb38e948a00, Disk Name: naa.5002538c4044d881:2
Votes: 1, Capacity Used(B): 486539264 (0.45GB), Physical Capacity Used(B): 478150656 (0.45GB), Host Name: ESXI2.lab.com
Witness: b5bf275e-0168-ac03-7612-ecf4bbec6050
Component State: ACTIVE, Address Space(B): 0 (0.00GB), Disk UUID: 525452e3-e6c0-884a-a6ea-edc03570a490, Disk Name: naa.5002538c4044d6ab:2
Votes: 1, Capacity Used(B): 12582912 (0.01GB), Physical Capacity Used(B): 4194304 (0.00GB), Host Name: ESXI3.lab.com
Type: vdisk
Path: /vmfs/volumes/vsan:523d5e5605a4d751-0c3304ae7a42599b/b1bf275e-24f9-b5ff-5860-ecf4bbec6050/VCSA70U1_6.vmdk (Exists)
Group UUID: b1bf275e-24f9-b5ff-5860-ecf4bbec6050
Directory Name: N/A
9. In vSAN 6.6 and above, it is critical to maintain vSAN virtual networking unicast communication between hosts on the vSAN network. Please see KB for configuration details.
Configuring vSAN Unicast networking from the command line (2150303)
10. Set the following configuration via CLI on each ESXi vSAN Cluster Member Host to be migrated.
esxcfg-advcfg -s 1 /VSAN/IgnoreClusterMemberListUpdates
This will ensure when the hosts are disconnected from the original vCenter, they will not lose their unicast configurations and therefore their connectivity between hosts. If this step is skipped, the unicast address table will be cleared and data availability will be affected temporarily until the table is manually repopulated.
11. Run the command "esxcli vsan cluster get" from any hosts in the cluster and take note of the host count to ensure the count matches before and after the migration.
12. Disconnect one host from vCenter by right clicking the host and using the disconnect option. (**Not Applicable if old vCenter is unavailable)
13. Remove the host from inventory. Make sure to follow KB 1029498 if using a vDS (**Not Applicable if old vCenter is unavailable)
14. Add the hosts only to the DataCenter level of the new vCenter inventory. Do NOT add the hosts directly to the new vSAN enabled cluster otherwise, the host will be forced into Maintenance Mode. (Avoiding Maintenance Mode is key for a seamless downtime migration.)
15. Once the host is registered in the new vCenter inventory, under the DataCenter Inventory Object, it can safely be dragged and dropped into the new vSAN enabled cluster from within the vCenter UI.
16. Before moving on to the next host, verify there is host-to-host vSAN communication. Run the command "esxcli vsan cluster get" from ESXi CLI and take note of the host count to ensure the count matches.
It is good to practice this command each time a host is disconnected, moved, and connected. If the host is not communicating, i.e. the host count is less than expected then it is highly likely a critical step has been missed. Please review the steps and ensure each host's unicast tables are complete and host-to-host communication on the vSAN network is fully functional.
To verify unicast agent table, you can use below command -
esxcli vsan cluster unicastagent list
17. Once the first host is moved and connectivity checks are performed you may move forward with the remaining hosts. Please be certain to follow the steps carefully and check the cluster membership along the way to prevent any data availably impacts.
18. Once all hosts have been added to the new cluster and access to the VMs has been verified, revert the hosts' unicast table configuration. This procedure should be completed at the completion of the migration, and before the hosts are rebooted next. Host rebooting is not necessary for the procedure's success.
esxcfg-advcfg -s 0 /VSAN/IgnoreClusterMemberListUpdates
20. Apply the vSAN storage policy to the VMs.
Click OK. Assuming the policy is consistent with the old policy and all the hosts have remained in a healthy state, a resync shouldn't be triggered. It is recommended to perform this action, one at a time. The goal is to move gradually to prevent creating a major resync in the cluster.
21. Repeat this action for the remaining VMs. When performed correctly, there will be no reconfigurations/resyncs. If the VMs start resyncing then there is an inconsistency in the ruleset from the original to the new storage policies. Please allow the resync to complete before proceeding.
22. Verify by browsing to Menu, Policies and Profiles, VM Storage Policies, desired storage policy, and VM Compliance. All the VMs should be shown.