SSO Domain Repoint fails in 6.7 U2, Authz Data export fails
Article ID: 316337
Updated On:
Products
VMware vCenter Server
Issue/Introduction
Symptoms:
When performing any operation utilizing "cmsso-util domain-repoint", including repointing a VCSA from an existing SSO Domain to another SSO Domain, or to a new SSO Domain, the operation will fail during the Autz data export phase, with a message like:
In the /var/log/vmware/cloudvm/domain_consolidator.log you see the following error:
The pre-check mode also fails with the following error on screen:
The mentioned .json file will not exist unless it was previously there. The end of the /var/log/vmware/cloudvm/domain_consolidator.log file will show the following error in pre-check mode:
When performing any operation utilizing "cmsso-util domain-repoint", including repointing a VCSA from an existing SSO Domain to another SSO Domain, or to a new SSO Domain, the operation will fail during the Autz data export phase, with a message like:
Starting License export ... Done
Starting Authz Data export ... Failed
Repoint failed. Restore from backup
Starting Authz Data export ... Failed
Repoint failed. Restore from backup
In the /var/log/vmware/cloudvm/domain_consolidator.log you see the following error:
2019-04-25T20:49:28.530Z INFO domain_consolidator Starting Authz Data export ...
2019-04-25T20:49:28.530Z INFO domain_consolidator Starting required services...
2019-04-25T20:49:28.531Z INFO domain_consolidator Executing command ['/bin/service-control', '--start', 'vmafdd', 'vmware-rhttpproxy', 'vmware-vpxd-svcs', 'vmware-vpostgres']
2019-04-25T20:49:29.215Z INFO domain_consolidator Started required services.
2019-04-25T20:49:29.659Z INFO domain_consolidator RC = 1
Stderr = Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M
Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/context/support/AbstractApplicationContext
at com.vmware.vim.vmomi.core.types.VmodlContext.initContext(VmodlContext.java:61)
at com.vmware.vim.vmomi.core.types.VmodlContext.initContext(VmodlContext.java:42)
at com.vmware.vim.lookup.client.LookupUtil.ensureVmodlContextInit(LookupUtil.java:49)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:185)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:156)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:93)
at com.vmware.vim.dataservices.ExportImportUtils.getQueryClientFromLS(ExportImportUtils.java:802)
at com.vmware.vim.dataservices.ExportImportUtils.createClient(ExportImportUtils.java:266)
at com.vmware.vim.dataservices.ExportAuthzData.main(ExportAuthzData.java:218)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.support.AbstractApplicationContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
2019-04-25T20:49:28.530Z INFO domain_consolidator Starting required services...
2019-04-25T20:49:28.531Z INFO domain_consolidator Executing command ['/bin/service-control', '--start', 'vmafdd', 'vmware-rhttpproxy', 'vmware-vpxd-svcs', 'vmware-vpostgres']
2019-04-25T20:49:29.215Z INFO domain_consolidator Started required services.
2019-04-25T20:49:29.659Z INFO domain_consolidator RC = 1
Stderr = Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M
Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/context/support/AbstractApplicationContext
at com.vmware.vim.vmomi.core.types.VmodlContext.initContext(VmodlContext.java:61)
at com.vmware.vim.vmomi.core.types.VmodlContext.initContext(VmodlContext.java:42)
at com.vmware.vim.lookup.client.LookupUtil.ensureVmodlContextInit(LookupUtil.java:49)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:185)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:156)
at com.vmware.vim.lookup.client.LookupUtil.createVmomiClient(LookupUtil.java:93)
at com.vmware.vim.dataservices.ExportImportUtils.getQueryClientFromLS(ExportImportUtils.java:802)
at com.vmware.vim.dataservices.ExportImportUtils.createClient(ExportImportUtils.java:266)
at com.vmware.vim.dataservices.ExportAuthzData.main(ExportAuthzData.java:218)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.support.AbstractApplicationContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
The pre-check mode also fails with the following error on screen:
Starting License pre-check ... Done
Starting Authz Data export ... Failed
Conflict data, if any, can be found under /storage/domain-data/Conflict*.json
Pre-checks failed.
Starting Authz Data export ... Failed
Conflict data, if any, can be found under /storage/domain-data/Conflict*.json
Pre-checks failed.
The mentioned .json file will not exist unless it was previously there. The end of the /var/log/vmware/cloudvm/domain_consolidator.log file will show the following error in pre-check mode:
2019-04-26T15:10:09.229Z INFO domain_consolidator Failed
2019-04-26T15:10:09.229Z INFO domain_consolidator Export of authz failed
2019-04-26T15:10:09.334Z INFO domain_consolidator Failed to execute script /usr/lib/repoint/authzservice_component_script.py
2019-04-26T15:10:09.334Z INFO domain_consolidator Conflict data, if any, can be found under /storage/domain-data/Conflict*.json
2019-04-26T15:10:09.335Z INFO domain_consolidator Failed executing <cis.component_data.DcComponentsPreCheck object at 0x7f7320ea05f8>
2019-04-26T15:10:09.335Z ERROR domain_consolidator Failed to run pre-checks for domain consolidation.
2019-04-26T15:10:09.335Z INFO domain_consolidator Cleaning up the temp directories
2019-04-26T15:10:09.336Z INFO domain_consolidator Successfully cleaned the storage directory
2019-04-26T15:10:09.337Z INFO domain_consolidator Pre-checks failed.
2019-04-26T15:10:09.229Z INFO domain_consolidator Export of authz failed
2019-04-26T15:10:09.334Z INFO domain_consolidator Failed to execute script /usr/lib/repoint/authzservice_component_script.py
2019-04-26T15:10:09.334Z INFO domain_consolidator Conflict data, if any, can be found under /storage/domain-data/Conflict*.json
2019-04-26T15:10:09.335Z INFO domain_consolidator Failed executing <cis.component_data.DcComponentsPreCheck object at 0x7f7320ea05f8>
2019-04-26T15:10:09.335Z ERROR domain_consolidator Failed to run pre-checks for domain consolidation.
2019-04-26T15:10:09.335Z INFO domain_consolidator Cleaning up the temp directories
2019-04-26T15:10:09.336Z INFO domain_consolidator Successfully cleaned the storage directory
2019-04-26T15:10:09.337Z INFO domain_consolidator Pre-checks failed.
The log will still contain the previously mentioned Java error as well in pre-check mode.
Environment
VMware vCenter Server 6.7.x
Cause
This is caused by an update of the spring version in vCenter 6.7 U2 from 4.3.9 to 4.3.20. The script /usr/lib/repoint/authzservice_component_script.py has hard set references to the 4.3.9 version, which is why we see the java.lang.NoClassDefFoundError as the 4.3.9 files no longer exist in /usr/lib/vmware/common-jars/.
Resolution
This has been resolved in 6.7 Update 3.
Workaround:
Ensure that you have a backup of the system before modifying any files or starting the domain repoint.
The current workaround which has been tested in a lab and works is to edit the /usr/lib/repoint/authzservice_component_script.py script and find the following lines:
Modify these entries to reference the 4.3.20 versions:
sed -i 's/4.3.9/4.3.20/g' /usr/lib/repoint/authzservice_component_script.py
Workaround:
Ensure that you have a backup of the system before modifying any files or starting the domain repoint.
The current workaround which has been tested in a lab and works is to edit the /usr/lib/repoint/authzservice_component_script.py script and find the following lines:
common_jars = ['spring-aop-4.3.9.RELEASE.jar',
'spring-beans-4.3.9.RELEASE.jar',
'spring-context-4.3.9.RELEASE.jar',
'spring-expression-4.3.9.RELEASE.jar',
'spring-web-4.3.9.RELEASE.jar',
'spring-webmvc-4.3.9.RELEASE.jar',
'spring-core-4.3.9.RELEASE.jar',
'spring-oxm-4.3.9.RELEASE.jar',
'spring-aop-4.3.9.RELEASE.jar'
'spring-beans-4.3.9.RELEASE.jar',
'spring-context-4.3.9.RELEASE.jar',
'spring-expression-4.3.9.RELEASE.jar',
'spring-web-4.3.9.RELEASE.jar',
'spring-webmvc-4.3.9.RELEASE.jar',
'spring-core-4.3.9.RELEASE.jar',
'spring-oxm-4.3.9.RELEASE.jar',
'spring-aop-4.3.9.RELEASE.jar'
Modify these entries to reference the 4.3.20 versions:
common_jars = ['spring-aop-4.3.20.RELEASE.jar',
'spring-beans-4.3.20.RELEASE.jar',
'spring-context-4.3.20.RELEASE.jar',
'spring-expression-4.3.20.RELEASE.jar',
'spring-web-4.3.20.RELEASE.jar',
'spring-webmvc-4.3.20.RELEASE.jar',
'spring-core-4.3.20.RELEASE.jar',
'spring-oxm-4.3.20.RELEASE.jar',
'spring-aop-4.3.20.RELEASE.jar'
'spring-beans-4.3.20.RELEASE.jar',
'spring-context-4.3.20.RELEASE.jar',
'spring-expression-4.3.20.RELEASE.jar',
'spring-web-4.3.20.RELEASE.jar',
'spring-webmvc-4.3.20.RELEASE.jar',
'spring-core-4.3.20.RELEASE.jar',
'spring-oxm-4.3.20.RELEASE.jar',
'spring-aop-4.3.20.RELEASE.jar'
This can be accomplished with the following command (take a backup of the file before editing):
sed -i 's/4.3.9/4.3.20/g' /usr/lib/repoint/authzservice_component_script.py
Additional Information
Impact/Risks:
All cmsso-util domain-repoint operations will fail, leaving the node in a failed state which will require a restore to backup or revert to snapshot from before the operation began.
All cmsso-util domain-repoint operations will fail, leaving the node in a failed state which will require a restore to backup or revert to snapshot from before the operation began.
Feedback
Yes
No
Powered by
