Symptoms:

• You cannot select rights when creating a Role
• You cannot see existing rights when editing a Role
• Viewing the Console Browser Logs (F12), you see API Calls failing when selecting roles
• Cloud Director API Requests using the /cloudapi/ API fail with a TCP RST (ERR_CONNECTION_RESET)
• "unexpected character at line 1 column 1 of the JSON data" error message observed in the Cloud Director HTML5 UI

This issue occurs when you are connecting to Cloud Director via a Load Balancer which has limitations set to the Header Size or Header Count.
The Header Size or Header Count sent by Cloud Director to the Load Balancer exceed these values, and the Load Balancer resets the connection causing the API Call to fail.

The new Cloud API (cloudapi) is based on OpenAPI which places links in the headers making the Payload and Header Count substantially larger than the traditional /api/ API.

To resolve the issue, you can either a) Edit the Load Balancer Config, or b) Edit the Cloud Director Config
 

Edit the Load Balancer Config

Ensure that the values set on the Load Balancer are sufficient to receive the API Request responses from Cloud Director.

• If you are using an NSX Load Balancer please update the configuration using the NSX API
  • PUT https://<NSX-Manager>/api/4.0/edges/<Edge-ID>/systemcontrol/config
     
    <systemControl>
        <property>lb.global.tune.http.maxhdr=1024</property>  -- 256 may also be valid, but 1024 is a recommended value
        <property>lb.global.tune.bufsize=65536</property>
    </systemControl>

• If you are using an F5 Load Balancer, the below values have previously been noted to be sufficient.
  • Maximum Header Size:  65536
  • Maximum Header Count: 256

Note: Whilst the above values are currently sufficient, this may not always be the case. VMware Engineering may look to update the Headers section of various cloudapi requests in future releases of Cloud Director which may require the existing Load Balancer settings to be increased
 

Edit the Cloud Director Config

You can also amend Cloud Director to reduce the overall size of Requests and Headers sent to Load Balancers.
See the section vCloud Director UI and API users might encounter HTTP 502 errors if the load balancer does not permit large payloads in the vCloud Director 9.5 Release Notes