Java heap space failure on install or upgrade of SovLabs modules in vRO

Problem

When you install or upgrade the SovLabs module in vRO, the SovLabs plugin version updates on the vRO Control Center, but in the vRO client, the version does not change. If this happens, you should look for an error like this in the /var/log/vco/configuration/controlcenter.log file on the vRO appliance (if using embedded vRO, this will be on the vRA appliance):

2020-07-23 19:43:25.851+0000 [syncScheduler-1] WARN {} [SqlExceptionHelper] SQL Error: 0, SQLState: 53200
2020-07-23 19:43:25.851+0000 [syncScheduler-1] ERROR {} [SqlExceptionHelper] Ran out of memory retrieving query results.

Caused by: java.lang.OutOfMemoryError: Java heap space

Cause

This issue is due to the vRO configurator JVM Memory running out while attempting to load plugins. It causes the plugin load to fail, which is why the version doesn't update in vRO client. The plugin does not load properly and will not function.

Affected Versions

All versions

Workaround or Solution

Overview

As a part of our prerequisite configurations we do run a script and recommend the increase of -Xmx for the configurator to 768mb. If you find you have failures during the installation or configuration with the mentioned error, you may need to increase this beyond 768mb. We have seen customers have to increase this to up to 4096mb in order to alleviate the issue. Below is how you can increase the Configuration JVM memory for the configurator.

NOTE: If you're running a cluster, you have to change the JVM memory for configurator on all vRO appliances and perform a restart of configurator).

Updating your configurator setenv.sh file on each vRA/vRO server

It is assumed that the person performing these changes is familiar with updating files within a linux OS. The examples below do not provide the key sequences you must press in order to edit the file and save the file

SSH To your vRO server. Choose 1 server to work on first if you're in a cluster, and do all these steps on that server.
In the command line type in your favorite editor followed by the file location, this example is using vi, press Enter
```
vi /usr/lib/vco/configuration/bin/setenv.sh
```
NOTE
This is not the same as the heap setting for the vco-server jvm. Make sure the setenv.sh you're updating is in /usr/lib/vco/configuration/bin.
Edit the line below the heading of MEM_OPTS and update the -XmxNNNNm (Where NNNN is the current value). Example; change -Xmx768m to -Xmx1024m. We recommend incrementing by 256m or 512m and increasing until the out of memory errors in /var/log/configuration/controlcenter.log stop. We don't recommend going past 4096m. If you get to 4096m and the problem is still occurring, we recommend opening a ticket with VMware support at that point. Disclaimer: this information is provided as a courtesy and we expect that you'll take reasonable precaution when implementing this update like making a copy of the setenv.sh file before making edits, and that you won't update the heap size beyond what the physical memory on the appliance can handle. In our experience, 4096m has worked without issue.
```
MEM_OPTS="\
-Xms128m \
-Xmx768m \
-Xmn128m \
-Xss256k \
```
Once you have updated the value, save the file and go back to your prompt.
Restart your configurator service (no need to restart vco-server at this point)

service vco-configurator restart
Tail the configurator log as vco-configurator is restarting and monitor for the out of memory error. If you see the out of memory error, go back repeat steps 2-6 until you no longer see the error.
```
tail -f /var/log/vco/configuration/controlcenter.log
```
If you're on a cluster, once you you have eliminated the out of memory error on one node, repeat steps 1- 6 for all the other nodes and set the -Xmx in the setenv.sh to the value you ended on on your first node.
Once all nodes have been updated and the out of memory errors have ceased, restart the vco-server services on all nodes.

service vco-server restart
Check the vRO client and see if your SovLabs plugin version has updated to the correct one. (See Screenshot below)

Most of the time, a re-install is not necessary as the plugin is already present in the file system and once you've eliminated the issue with configurator loading it, everything loads. Check all the nodes by opening vRO client to each one individually.