vCenter Operations Manager - Troubleshooting Network Connectivity

Isn't vCenter Operations Manager great? Finally, you can do something with all the data that vCenter has been quietly and dutifully collecting over the years (you knew it was doing that, right?).

You can't get there from here. - vCOPS

vCOPS is a vApp that consists of two VMs: one to run analytics on vCenter's collected data, and one to handle the presentation components of the application. To work properly, these two VMs need to communicate with one another, and with vCenter. But what do you do if the IP address for one of these hosts changes?

In this example, I'm trying to access the GUI on the UI VM at 192.168.1.19. But I'm getting an error message that indicates the UI VM cannot communicate with the Analytics VM.

First, confirm that the Analytics VM has, in fact, decided to use a different IP address (this happens on lab networks often, when DHCP is used for IP allocation. Please don't do this in production. :) ). Use your vSphere Web Client to see the address it's using. Notice in the image below that the VM is using 192.168.1.68, while the UI VM is expecting it to be at 192.168.1.4.

Right host, wrong address.

To correct this, you'll need to connect to the VM either via ssh or the remote console. Remember that to connect to the shell, you'll use the root account (admin is used only for the GUI).

Changing the IP address of the Analytics VM requires the use of a utility named vami_set_network. This utility is located in /opt/vmware/share/vami. It's easy to correct this problem. You'll pass a series of values to this utility, and it changes the properties of your ethernet adapter for you. A few assumptions: you are using eth0 for your VM (this is the default interface), and you are using IPv4.

Here's the syntax: 

In the example, we need to change the Analytics VM's IP back to 192.168.1.4. Here's how you'd do that:

cd /opt/vmware/share/vami

./vami_set_network eth0 STATICV4 192.168.1.4 255.255.255.0 192.168.1.1

And here's how it looks in action:

Give vCenter a minute to detect the IP change and refresh the Summary page for this VM. Then you'll see that the IP has been properly changed. Now when you browse to the vCOPS UI, you'll get the login page that you were looking for.

Note: It's possible that a restart of the vCenter Operations Manager application is required to restore connectivity. If you get 503 errors when trying to hit the login page, here's what to do:

1. Connect to the IP of your UI VM, and add /admin to the URL (e.g., https://192.168.1.19/admin).
2. Log in as admin.
3. Click the Status Tab.
4. In the Application Controls tab, click the Restart button. This will take about 5 minutes to stop and start the application.
5. Log out of the admin console, and refresh the vCOPs login page.

Getting familiar with the utilities in /opt/vmware/share/vami is a GOOD THING. Spend some time learning what's there. I have a feeling that all virtual appliances from VMware will share this same framework, which means these skills will come in handy in the future.