vStable.com

vCenter Operations 5.x vCenter Plugin uses IP instead of DNS hostname

Posted on the April 2nd, 2012 under VMware by Josh Perkins

Update 12-10-2012: The below process works the same on the vApp vCenter Operations Suite 5.6.

If you have changed the default self-signed vCenter Operations Manager (vCOPS) 5 SSL certificate you probably have noticed that your users receive an invalid SSL certificate warning when connecting to vCenter using the vSphere Client and also when clicking the vCenter Operations Manager plugin icon. This behavior occurs because vCOPS registers its plugin using an IP address instead of the DNS name you specify in your local DNS and as a result does not match your SSL certificate CN value.

Below is how you resolve the issue, but first lets quickly discuss how the vCenter Plugin-to-vCOPS communication works. When registering a vCOPS installation with vCenter using the vCOPS admin interface, an entry is created within the vCenter database table “VPX_EXT_SERVER” that specifies the URL to the vCOPS configuration file (viClientConfig.xml).

When the vCenter service is started (or when you initially register vCOPS using the vCOPS admin interface) the URL value for the vCOPS plugin is read directly from the VPX_EXT_SERVER table and persisted to the vCenter Managed Object Browser (MOB) located at https://<vcenter>/mob. If you drill-down in the MOB (“content”–>”ExtensionManager”–>”extensionList[com.vmware.vcops]“–>”server”) you will notice the string value for URL will be https://<vcops-ipaddress>/vcops-vsphere/viClientConfig.xml.

When the vSphere Client connects to vCenter all the plugins and plugin settings that are registered with vCenter are read from the MOB. In the case of vCOPS, the MOB is queried for the vCOPS server URL value and then the vSphere Client reaches out to read the viClientConfig.xml from the vCOPS UI appliance.This process will throw an SSL warning because the vSphere Client is reaching out to vCOPS using an IP address instead of hostname.

Here’s how you fix the issue:

You will notice that I have already created and installed a valid SSL certificate (see certificate image) on the vCOPS UI device. You can follow the great instructions by Erik Bussink (http://www.bussink.ch/?p=458) to create the SSL certificate for vCOPS (note: the procedure I am providing assumes you are not going to use the SubjectAlternativeName (SAN) certificate workaround noted in Erik’s instructions. Using an IP address may cause problems in some scenarios that use proxies or have high security IE security zone settings. Additionally, many security departments may not allow SSL certificates issued to an IP address).

My certificate is issued to a “short-name” instead of the FQDN of the vCOPS appliance. I did this because my users don’t typically type a FQDN when accessing a webpage on the internal network and we don’t use SAN certificates. You can use either short-name or FQDN in your certificate–it doesn’t matter as long as the certificate CN value (“issued to”) matches the hostname in the URLs we will be modifying below.

Using an SSH client, login to the vCOPS UI appliance using the root account. Change your working directory (CD) to /usr/lib/vmware-vcops/tomcat/webapps/vcops-vsphere”. Using vi, edit the viClientConfig.xml so that the URL’s specified in the “VC 4.0″ section contain the vCOPS hostname instead of the IP address (see image). Remember to save the config file when exiting VI.

Next, edit the URL entry in the VPX_EXT_SERVER table within the vCenter database. This can be done very easily using Microsoft’s SQL Server Management Studio or simply by using an “update” SQL statement. The URL value should contain the vCOPS UI hostname instead of an IP address (see image).

Restart the VMware VirtualCenter Server service which will reload the updated URL value from the database into the MOB.

Done!! vSphere Client users will no longer receive the certificate warnings.