Note: vCenter 5 introduced a new “Inventory” service that communicates over its own web services port and uses its own SSL certificate. vCenter 5 specific instructions will be noted below.
|
Step 1
|
To proceed with the below process you will need to install the latest version of OpenSSL on Windows/Linux or optionally you can leverage the OpenSSL install on a VMware Management Assistant (vMA) appliance.
Tip: If you install OpenSSL on Windows you will need to set the environment variable OPENSSL_CONF to the directory where the default openssl.cfg file is located (this is typically c:\OpenSSL-Win32\bin\openssl.cfg). In the default scenario, at the command-prompt type set OPENSSL_CONF=c:\openssl-win32\bin\openssl.cfg
You can confirm the environment variable is correct by simply typing set at the command prompt and looking for the OPENSSL_CONF line.
|
|
Step 2
|
The very first item you will need to create is the replacement certificate Private Key.
At a command-prompt type openssl genrsa 2048 > rui.key
This will create a file within your current working directory called “rui.key”—this is your private key.
|
|
Step 3
|
Using the private key you will need to create a Certificate Signing Request (CSR) used by your Certificate administrator (or alternately used by a public/commercial Certificate Authority) to issue the Public Key.
At the command-prompt type openssl req –new –key rui.key > rui.csr
You will be prompted for the following information:
- Country Name (2 letter code): US
- State or Province Name (full name): California
- Locality Name (eg City): San Francisco
- Organization Name (eg Company): DuckWorks
- Organizational Unit Name (eg Section): Information Technology
- Common Name (this is your fully qualified server name): vCenter.duckworks.com
- Email Address: <don’t enter one>
- A challenge password: <don’t enter one>
- An optional company name: <don’t enter one>
This will create a file within your current working directory called rui.key—this is your Certificate Signing Request (CSR).
|
|
Step 4
|
Using notepad or any file editor, open the file rui.csr you created in step 3 above. Copy the text starting with (including) —–BEGIN CERTIFICATE REQUEST—– and ending (including) —–END CERTIFICATE REQUEST—–.

|
|
Step 5
|
You are now going to create the replacement certificate Public Key using a Microsoft Certificate Authority.
- Browse to your Microsoft Certificate Authority website (usually https://<servername>/certsrv/). Note: Your Certificate Authority may not use https but may be accessible using http://<servername>/certsrv/.
- Select Request a Certificate.
- Select Advanced Certificate Request.
- Select Submit a certificate request by using a base-64-encoded CMC or PKCS #10 file, or submit a renewal request by using a base-64-encoded PKCS #7 file.
- Copy the contents gathered in Step 4 into the textbox (you can also browse to your CSR file as well).
- Select the Web Server Certificate Template.
- Select Submit.
- Select Base 64 encoded option.
- Select Download certificate
- Important: When saving the certificate make sure you rename the certificate to rui.crt (note the .CRT file extension—don’t leave .CER as the file extension). When saving a base64 type certificate, .cer and .crt are interchangeable.
You can open the rui.crt file within Windows and it should look similar to the following:

|
|
Step 6
|
Create a PFX (pkcs12) file containing the public and private key pairs.
- Copy the two files rui.key and rui.crt into a folder (it’s easier to keep the files together).
- Using a command-prompt, type openssl pkcs12 –export –in rui.crt –inkey rui.key –name rui –passout pass:testpassword –out rui.pfx (note: using “testpassword” is significant because it’s used as the keystore password in the Tomcat server.xml file—you can use a different password to secure the PFX file but you will need to update the server.xml file to match the password you used.)
- Copy all three files (rui.crt, rui.key, rui.pfx) to C:\ProgramData\VMware\Vmware VirtualCenter\SSL\ (WIN2008) (recommend archiving the existing SSL certificates).
- From the command-prompt, type net stop vpxd (This will stop your core vCenter service so make sure you have a maintenance window).
- From the command-prompt, change your directory (CD) to the installation path of vCenter.
- From the command-prompt, type vpxd –p (you will be prompted for the database password used on your ODBC connection—the password will be re-encrypted using the new certificate).
- From the command-prompt, type net start vpxd
|
|
Step 7
|
This step is for vCenter 5 only. Use the following steps to replace the certificate used by the vCenter 5 Inventory service. This process is simple because you can use the certificate generated using the steps above for the inventory service.
- Copy rui.key, rui.crt, and rui.pfx to your vCenter Inventory Service installation path (ex. C:\Program Files\VMware\Infrastructure\Inventory Service\SSL\)
- Restart the vCenter Inventory Service within the Windows Service Control panel (services.msc).
|
After following the above steps your vCenter server will now be using the new certificate for all web services. Additionally, you will no longer be presented with a certificate warning popup when using the vSphere Client if the certificate authority that issued the replacement certificate is trusted by your computer (in this specific case any domain joined computers will automatically trust all certificates issued by your internal enterprise Microsoft Certificate Authority).
Stumbled upon an issue with the latest release of vSphere ESXi 4.1 Update 1 (fully patched) where following VMware KB 1032666 to modify ESXi default password hashing from MD5 to something stronger such as SHA-256 or SHA-512. Some federal government agencies cannot use MD5 for password hashing since it considered cracked (see wikipedia MD5). Tried two different “approved” ways to edit the system-auth PAM file. One, use [#chmod 644 system-auth] to set permissions on the file so that it is user editable (or just use :wq! after editing). Two, use [#chmod +t system-auth] before editing. Unfortunately, after a reboot the system-auth file returns back to its pre-edited version.
I opened an SR with VMware and they in turn opened a PR and shortly thereafter confirmed the issue is not by design and in-fact is a bug. VMware estimates that this issue will be resolved in the Update 2 release of vSphere ESXi 4.1. Haven’t had a chance to see if the issue is present in vSphere ESXi 5.0.