“Active Directory Web Services encountered an error while reading the settings for the specified Active Directory Lightweight Directory Services instance.  Active Directory Web Services will retry this operation periodically. In the mean time, this instance will be ignored. Instance name: ADAM_VMwareVCMSDS”

This event by itself is not something that should cause you to believe your vCenter installation is not working properly. Essentially what is occurring is that the VMwareVCMSDS ADAM instance does not have a valid SSL Port set within the registry. You can confirm this by browsing to the following registry location on your vCenter server:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\ADAM_VMwareVCMSDS\Parameters]

You should see that the Port SSL registry entry is either missing or does not contain a value. You can simply add the REG_DWORD value while you are already in the registry with the value of 636. Or you can use the below Microsoft approved method to add the port value.

Add ADAM_VMwareVCMSDS SSL Port Value

1. Stop the vCenter ADAM instance within the Microsoft Services Control Panel. The service name is VMwareVCMSDS. Optionally, you simply run net stop vmwarevcmsds from an elevated command prompt.
2. At a command prompt, type dsdbutil.
3. Within the dsdbutil, type activate instance VMwareVCMSDS.
4. Type SSL port 636.
5. Type Quit.
6. Type net start vmwarevcmsds.

Once the service starts, I highly suggest you reboot the vCenter server to allow all the vCenter services as well as the Microsoft Active Directory Web Services service to restart. If you don’t reboot the server you may encounter a fairly nondescript error message when attempting to logon using the vSphere Client until you reboot.

VMware has issued a KB for this particular issue:
Active Directory Web Services fails to read the settings for the specified Active Directory Lightweight Directory Services instance

After following the above steps your vCenter server will now be using the new certificate for all web services. Additionally, you will no longer be presented with a certificate warning popup when using the vSphere Client if the certificate authority that issued the replacement certificate is trusted by your computer (in this specific case any domain joined computers will automatically trust all certificates issued by your internal enterprise Microsoft Certificate Authority).

I opened an SR with VMware and they in turn opened a PR and shortly thereafter confirmed the issue is not by design and in-fact is a bug. VMware estimates that this issue will be resolved in the Update 2 release of vSphere ESXi 4.1. Haven’t had a chance to see if the issue is present in vSphere ESXi 5.0.

Microsoft surely must have provided a way to directly manage their CA implementation through script…right? Unfortunately they didn’t. Microsoft provides access to their CA through a command-line tool “certutil” and also minimally through COM. Management through COM is possible but likely a complicated endeavor for the non-programmer. Ideally, Microsoft would have provided an easily accessible API that’s consumable by VBScript or Powershell. However, with recent changes to the certutil command-line tool in Windows Server 2008, Windows Vista and Windows 7, and some quick Powershell magic a decent certificate management tool is possible.

The following Powershell script exports the full list of issued certificates from a Microsoft CA using certutil and saves the certificate information as a CSV file. The CSV file is then imported and each certificate is checked to see if it falls within the certificate expiration window (definable in script). An e-mail is generated that contains basic certificate information for certificates that fall within the expiration window. This script can be scheduled to run using Microsoft Task Scheduler. Remember, this will only work if run on a Windows Server 2008, Windows Vista or Windows 7 computer due to the “CSV” option only being available in later versions of certutil.

# variables
[string]$caServerName = "<your ca server name\ca name>"
[string]$caCertExportPath = "c:\Temp\certlist.csv"
[string]$smtpSender = "no-reply@your-domain.com"
[string]$smtpRecipient = "user@your-domain.com"
[string]$smtpServer = "<fqdn of SMTP server>"
[int]$daysUntilExpiry = 30
$expiringCerts = @()

function Send-EmailCertNotice ([string]$_certificateList) {
$MailMessage = @{
    To = $smtpRecipient
    From = $smtpSender
    Subject = @"
The following issued digital certificates will expire soon.
"@
    Body = @"
The following digital certificates issued by <your_company_name> will expire in the next ($daysUntilExpiry) days.
Please request a certificate replacement/renewal from <e-mail@your-domain.com> if the following certificates are still needed.
$_certificateList
NOTE: This notification is being sent by an automated certificate management process and
cannot receive reply e-mail. If you have any questions please contact <e-mail@your-domain.com>.
"@
    Smtpserver = $smtpServer
	BodyAsHtml = $false
    ErrorAction = "SilentlyContinue"
	}

Send-MailMessage @MailMessage

}

# export certs to CSV file
certutil -view -config $caServerName csv > $caCertExportPath

# load cert CSV into an array
$issuedCerts = Import-Csv $caCertExportPath
if ($issuedCerts.Length -gt 0) {
	foreach ($cert in $issuedCerts) {
		try {
			$certExpires = [datetime]$cert."Certificate Expiration Date"
			$cert."Certificate Expiration Date" = $certExpires
		}
		catch [Exception] {

		}

		if ($certExpires -gt $(Get-Date) -and $certExpires -lt $(Get-Date).AddDays($daysUntilExpiry)) {
# filter out EFS type certs.
			if ($cert."Certificate Template" -ne "EFS") {
				$expiringCerts += $cert
			}
		}
	}

	$bodyVal = $expiringCerts | Select-Object @{n="Certificate ID"; e="Request ID"},
	"Certificate Template", "Certificate Expiration Date", "Issued Common Name",
	"Serial Number" | Sort-Object "Certificate Expiration Date" | Out-String
	Send-EmailCertNotice $bodyVal

}

This document is the public draft release of the vSphere 4.1 Security Hardening Guide.  This guide is an incremental update to the vSphere 4.0 Security Hardening Guide based on new and changed features of vSphere.  Please provide your feedback in the comments section.  This draft will remain posted for comments until approximately the end of February 2011.

Link to document: vSphere 4.1 Security Hardening Guide (draft)

Of all the browser options, I tend to enjoy Google’s Chrome browser the most. Though, I keep giving up on Chrome because I use a lot of bookmarks and Chrome has historically handled bookmarks poorly. For example, within the native Chrome UI one must set the homepage to “Use the new tab page” in order to see a tree-view style bookmarks menu. Alternately, a Chrome user could access his bookmarks using the Bookmark Manager, though, the Bookmark Manager by-design fills the entire web UI and clicking on any one bookmark opens the webpage in a new tab instead of in the same tab the Bookmark Manager is running in–not a sleek implementation in my opinion.

Funny thing, even though I already extend Chrome with extensions I never considered looking for a Chrome extension that provided a tree-view style bookmark menu until a couple days ago. I stumbled across an extension called Neat Bookmarks which provides a great bookmark UI experience similar to IE’s bookmark tree-view and is accessible as a single toolbar button. If you are looking for a more traditional bookmark menu in Chrome I recommend you give Neat Bookmarks a try. I hope now that I have found a solution to my bookmark woes in Chrome I can start focusing on other issues other than browser brand.

Check it out here: Maximizing VM Performance

I also wonder if VMware has given up on the vCenter Mobile Access (vCMA) product since it is still in a community technology preview version, there have been no major enhancements, and its been a long time since it was introduced. The vCMA had the “cool” factor when it was released–I remember showing people how I could vMotion a VM from one ESX host to another from my Blackberry. That cool factor faded away to the point where I haven’t used the vCMA in over a year–it’s just too kludgy to get anything done. Is the iPad vSphere application the new vCMA–the new vendor specific application that will introduce the iPad into corporate virtualization environments? Will it take over the functionality of the vCMA?

Don’t get me wrong, I see tremendous possibilities for the iPad within the corporate environment. The VMware vSphere iPad application could be very useful to large organizations that have lots of ESX hosts. Imagine an administrator being able to evacuate and place an ESX host in maintenance mode while troubleshooting a hardware issue within the datacenter (or from Hawaii on business). I can even see the VMware vSphere iPad application allowing virtualization administrators to manage a significant portion of their daily workload away from the office.

In conclusion, it’s great that VMware is working on new innovative ways to enhance access to vCenter from various devices; however, if I had my way I would rather VMware spend more time doing the following (in this order):

1. Enhance the vCenter product for Windows. When I say “enhance” I mean work on the fit-and-finish of the product. All too often I am presented with ambiguous error messages or stumble on a failed process, yet the event reporting within vCenter can’t seem to tell me what’s wrong.
2. Work on the overall performance of the vCenter UI. There are reports all over the Internet of the horrific performance within the vCenter user interface. I see it everyday. Viewing inventory takes 10 seconds to load up once the vCenter interface is visible (this doesn’t count the time to logon and load the plugins). Granted, performance is linked to hardware specifics and one must build an appropriate server environment to support vCenter; though, I am talking about poor performance on vCenter servers running with new multiple Xeon quad-cores with 8GB+ physical memory with a large dedicated physical DB server back-end.
3. Enable performance monitoring across all hosts from a single UI window. Since a DRS cluster is essentially a pool of CPU and memory resources–why are we still required to troubleshoot performance by analyzing single ESX servers (think esxtop)?
4. Enhance command-line troubleshooting tools. For example, an esxtop command that has a global view of clusters and storage. Yes it’s great to see the read/write MBps to a specific VMFS LUN but I want to see the total across all hosts not just the localized view of a single ESX host.
5. Stop developing  new features that are only added to the growing list of VMware products including vCloud Director, vCloud Request Manager, Orchestrator, CapacityIQ, Site Recovery Manager, Lab Manager, and Configuration Manager. Put some of the features in vCenter for continued value-add. For example, why haven’t we seen simple Virtual Machine replication in vCenter?
6. Finish and release a fully functional Linux vCenter server with associated Linux vSphere client.
7. Create better quality upgrade and patch bundles. Why do customers cross their fingers hoping everything is going to work as expected after upgrading vCenter or an ESX host? How many times have I seen an upgrade break vCenter (for example, certificates, web services, health monitoring)? Answer, many times.
8. (last) Develop a mobile vSphere client.

It is probably safe to assume most VMware administrators have implemented vCenter Server as a virtual machine within a DRS/HA cluster. A virtual vCenter Server running within a DRS/HA cluster provides many great high-availability and manageability benefits; however, there is a specific challenge that has not been solvable until vCenter Server 4.1. In the event vCenter Server become unavailable, an administrator would need to connect directly to an ESX/ESXi host using the vSphere Client where vCenter is located to manage the server there (i.e. open a console connection, restart the vCenter server, power the vCenter server up, etc.). But because vCenter Server is running inside a DRS cluster it is sometimes very time consuming locating the specific ESX/ESXi host where vCenter is running if you have many ESX/ESXi hosts within the DRS/HA cluster. For example, if there are 12 ESX/ESXi hosts running within a DRS cluster; the vCenter server could be running on any one of the 12—could you imagine using the vSphere Client and connecting to up to 12 hosts before locating the vCenter Server? Could you afford wasting 20 minutes during an emergency trying to locate vCenter Server?

Using DRS Groups vCenter Server can be limited to run on a limited number of ESX/ESXi hosts within a DRS cluster. For example, using DRS Groups an administrator can designate three of the 12 hosts where vCenter Server can run. In the event vCenter Server becomes unavailable it would be much easier to locate vCenter if you know it is primarily running on any of three hosts instead of 12 hosts.

The following VMware KB article provides a starting point for you to further investigate DRS Groups: http://kb.vmware.com/kb/1022842 .

The second issue was in regards to troubleshooting a vMotion related problem with a virtual machine (well, what appeared at the time to be a vMotion issue).  Basically, the virtual machine would not vMotion regardless of what was tried.  Even after confirming no virtual devices were causing the problem the only solution was to power off the virtual machine and then perform the migration.  I attempted to review the virtual machine vmware.log file after the virtual machine was powered back on.  Unfortunately, the only way to read a vmware.log file is to view it directly from the console of the ESXi host that is running the virtual machine.  Because SSH is not supported in ESXi (yes, it can be enabled) I was not able to read the vmware.log file remotely.  There is no “supported” workaround to remotely view the vmware.log file when using ESXi.

These two issues alone can be deal-breakers for some.

UPDATE 1: VMware made huge steps towards closing the supportability and functionality gap with the ESXi 4.1 release. The two issues identified above have been mitigated as ESXi 4.1 allows supported command-line access locally and remotely via SSH. Additionally, I am happy to report the vscsiStats tool is now available and officially supported in ESXi 4.1 at /usr/lib/vmware/bin/vscsiStats. Great job VMware!!!