Posts Tagged ‘esxtop’

Effectively Associating NAA Identifiers to Datastores for Troubleshooting

Posted on the April 7th, 2010 under VMware by

Now that you know what NAA identifiers are you may be wondering how to use NAAs in a meaningful way.  You will likely utilize the NAA numbers when troubleshooting storage issues.  For example, when a VM is performing poorly you will want to find out specific storage statistics related to the troublesome VM in question.  Using the NAA identifier directly when troubleshooting is very difficult since the identifier is quite long.  To make matters worse, each LUN NAA is very similar when multiple LUNs are presented from the same storage array.  Another example is when troubleshooting storage issues related to SCSI reservations or sense code type problems; the VMware vmkernel and messages logs report storage issues by NAA in most cases.

Let’s assume you have a Virtual Machine named “ServerA” and it resides on a Datastore named “RAID_5_15K”.  On Monday when you arrive to work you are greeted with customer complaints that ServerA is performing poorly.  You start by checking the usual suspects; CPU and Memory.  Both CPU and Memory look to be operating within normal ranges.  So you think to yourself the performance issue may be related to storage.  In vCenter you begin looking through the graphs related to storage and you quickly realize the graphs are showing LUN information by NAA identifier.  At this point you are not sure which NAA is related to ServerA because all you know is that ServerA resides on Datastore “RAID_5_15K”.

Fortunately, VMware provides a couple different ways to map the NAA identifiers to Datastores—none of which are available through the built-in graphing tools.  The most difficult way to identify the NAA to Datastore mapping is to use the vSphere client and by browsing to the ESX host Configuration tab and selecting Storage.  In the list of Datastores you will see a column with the friendly user-defined Datastore name as well as the NAA identifier for each Datastore.  A quicker way to find the NAA to Datastore mapping is to run the ESX(i) command “esxcfg-scsidevs -m” at the ESX Service Console or remotely using the vMA or Remote CLI and grepping (UNIX: grep) the Datastore in question.  The esxcfg-scsidevs -m command provides a text based reference of all Datastores and their associated NAA identifiers.  You can copy this information to a document and keep it around for quick reference.

Now that you have the NAA to Datastore map you can quickly identify the NAA that belongs to the Datastore “RAID_5_15K”.  Using either the built-in graphing functionality in the vSphere Client GUI or the more robust real-time performance tool ESXTOP you can effectively identify storage issues that may be negatively impacting ServerA.

Device Name (NAA) Truncated in ESXTOP

Posted on the April 5th, 2010 under VMware by

With the introduction of ESX(i) 4 and the included support of the SCSI-3 protocol comes a new way of displaying LUN information in the virtual environment called Network Naming Address Authority (NAA) identifiers.  Each Fiber Channel or iSCSI LUN presented to an ESX(i) host has a unique SCSI NAA identifier.  The NAA for each LUN is reported by the storage device to the ESX(i) host through an INQUERY which returns a VPD (Vital Product Data) response that includes the NAA identifier.  Although the NAA is unique for each LUN, the NAA should remain consistent across all hosts for any particular LUN.  The NAA identifier is generated by combining the word naa. with the 128-bit WWN of the LUN (ex. naa.600601607a401900d41ffaa5c17add11).  In the preceding example, you can see the LUN is first identified by the word NAA; it is then followed by the WWN of the LUN.  Because the WWN contains the IEEE assigned vendor OUI 006001 you can use a tool to find the LUN storage vendor (in this case “CLARIION”).

The length of the NAA presents a problem for the ESXTOP tool.  ESXTOP is designed to show shorter device names (as seen in ESX 3.5 and earlier) when viewing “disk device” information.  When you view disk device information through ESXTOP on an ESX(i) 4 host the NAA identifier is truncated which prevents the ability to identify a specific LUN (see truncated example below).  ESXTOP contains an “L” option which allows users to specify the length of the Device field.  By pressing the “L” key while in the “disk device” view and setting the value to 40, the NAA of each LUN become visible (see expanded example below).

Truncated Expanded