EMC Symmetrix bit/flag settings for VMWare ESX

 

ESX requires the following bits to be enabled: EAN, PP, UWN, VCM, C, SC3 and SPC-2

you should also disable the D bit if it is set

 

Best practice is to enable these bits on a "per HBA" basis

 

 

 

Checking bits / flags:

 

Checking bit settings per-FA on array #### :

symcfg -sid ####  list -fa <7a> -p <1> -v

 

Checking bit settings per HBA-level on host:

symmaskdb -sid #### list database -v -wwn <host HBA pwwn>

 

 

Recommendation:

 

1. Disable the D-bit on a per-HBA basis using symmask command.

symmask -sid ####  -wwn <wwn_of_VMwareHBA>  set hba_flags on D -disable  -dir <4a> -p <0 | 1>

Repeat for every HBA WWN in the cluster.

 

2. Enable EAN, SPC-2 and SC3 bits on a per HBA basis using symmask command.

symmask -sid ####  -wwn <HBA pwwn>  set hba_flags on SC3,SPC2,EAN -enable  -dir <4a> -p <0 | 1>

Repeat for every HBA WWN in the cluster

 

Additional Information:

 

Auto negotiation (EAN)

  • When EAN is enabled, the Symmetrix FA/director port negotiates to the speed of the switch port to which it is connected. You can hard code the speed of the Symmetrix port to 1GB, 2GB, or 4GB depending on the Symmetrix hardware.
  • Changing this setting causes the Symmetrix FA/director to log out of a fibre-channel switch and log in again. Downtime is not required.
  • Enabling or changing this setting causes a temporary loss of service for all connected devices. This does not cause snapshot LUNs.

 

SCSI-3 (SC3)

  • When enabled, SCSI-3 provides SCSI-3 support for attached ESX hosts.
  • This bit is required for ESX 2.5, 3.0, and 3.5, although ESX 2.5 and ESX 3.X do not use the full range of SCSI-3 commands.
  • The SC3 bit must be set so that Linux virtual machines using RDMs to the Symmetrix LUNs treat the LUNs correctly. This has to do with Linux guest operating systems issuing an INQUIRY SCSI command rather than a REPORT_LUNS command, if a Symmetrix LUN is detected.
  • Enabling this setting is a non-trivial change. Changing this bit requires ESX host downtime.
  • Enabling the SC3 bit triggers a LUN UUID change on a vmfs3 datastore. For more information, see  Resignaturing VMFS3 Volumes that are not Snapshots (9453805).
  • Enabling the SC3 bit alters the ID of the LUN  presented to the ESX host as this change alters the ANSI version of the LUN presented from Version 0x02 to Version 0x03.
  • To ensure the ESX server negotiates the correct ANSI version (0x3), VMware recommends that you force the ESX host to renegotiate the LUN(s). There are numerous ways to do this. Here are two options:
    1. Reboot the affected ESX host(s)
    2. Make the affected FA port go offline. Make it go back online, then issue a rescan from the attached ESX hosts(s)

 

SPC-2 (SPC2)

 

  • Enabling the SPC-2 director flag for iSCSI and Fibre Channel directors allows Symmetrix FA/Director ports to conform to the SPC-2 (SCSI-3 Primary Commands) to industry specifications and standards. This bit is required for ESX 3.x.
  • Enabling this bit is a non-trivial change. Changing this bit requires ESX host downtime.
  • This is not an online change. The ESX host needs to be rebooted.
  • Enabling the SPC-2 bit alters the ID of the LUN presented to the ESX host as this change alters the ANSI version of the LUN presented from Version 0x03 to Version 0x04. This means that once the SPC-2 bit is enabled, the LUN is SCSI-3 compliant.
  • The NAA identifier on page 0x83 is used for identification instead of the LUN serial number stored on page 0x80. This enables ESX 3.x systems to use the industry standard for LUN identification by extracting the inquiry data from a device on INQUIRY page 0x83. With the SPC-2 flag set, the Symmetrix array returns an NAA identifier. Without the SPC-2 flag set, the NAA identifier is not be returned.
  • Enabling the SPC-2 bit triggers a LUN UUID change on a vmfs3 datastore. For more information, see  Resignaturing VMFS3 Volumes that are not Snapshots (9453805).

 

 Notes :

 

  • See WhitePaper Enabling SPC-2 Compliancy on EMC Symmetrix devices connected to VMware environments.
  • If there is a requirement to change both the SC3 and SPC-2 FA director flag settings, VMware recommends that you enable both FA director flags at the same time across all FA ports connected to ESX systems. This is an offline change.