Optimizing VMware with EMC’s XtremIO

Paris Fraser

by Paris Fraser

EMC’s all flash array, XtremIO has quickly become the fastest selling storage array in EMC’s history. XtremIO is the # 1 all flash array in the market and is Gartner’s magic quadrant leader for all flash arrays.  XtremIO offers many benefits that make it an ideal solution for VMware environments.

Today, I’m going to discuss not only the benefits but also best practice parameter settings for the hosts and vCenter as well as configuration for VAAI, HBA and NMP for a VMware environment.

  • Exceptional scale-out performance – consistent sub 1 MS latency with mixed workloads
  • Best in breed data reduction – In-line deduplication and compression – Consolidate even the most IO-intensive servers and application. We are consistently seeing customers with virtual environments achieve space savings anywhere from 4:1 up to 9:1 (almost 90%)
  • Space efficient writable snapshots
  • 100% thin provisioning
  • XtremIO Data protection – XDP – Performance of RAID1, efficiency of RAID 5, Redundancy of RAID 6
  • Ease of management – No pools, RAID groups, or hot spares to configure, no tuning, provisioning is a simple 3 step process.
  • More power and space efficient
  • In-line data at rest encryption
  • Full VAAI integration – offload storage tasks from the server to the XtremIO – near instantaneous clones
  • All data services are always on and in-line with no performance impact

Many of my recent XtremIO implementation projects have included host integration with VMware.  To optimize the benefits of XtremIO and maximize your investment, it is important to follow EMC’s recommended ESX host settings (included in EMC XtremIO Users Guide).
Note: These settings may cause hosts to over-stress other arrays connected to the ESX hosts, resulting in performance degradation while communicating with them.  In mixed environments with multiple arrays, compare these recommendations with those of other platforms before applying them.

Host Parameter Settings

  • Disk.SchedNumReqOutstanding – Determines the maximum number of active storage commands (I/Os) allowed at any given time at the VMkernel. Set the parameter to the maximum value of 256. Note: When using vSphere 5.5 this parameter can be set on a specific volume rather than on all volumes presented to the host. It should be set only after XtremIO volumes are presented to the ESX host.
  • Disk.SchedQuantum – Determines the maximum number of consecutive “sequential” I/Os allowed from one VM before switching to another VM (unless this is the only VM on the LUN). Set the parameter to the maximum value of 64.
  • Disk.DiskMaxIOSize – Determines the maximum I/O request size passed to storage devices. Change the parameter from 32767 (default setting of 32MB) to 4096 (4MB).

Note: For details on adjusting the maximum I/O block size in ESX, refer to VMware KB article 1003469.

vCenter Server Parameter Settings

  • The maximum number of concurrent full cloning operations should be adjusted. The vCenter Server parameter config.vpxd.ResourceManager.maxCostPerHost determines the maximum number of concurrent full clone operations allowed (the default value is 8). Adjusting the parameter should be based on the XtremIO cluster size as follows:
    • One X-Brick – 8 concurrent full clone operations
    • Two X-Bricks – 16 concurrent full clone operations
    • Four X-Bricks – 32 concurrent full clone operations
    • Six X-Bricks – 48 concurrent full clone operations

 VAAI Settings with vSphere 5.x

Validate that VAAI is enabled – the following parameters should be set to 1:

  •  DataMover.HardwareAcceleratedMove
  • DataMover.HardwareAcceleratedInit
  • VMFS3.HardwareAcceleratedLocking

 Fibre Channel HBA Configuration

  • Qlogic – Execution Throttle – Change the default value (32) to 65536
  • Emulex – lpfc_hba_queue_depth – No need to change the default and maximum value (8192).
  • Cisco UCS FNIC – Adjust the I/O throttle to 1024. Note: This setting adjustment apply to VMware vSphere only. This is a global setting to the UCS chassis and may impact other blades in the chassis running a different OS.

Note: For details on adjusting HBA queue depth in ESX, refer to VMware KB article 1267

Native Multipathing Configuration (NMP)

  • Set the native round robin path selection policy on XtremIO volumes presented to the ESX host.
  • Set the vSphere NMP Round Robin path switching on XtremIO volumes from the default value (1000 I/O packets) to 1.

 Virtual Machine Formatting

  • For optimal performance, format virtual machines using Thick Provision Eager Zeroed.
  • Logical space is allocated and zeroed out when the virtual machine is created.
  • Allows the XtremIO to manage the thin provisioned volumes.
  • Cloning performance is significantly improved.

I hope this makes implementing XtremIO with VMware easier for you.  If you are interested in hearing more about XtremIO or if you would like a storage assessment to see if XtremIO would be a good fit in your environment, let us know so we can help!