Wednesday, August 31, 2011

Veritas Storage Foundation and NTFS File Cluster Sizing


I ran a few tests recently to figure out if I/Os might be crossing cache boundaries when the NTFS file cluster size is increased beyond 32Kb. Veritas Storage Foundation VEA default Track Alignment offset for EMC Symmetrix arrays (we use VMAX) is 64 sectors (32 Kbytes). In theory, using a 64Kb NTFS file cluster size may push I/Os across storage cache boundaries and reduce throughput. For more information on the effects of incorrect Track Alignment see here.

This seems to be the case based on the throughput improvements seen below with 64k cluster sizing and 2048 sector offset, see below. Increasing the Track Offset to 2048 sectors increases throughput substantially.

If we start using >32Kb NTFS cluster sizes on any of our Veritas equipped Wintel fleet we'll need to alter the default Track Alignment settings in VEA from 64 to 2048 sectors.



Friday, August 26, 2011

New white paper providing guidance for sizing NTFS volumes

Multipathing Support in Windows Server 2008

Understanding Windows Server MPIO Features and Components

Windows Server Device Management Architecture

Understanding Requirements for Windows Server 2008 Failover Clusters

How the Cluster service reserves a disk and brings a disk online

Friday, August 19, 2011

Checking SCSI3 Persistent Reservation Status of a LUN

The Veritas supplied SCSIcmd utility provides a useful way to confirm if a Symmetrix presented LUN has the SCSI3 Persistent Reservation attribute set. This is requirement for Windows Server Clustering. Use Emulex HBAnyware/OC Manager to identify the correct LUN Id (E.g. 9) to use with the -d switch.

> scsicmd -d9 -sscsi3_test

All tests should PASS.

EMC Related Blogs

Non-disruptive expansion of a Windows Server 2008 disk volume with EMC Symmetrix VMAX

We've been looking for a way to avoid purchasing Veritas Storage Foundation for our Windows cluster servers, we use Veritas to manage non-disruptive volume expansion and it works very well but is expensive.

Windows 2008 server now provides a relatively mature and supported toolset for managing SAN attached storage at no additional cost, why not use it?

Meta expansion has been around for some time on EMC Symmetrix but is not something we have used. Our goal was to prove, test and create a procedure for non-disruptively expanding Symmetrix Meta luns we present to Windows 2008 Cluster Servers.

Tested Pre-requisites:
  • Windows Server 2008 Server R2 x64
  • NTFS file systems
  • EMC SYMCLI 7.1.2.0
  • EMC Symmetric Management Console (SMC) v7.1.2.1
  • EMC Powerpath v5.3.SP1.b009
  • Symmetrix VMAX1
  • EMC Symmetrix, SMC, SYMCLI, Windows 2008 Server admin experience
Objectives:
  • Expand Thickly provisioned Symdevs without a service outage.
  • Grow a single presented Symdev online - 100GB symdev to 200GB meta.
  • Grow a presented concatenated Meta Lun online - 200GB meta to 300GB meta.
  • Convert a concatenated Meta Lun to a striped Meta Lun online.
Online/Non-disruptive = Windows Server Cluster is Online and running workload, no loss of data from file systems.

Grow a single presented Symdev online
This is not possible without a service outage. However, it is possible with a service outage. The single Symdev must be unpresented from the host, that is, remove from the SMC Storage Group and Unmap from the storage ports.
  1. Offline the Disk Resource in Windows Cluster Management
  2. If required, Offline the Disk Volume in Windows Disk Management
  3. Unpresent/unmap the Symdev by removing it from the Storage Group or via SYMCLI.
  4. Form a new meta from the existing Symdev as the head and add a new Symdev.
  5. Present the new meta device, adding the meta to the Storage Group using SMC.
  6. Online the expanded disk in Disk Management, you should now see a 200GB disk with 100GB unallocated.
  7. Expand the filesystem using Disk Management tools.
  8. Remember, Windows 2008 Cluster requires the SCSI-3 Persistent Reservation attribute is set on the presented device. Check the setting if necessary.
Grow a presented concatenated Meta Lun online - 200GB meta to 300GB meta. This is supported using the SYMCLI but not from the SMC GUI.
  1. Windows Server 2008 Cluster is online and processing workload.
  2. Use the SYMCLI to set SCSI-3 Persistent Reservation.
  3. Add a device to the existing meta. Note that I could not get the SMC GUI to do this, I had to use the SYMCLI. Not sure if this is related to the version we are running or unsupported in the SMC GUI. If you do this regularly you can place the commands in quotes in the above screenshots in a 'commands' text file and run symconfigure -sid xxx -f textfile.txt commit, I chose to run them inline.
Convert a concatenated Meta Lun to a striped Meta Lun online
TBD

Lessons Learnt, Caveats, Questions
All of the following apply to the above tested scenarios.
  • An existing single symdev can be safely expanded with a service outage, not Online.
  • An existing concatenated meta device can be expanded Online without service disruption.
  • An existing striped meta device can be expanded online.
  • An existing data device can safely be used as a metahead.
  • Ensure you are working with a Concatenated Meta, we were unable to confirm you can expand thickly provisioned Striped Metas, seems unlikely. You can however, expand a concat and then convert it to a striped meta or do it at the host level
  • Works with Concatenated Metas
  • Does not work on SRDF devices, Cloned devices, only STD devices
  • Expanding thin concat AND striped metas is supported in Solutions Enabler (SYMCLI) 7.2 and Enginuity 5875
References

Powerlink Solutions
  • emc223563 - How can I expand a LUN into a meta LUN while preserving data?
  • emc269273 - Symmetrix VMAX: What are the restrictions on thin meta expansion?

Thursday, August 18, 2011

Australian Storage Blog

An Australian storage perspective - IBM and Brocade centric...

http://aussiestorageblog.wordpress.com/

Multipathing in Windows Server 2008

There is a number of enhancements to Multipathing (MPIO) architecture for Storage systems in Windows Server 2008. New features include the Microsoft Device Specific Module (DSM) designed to work with a variety of vendor specific arrays and multipathing solutions. See the following Microsoft Technet document for full details:-

Multipath I/O Overview

EMC Powerpath users need to be aware of the DSM feature, checking/changing path fail-over settings is now done using the DSM functionality under the Multipath device, in this screenshot, Device Manager>Disk Drives>EMC Symmetrix Multi-Path Disk.

EMC Symmetrix with Windows Server Best Practices

Anyone responsible for implementing or administering large EMC storage or Windows server environments should be familiar with the details of this document:
EMC Symmetrix with Microsoft Windows Server 2002 or 2008: Best Practices Planning