Definition

It’s worth pausing for a moment and discussing the terms Chargeback and Billing.  When computing was first made available as timesharing, customers were billed for their usage of the shared system.  The billing unit may have been time, CPU resources or some combination of metrics that represented utilisation.  Mainframe resources were so expensive that there had to be an efficient charging mechanism.   The concept of billing is something that was intrisically built into the mainframe design and even to this day, resources can be tracked using records produced by SMF (System Management Facility) and reported on through RMF (Resource Measurement Facility).  So billing represented a method of charging for usage that wasn’t directly related to the underlying hardware.

Chargeback implies a different methodology where the direct cost of delivering the service is charged back to the customer.  This can include people costs, but typically hasn’t, only covering the hardware provided itself.  Chargeback has its place, but when looking to develop a service, isn’t as flexible as billing.  All too often, chargeback is tied to a poorly implemented service catalog (or non-existent one).  Whilst the customer may pay for their equipment, there isn’t any flexibility when it comes to hardware replacement as the customer is aware of the technlogy used to deliver their service (and may be unwilling to move to new, untried hardware).  Here are a few additional chargeback/billing combinations that could be implemented:

• No chargeback – IT has a budget and they provide the resources to the business.  When resources are exhausted, the business have to justify or provide additional funds.
• Consumption-based – customers are charged directly for their usage.
• Shared-usage – customers are charged a share of the costs, not directly related to their usage, but perhaps size of business unit.
• Dedicated – customers are charged the whole cost of acquiring the technology.  Ths doesn’t work well for shared environments.
• Service-based – customers are charged for a service provided; this isn’t directly related to the specific technology in use.

Rationale

Whether you are implementing chargeback or billing, there needs to be a good reason for implementing.  Here are a few for consideration.

• To Reduce Costs – If resources appear to be free they will be consumed inefficiently; charging for usage helps controls this.
• Improved Utilisation – Being charged in proportion to your usage makes customers validate whether they really need the storage they are using.
• Improved Efficiency – this goes hand in hand with utilisation, however charging customers for storage can enable tiering to be implemented more efficiently.
• Charging Fairly – there will always be sensible customers and abusers (the broadband market shows us that).
• Manage Demand – It is possible to make charges both time and planning dependent (more on that later).
• Manage Tech Refresh – Abstracting cost and service catalogue from the hardware means new/cheaper/efficient technology can be introduced more easily.

What’s clear from the above points is that chargeback/billing can be used to change customer behaviour; users can be incentivised to be more efficient or to use cheaper technology.  Structured correctly, the overall cost of delivery of storage can include refresh funding, so as old devices are decommissioned, the cost of data migration is part of the overall charge.  I see this as one of the major issues with the way customers pay for their technology; the overall costs in the lifecycle of deployment, operation and refresh simply aren’t considered.

Metrics

What’s the best way to charge?  Here are a few typical metrics:

• Per GB of storage used.
• Per port on the SAN fabric.
• By Tier of storage.
• By contention ratio of storage port (higher cost for fewer hosts on a shared port)
• Charge for replication (both local and remote)
• Charge for deduplication (which may be a lower cost)
• Charge for thin versus thick provisioned LUNs
• Charge for SAN network bandwidth
• Charge for multi-path software
• charge for online backup copies
• charge for offline backup copies

Whatever metrics are used, the key intent is to charge for customer use of a service.  This needs to be abstract enough to be disconnected from technology, so charging for fibre channel ports may be too prescriptive; the cost may be described as “to be connected to the SAN” in general, providing a blended charge that would cover iSCSI, Fibre Channel or FCoE connectivity.

Implementing a Chargeback Process

As part of the implementation process, it’s worth considering having billing/chargeback principles established.  These can be provided to the customer.  Here are some examples:

• The charging model will be based on resource consumption of each user independently (e.g. user changing their utilisation doesn’t affect another user)
• Charging costs will be reviewed and changed on an annual/bi-annual/quarterly basis from 1 Jan 200x
• Charging will be based on storage in use on 28th day of each month
• Charging will/will not be based on utilisation (rather than allocation)
• Charging will be attributed at the host/server/LUN/file level
• A target of 100% cost recovery is the target goal
• Charging may result in an IT surplus/deficit from year to year, but will be a non-profit business
• Billing charges will be based on the published “Storage Catalogue”
• Users of equipment classed as legacy will be notified 6 months in advance of technology acquiring legacy status
• IT/Storage Team will strive to deliver price stability and/or reductions year-on-year
• Chargeback will be implemented as evolution rather than revolution

The internal cost of delivery of storage will include:

• Hardware and software costs
• Additional feature licences
• Power/cooling/space (environmental costs)
• People costs
• Training
• Network costs
• DR costs

There may be more, depending on how your technology is delivered (for instance managed data centres), but what’s essential is to baseline what it takes to deliver the service.  Quite simply the process would be:

1. Identify service cost components (as above)
2. Identify consumption metrics (service charging units)
3. Measure use
4. Model costs based on consumption metrics
5. Bill customers.

Other considerations, which I’ll save for future posts are:

• Standards – how they are important to chargeback
• Measuring tools
• Measurement interval
• Incentivising customer behaviour in favour of technology refresh
• Outsourcing some components
• Determining the customer
• Forecasting/Capacity Planning

There’s lots more to come, feedback on the article so far is very welcome.

© Chris for Gestalt IT, 2010. | Four Pillars – Service: Chargeback
Read more posts categorized as Featured, Storage

FAST (FULLY AUTOMATED STORAGE TIERING)

FAST made a debut in the storage market yesterday (12/08/09). Finally after the market buzz we got a preview of the product in terms of its features, functionality, characteristics, possible shortcomings and use cases.

This blog post focuses on the features, the drawbacks and some applications around FAST.  By no means is this a comprehensive or an exhaustive list of the above.

After all, FAST makes a debut, (The previous post on FAST).

..

NOTE: Out of the box thinking by EMC, imagine the flexibility one would have with a large infrastructure and FAST moving data based on policy. Its all about the big picture.

..

..

Here are some features, highlights of FAST and how it operates.

• FAST will perform data movement based on IOPS, average I/O size and write percentage. This is currently true for the Symmetrix V-Max, Clariion CX4 and Celerra NS.

• Three elements that define FAST: Storage Type, FAST policies and Storage Groups.

• FAST is based on user defined – configuration policies.

• The configuration of FAST is typically done through FAST wizards (Symmetrix Management Console) on Symmetrix V-Max, FAST LUN Migrator for Clariion CX4 and Rainfinity File Management Appliance or VE for Celerra NS.

....

..

..

• User defined analysis period for FAST. That will enable FAST to recommend or perform data analysis and then a data move based on policy.

• FAST created policies will associate with Storage Groups.

• FAST policies will be configurable at a LUN / drive / drive type / speed etc level.

• Data movement will take place based on a time of the date policy called “COLD” move or on usage policy called “HOT” move.

• Data movement will take place between various drive types, various LUN types and LUN sizes. (LUN types, LUN sizes will need to be same for the source and destination LUNs). For example a 9GB FBA LUN being migrated from Fibre to FLASH will need similar source and destination LUN properties.

• FAST data analysis will be performed in the background.

• For Symmetrix V-Max platform FAST will perform all analysis without the use of Symmetrix Performance Analyzer. Understanding is there will be some sort of API plugin available on the Service Processor of the V-Max that will enable Symmetrix Management Console (FAST plugin) to interface with the Symmetrix through the SYMAPI interface.

• For Clariions, the performance data of the array will be monitored and collected by Navisphere Analyzer.

• Based on the source LUN analysis, FAST will recommend the user to move the data either a faster speed drives or a slower speed drives.

• FAST will enable roll back based on user preferences (automated).

• FAST will be configurable by Symmetrix Management console wizards or SYMCLI

• FAST configurable by Clariion Naviseccli and installed on a host connected to Clariion, Host software called FAST LUN Migrator.

• FAST configurable by Rainfinity File Management Appliance GUI or CLI for Celerra NS “out-of-box data” movement. This enables the Celerra to move data to another Celerra or Centera or Atmos.

• FAST configurable by Rainfinity File Management / VE (Virtual Appliance) for Celerra NS “in-the-box” data movement.

• FAST can be installed non-disruptively on all platforms.

• FAST will operate both at an FBA and CKD level supporting open systems and mainframes for V-Max.

• FAST will operate at a LUN level on Symmetrix V-Max

• FAST will operate at a LUN level on Clariion CX4

• FAST will operate at a file level on Celerra NS

• FAST v1 users will be able to purchase a FAST v2 upgrade when it’s released in second half of 2010.

• FAST can be purchased as a FAST suite or part of an ATSM (Advanced Tiering Storage Management) suite with bundled discounting prices.

• For a unified storage system like a front end NAS (Celerra) with a backend SAN (Clariion), FAST can coexists at both levels. But it is not recommended to deploy FAST at a Celerra LUN level.

• FAST integrates with Symmetrix Management Console and with Rainfinity GUI (Celerra) for simple management. Though Clariion implementation will need one to specialize in CLI.

.

FAST introduction by EMC

..

..

FAST Drawbacks

Here are some drawbacks of FAST as I see it today.

• Does not support Virtual provisioning. So the Virtual provisioned LUNs will not be FAST enabled. Sub-LUN expected in second half of 2010

• FAST is not free and is charged based on RAW CAPACITY of the Storage Array.

• FAST will only work with similar LUN types (example FBA LUNs can be migrated to FBA) and LUN sizes (9GB LUN can be migrated to a 9GB LUN) only.

• FAST will require Professional Services and is not recommended for customer self implementation at least for the Clariion and the Celerra platforms. EMC is making a claim that FAST can be self provisioned on the Symmetrix V-Max platform.

• FAST works only on current generation systems like EMC V-Max Enginuity 5874, Clariion CX4 Release 29 and Celerra NS.

• With Clariion CX4 and FAST implementation, requirement is to have Navisphere Analyzer.

• With Celerra NS and FAST implementation, requirement is to have Rainfinity File Management /VE or Appliance.

• With Symmetrix V-Max FAST implementation, requirement is to have Symmetrix Management Console (Not free any more starting with the V-Max)

• For the Clariions, FAST only analyzes Fibre drives and LUN movement has to initiate from Fibre channel to FLASH or Fibre channel to SATA drives only. Movement from FLASH to Fibre or SATA to Fibre has to be initiated manually.

• A customer implementing Virtual Provisioning on Clariion CX4 and Symmetrix V-Max that is using 50% Thick Provisioning and 50% Virtual Provisioning. FAST will not work with Virtual Provisioned LUNs. But the customer will probably pay for 100% of Storage Raw Capacity when it comes to paying for FAST either non-bundled or bundled in an ATSM (Advanced Tiering Storage Management).

• FAST does not support IBM System I platform currently.

..

..

Some real world FAST applications

• Acadia Service Model can now add granularity with offerings around on-demand resources using FAST at its core.

• On Demand application and workload needs can be met for OLTP, Data Warehousing, Mainframe compute and Virtual compute using FAST driven policies.

• Multi-tenancy with Private Clouds

• Move high demand data on faster drives while rarely used data goes on slower drives.

• Sell ITaaS based on SLA’s. Higher SLA’s can mean higher price. All automated processes controlled by policy.

• Higher transactions typically mean low overall cost; mean higher efficiency means higher profits all achievable through FAST policy engine.

Need to see some real world implementations of FAST now. In theory and on paper, FAST looks pretty compelling but practically will it do the magic.

..

..

Some unanswered questions today

• How does it prevent from LUNs jumping between Fiber, FLASH and SATA if the application has un-expected performance that day?

• Manageability of FAST interfaces.

• Usability of FAST interfaces.

• Granular configuration policies associated with analysis.

• Future upgrades.

• Current implementations and how long and how effective are those.

• Can you improve performance without using SSD’s. Example with Fibre and SATA drives only.

• With future upgrades from FAST v1 to FAST v2 what happens with current user defined policies.

• Would Symcli scripts change with FAST implementation?

• During lockout periods (Where Symmetrix configuration cannot change, eg before a BIN file change) would FAST still operate.

..

To read some of the answers to the questions, as commented by Barry Burke on the StorageNerve Blog

FAST is a bit of a new subject for me. Any experts please feel free to correct me if my understanding of FAST is incorrect at any level.

Stay tuned for a series of FAST posts over the next few days talking about various topics and how FAST plays within those areas.

© Devang for Gestalt IT, 2009. | FAST: Features, Drawbacks, Applications and some Questions
Read more posts categorized as Storage

I was talking to someone about policy driven storage management and what they described sounded awfully familiar, so I thought I’d do some quick googling and point them in the right direction. I found this series of articles on the IBM website.

It’s worth reading up on System-Managed-Storage; there are so many concepts within it which should strike a chord with you especially if you are thinking about what storage is going to look like in the future. And probably like a lot of Bods, you’ve never had the chance to be indoctrinated into ‘the Cult of the Mainframe’.

© Martin for Gestalt IT, 2009. | Driven by Past Policy
Read more posts categorized as Storage

Below are some stats related to the supported drive types on a Symmetrix V-Max system with 5874.123.104 microcode.

Possibly with introduction of FAST (Fully Automated Storage Tiering) later in the year we will see an upgrade to the microcode family for the V-Max systems to 5976, also with that expect a much denser EFD support.

In the mean time we should atleast see some additional support for VSphere 4.0 (Vmware) in 2009 with 5875 family of microcode. With that we should see sort of a new concept of Federation with Symmetrix V-Max Systems where EMC might give some clues on how the 8 engine systems might be expanded into either 16 or 32 engine systems. A nice blog post by @edsai on the breathing data site. http://breathingdata.com/?p=20

The following size drives types are supported with Symmetrix V-Max Systems at the current microcode 5874: 146 GB, 200 GB, 300 GB, 400 GB, 450 GB, 1000 GB.

Drive Types, Rotational Speed and Formatted Capacity

146 GB FC Drive

Drive Speed: 15K

Open Systems Format Cap: 143.53 GB

Mainframe Format Cap: 139.34 GB

300 GB FC Drive

Drive Speed: 15K

Open Systems Format Cap: 288.19 GB

Mainframe Format Cap: 279.77 GB

400 GB FC Drive

Drive Speed: 10K

Open Systems Format Cap: 393.84 GB

Mainframe Format Cap: 382.32 GB

450 GB FC Drive

Drive Speed: 15K

Open Systems Format Cap: 432.29 GB

Mainframe Format Cap: 419.64 GB

1000 GB SATA II Drive

Drive Speed: 7.2K

Open Systems Format Cap: 984.81 GB

Mainframe Format Cap: 956.02 GB

200 GB EFD

Drive Speed: Not Applicable

Open Systems Format Cap: 196.97 GB

Mainframe Format Cap: 191.21 GB

400 GB EFD

Drive Speed: Not Applicable

Open Systems Format Cap: 393.84 GB

Mainframe Format Cap: 382.33 GB

Support for 73GB and 146GB EFD’s have been dropped with the Symmetrix V-Max Systems, they will still be supported with the Symmetrix DMX-4 Systems which in addition to 73 GB and 146GB also supports 200GB and 400GB EFD’s.

© Devang for Gestalt IT, 2009. | EMC Symmetrix V-Max: Supported drive type
Read more posts categorized as Storage

The following size drives types are supported with Symmetrix DMX-4 Systems at the current microcode 5773: 73GB, 146GB, 200GB, 300GB, 400GB, 450GB, 500GB, 1000GB. Flavors of drives include 10K or 15K and interface varies 2GB or 4GB.
The drive has capabilities to auto negotiate to the backplane speed. If the drive LED is green the speed is 2GB, if its neon blue its 4GB interface.

To read a blog post on supported drive types on EMC Symmetrix V-Max System

The following are details on the drives for the Symmetrix DMX-4 Systems. You will find details around Drive Types, Rotational Speed, Interface, Device Cache, Access times, Raw Capacity, Open Systems Formatted Capacity and Mainframe Formatted Capacity.

73GB FC Drive

Drive Speed: 10K

Interface: 2GB / 4GB

Device Cache: 16MB

Access speed: 4.7 – 5.4 mS

Raw Capacity: 73.41 GB

Open Systems Formatted Cap: 68.30 GB

Mainframe Formatted Cap: 72.40 GB

73GB FC Drive

Drive Speed: 15K

Interface: 2GB / 4GB

Device Cache: 16MB

Access speed: 3.5 – 4.0 mS

Raw Capacity: 73.41 GB

Open Systems Formatted Cap: 68.30 GB

Mainframe Formatted Cap: 72.40 GB

146GB FC Drive

Drive Speed: 10K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 4.7 – 5.4 mS

Raw Capacity: 146.82 GB

Open Systems Formatted Cap: 136.62 GB

Mainframe Formatted Cap: 144.81 GB

146GB FC Drive

Drive Speed: 15K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 3.5 – 4.0 mS

Raw Capacity: 146.82 GB

Open Systems Formatted Cap: 136.62 GB

Mainframe Formatted Cap: 144.81 GB

300GB FC Drive

Drive Speed: 10K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 4.7 – 5.4 mS

Raw Capacity: 300.0 GB

Open Systems Formatted Cap: 279.17 GB

Mainframe Formatted Cap: 295.91 GB

300GB FC Drive

Drive Speed: 15K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 3.6 – 4.1 mS

Raw Capacity: 300.0 GB

Open Systems Formatted Cap: 279.17 GB

Mainframe Formatted Cap: 295.91 GB

400GB FC Drive

Drive Speed: 10K

Interface: 2GB / 4GB

Device Cache: 16MB

Access speed: 3.9 – 4.2 mS

Raw Capacity: 400.0 GB

Open Systems Formatted Cap: 372.23 GB

Mainframe Formatted Cap: 394.55 GB

450GB FC Drive

Drive Speed: 15K

Interface: 2GB / 4GB

Device Cache: 16MB

Access speed: 3.4 – 4.1 mS

Raw Capacity: 450.0 GB

Open Systems Formatted Cap: 418.76 GB

Mainframe Formatted Cap: 443.87 GB

500GB SATA II Drive

Drive Speed: 7.2K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 8.5 to 9.5 mS

Raw Capacity: 500.0 GB

Open Systems Formatted Cap: 465.29 GB

Mainframe Formatted Cap: 493.19 GB

1000GB SATA II Drive

Drive Speed: 7.2K

Interface: 2GB / 4GB

Device Cache: 32MB

Access speed: 8.2 – 9.2 mS

Raw Capacity: 1000.0 GB

Open Systems Formatted Cap: 930.78 GB

Mainframe Formatted Cap: 986.58 GB

73GB EFD

Drive Speed: Not Applicable

Interface: 2GB

Device Cache: Not Applicable

Access speed: 1mS

Raw Capacity: 73.0 GB

Open Systems Formatted Cap: 73.0 GB

Mainframe Formatted Cap: 73.0 GB

146GB EFD

Drive Speed: Not Applicable

Interface: 2GB

Device Cache: Not Applicable

Access speed: 1mS

Raw Capacity: 146.0 GB

Open Systems Formatted Cap: 146.0 GB

Mainframe Formatted Cap: 146.0 GB

200GB EFD

Drive Speed: Not Applicable

Interface: 2GB / 4GB

Device Cache: Not Applicable

Access speed: 1mS

Raw Capacity: 200 GB

Open Systems Formatted Cap: 196.97 GB

Mainframe Formatted Cap: 191.21 GB

400GB EFD

Drive Speed: Not Applicable

Interface: 2GB / 4GB

Device Cache: Not Applicable

Access speed: 1mS

Raw Capacity: 400.0 GB

Open Systems Formatted Cap: 393.84 GB

Mainframe Formatted Cap: 382.33 GB

Support for 73GB and 146GB EFD’s have been dropped with the Symmetrix V-Max Systems, they will still be supported with the Symmetrix DMX-4 Systems which in addition to 73 GB and 146GB also supports 200GB and 400GB EFD’s.

© Devang for Gestalt IT, 2009. | EMC Symmetrix DMX-4: Supported Drive Types
Read more posts categorized as Storage