EMC has been a market leader in bringing new innovative technology to the IT forefront. With the usage of RAID 6, EMC has again taken a very unique approach in designing this technology for its Symmetrix DMX products.
EMC has been a little late in adaption of RAID 6 for its products, not until recently did EMC introduce RAID 6 on its DMX-4 platform with 5773 version of Microcode. With RAID 6 and the large SATA drives, now the possibility of double failures in the same RAID group is considered a high probability and for that reason EMC has embraced the RAID 6 technology for all its mid-tier and enterprise level products…..Oh and also under a lot of pressure from competition and customers.
In this post we will uniquely talk about EMC’s modification of RAID 6 technology on EMC Symmetrix DMX products and how it redefined data protection on this platform.
In the next upcoming post, we might talk about RAID 6 as it relates to EMC Clariion and IBM Storage.
Here are links to some previous post related to RAID 6 technology.
EMC’s Business Case
RAID 6 is now available on EMC Symmetrix DMX products with microcode version 5773 and on EMC Clariion products with Flarecode release 26.
EMC Symmetrix DMX products are known to support RAID 1, RAID 10, RAID 1+0, RAID 5 (3+1), RAID 5 (7+1) and about 2 years ago introduced RAID 6 (6+2), RAID 6 (14+2).
With RAID 6 (6+2) technology, there are 6 data drives and 2 parity drives totaling 8 drives.
With RAID 6 (14+2) technology, there are 14 data drives and 2 parity drives totaling 16 drives.
RAID 5 has been common practice since the last 10 to 15 years for various storage and server based products. Back in the days, drive sizes varied from 4GB disk to 146GB SCSI or Fiber (which included various different sizes like 4.3GB, 9GB, 18GB, 36GB, 50GB, 73GB and 146GB). These days, seldom you see these size drives, customers are talking about disk sizes that are minimum 300GB (FC or SATA) and go up to 1TB. Over the next 2 to 3 years, we will absolutely see disk sizes that will be between 3TB to 4TB.
Here is an abstract about traditional RAID 6, again every manufacturer tends to change it a bit based on the products they release for performance and reliability.
Technology: Striping Data with Double Parity, Independent Data Disk with Double Parity
Performance: Medium
Overhead: 10% to 30% overhead, with additional drives you can bring down the overhead.
Minimum Number of Drives: 4
Data Loss: With one drive failure and two drive failures in the same Raid Group no data loss.
Advantages: RAID 6 is essentially an extension of RAID 5 which allows for additional fault tolerance by using a second independent distributed parity scheme (two-dimensional parity). Data is striped on a block level across a set of drives, just like in RAID 5, and a second set of parity is calculated and written across all the drives; RAID 6 provides for an extremely high data fault tolerance and can sustain multiple simultaneous drive failures which typically makes it a perfect solution for mission critical applications.
Disadvantages: Poor Write performance in addition to requiring N+2 drives to implement because of two-dimensional parity scheme.
We have in the past also discussed probability and failure rates (data loss situations) with RAID 5 and RAID 6. Please see the link below
To talk about some stats, the probability or the percentage of exposure related to RAID 5 double failures is as much as 7.5% while the chance of triple failure in a RAID 6 configuration is 0%. As the drive sizes are increasing, the usage of RAID 6 will become more prominent.
EMC’s Technology
RAID 6 as discussed earlier is a new technology introduced by EMC for Symmetrix DMX-4 products.
The actual definition of RAID 6 by EMC is “Any configuration that supports more than a single drive failure”.
Flash drives (EFD) also support RAID 6, only requirement is every drive in the RAID Group be a Flash drive (EFD). Also with RAID 6, permanent sparing is usable and incase of non availability of permanent sparing, dynamic spare pools are used for data reconstruction.
Default Track size on DMX-4 platform is 64K out of which each chuck of 4K is striped on each drive in the RAID Group.
As explained earlier, there are two supported versions of RAID 6 on EMC Symmetrix DMX platform.
RAID 6 (6D+2P) meaning 6 data drives and 2 parity drives. The overhead in this situation will be 25% [(2*100)/(6+2)].
RAID 6 (14D + 2P) meaning 14 data drives and 2 parity drives. The overhead in this case will be 12.5% [(2*100)/(14+2)].
We have discussed in the past blogs about how other OEM’s leverages RAID 6 on their storage platforms to make it faster and efficient. EMC’s version of RAID 6 is just very unique compared to any of the OEM’s I have discussed in the past.
HP’s version of RAID 6 is called RAID 6 ADG (Advanced Data Guarding)
Netapp’s version of RAID 6 is called RAID-DP (Raid Dual Parity)
HDS, Sun and EMC’s version are pretty much called RAID 6 but again the implementation is pretty unique (in terms of algorithms behind this technology implementation) for all manufacturers.
Since this process is pretty complicated and it will be very hard to explain without video or bunch of mathematical formulas or a white board, we will add couple of diagrams to make a user follow certain color schemes for understanding the parity calculation.
The Parity calculation for EMC Symmetrix DMX platform for RAID 6 is based on an Even-Odd Algorithm.
The first set of Parity is called HP (Horizontal Parity), for the rest of this document we will address this as HP.
The second calculated Parity is called DP (Diagonal Parity), for the rest of this document we will address it as DP.
Horizontal Parity (HP) is exactly similar to how RAID 5 parity is calculated. Later in the document we will discuss how the actual calculations happen.
Diagonal Parity (DP) parity is calculated based on diagonal dataset. DP is made up of segments of data; also each DP skips a different data drive while it is being calculated. The idea is with one lost drive, HP is used to reconstruct, while with 2 drive failures both HP and DP will be used to reconstruct failed drives.
So far with me…………….
EMC Symmetrix DMX RAID 6 utilizes the famous Even-Odd Algorithm for calculating parity.
We will talk about Prime numbers here (prime numbers are numbers that are not divisible by anything other than themselves to yield an integer).
Some prime numbers are 2, 3, 5, 7, 11, 13, 17, 19, …..
So for RAID 6 to work correctly, the number of drives we chose in the RAID group has to be a prime number (requirement of the Even-Odd algorithm).
With 6D + 2P we have 8 Drives in total
With 14D + 2P we have 16 drives in total
For consistency purposes, both the RAID Types above will need to have a set of drives that is a prime number; the closest number to 8 and 16 both is 17.
RAID 6D + 2P: 17 — 6D = 11 Null Drives.
RAID 14D + 2P: 17 — 14D = 3 Null Drives.
I know it’s getting too confusing…….think about the engineers that designed it, and think about everytime this is calculated for every set of data the customer generates and has to be written on RAID 6 disk.
All the null disk only have 0 as the data on it, in short the Null disk is also used to calculate the HP and DP, but in case one (Raid 6 6D+2P), all the data on 9 Null drives is 0 and in case 2 (RAID 6 14D+2P) all the data on 3 Null Drives is 0.
The Null drives do not physically consume any space, any drive, any memory, etc.
.GIF)
EMC Symmetrix DMX RAID 6 (6D+2P) implementation
D1, D2, D3, D4, D5, D6 are Data Drives
D7 is HP (Horizontal Parity Drive)
D8 is DP (Diagonal Parity Drive)
Drives that have a label “No Drive” are Null Drives with 0 data on them.
Diagonal in color RED is the center diagonal row and is used to calculate every DP in this raid group.
Each track is 64K with 4K stripes
HP = add all D1, D2, D3, D4, D5, D6, all null devices (in a row). HP does not include DP. Answer you get is 26, correct?
DP = add all the center diagonal row in RED plus the diagonal row below it (yellow) to come up with DP for row 1. Answer you get is 44, correct?
Similarly do the following to calculate diagonal parity 2: Add the diagonal red row and all the elements of diagonal row in color orange and you obtain the answer of 43, correct?
So far with me………………..
Again for simplicity purposes we managed to add these, in real life they are calculated based on Exclusive OR (XOR).
The HP will be calculated as
HP = D1 XOR D2 XOR D3 XOR D4 XOR D5 XOR D6 XOR Null devices
DP = Null Drives XOR D6 (12) XOR D5 (13) XOR D4 (14) XOR D3 (15) XOR D2 (16) XOR Null Drives XOR D6 (13) XOR D5 (14) XOR D4 (15) XOR D3 (16) XOR D1 (1)
The information listed in ( ) are the row numbers. Follow the color scheme things will be much easy. Also see above in the equation (highlighted in yellow) how we skip D2 in this case, the reason is you skip a drive in case of double fault, so we can rebuild from HP first and then from DP)
Since this calculation is pretty intense, we have only calculated the first two DP rows for you to compare the results.
.GIF)
EMC Symmetrix DMX RAID 6 (14D+2P) implementation
D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14 are Data Drives
D15 is HP (Horizontal Parity Drive)
D16 is DP (Diagonal Parity Drive)
Drives that have a label “No Drive” are Null Drives with 0 data on them.
Diagonal in color RED is the center diagonal row and is used to calculate every DP in this raid group.
Each track is 64K with 4K stripes
HP = add all D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14 and all null devices (in a row). HP does not include DP. Answer you get is 60, correct?
DP = add all the center diagonal row in RED plus the diagonal row below it (yellow) to come up with DP for row 1. Answer you get is 131, correct?
Similarly do the following to calculate diagonal parity 2: Add the diagonal red row and all the elements of diagonal row in color orange and you obtain the answer of 125, correct?
So far with me………….
Again for simplicity purposes we managed to add these, in real life they are calculated based on Exclusive OR (XOR).
The HP will be calculated as
HP = D1 XOR D2 XOR D3 XOR D4 XOR D5 XOR D6 XOR D7 XOR D8 XOR D9 XOR D10 XOR D11 XOR D12 XOR D13 XOR D14 XOR Null devices
DP = Null Drives XOR D14 (4) XOR D13 (5) XOR D12 (6) XOR D11 (7) XOR D10 (8) XOR D9 (9) XOR D8 (10) XOR D7 (11) XOR D6 (12) XOR D5 (13) XOR D4 (14) XOR D3 (15) XOR D2 (16) XOR Null Drives XOR D14 (5) XOR D13 (6) XOR D12 (7) XOR D11 (8) XOR D10 (9) XOR D9 (10) XOR D8 (11) XOR D7 (12) XOR D6 (13) XOR D5 (14) XOR D4 (15) XOR D3 (16) XOR D1 (1)
The information listed in ( ) are the row numbers. Follow the color scheme, things will be much easy. Also see above in the equation (highlighted in yellow) how we skip D2 in this case, the reason is you skip a drive incase of double fault, so we can rebuild from HP first and then from DP)
Since this calculation is pretty intense, we have only calculated the first two DP rows for you to compare the results.
Failure Scenarios
One Disk failure and recovery: Exactly similar to a rebuilt that happens with RAID 5, simple process.
Two Disk failure and recovery: Both Horizontal Parity and Diagonal Parity are used to rebuild data track by track.
More than two Disk failure and recovery: Possible data loss (chances of these are 0%)
Some Specific EMC Symmetrix DMX RAID 6 features
Uses Single mirror to show its status, failure on a device in the RAID Group is denoted by different colors like Yellow for 1 member failure and red for 2 member failure and purple for 3 member failure (data loss).
DAF (Disk Directors) are used to perform XOR operations — calculations with parity generation and rebuild.
Support for MetaLUN’s that are RAID 6
Support for BCV’s that are RAID 6
Support for Optimizer with RAID 6
Support for SRDF with RAID 6
Support for Snaps with RAID 6
Support for DRV and LOG devices with RAID 6
Support for Concurrent copy with RAID 6
Support for Permanent Sparing and Dynamic Spare Pools with RAID 6
Support for EFD’s with RAID 6 (all drives in the Raid group have to be similar)
There is no sort of benchmarking data that is available on RAID 6 performance (for EMC Symmetrix DMX) when we relate to RAID 6 (6D+2P) and RAID 6 (14D+2P) with regards to performance overheads, rebuild times with comparative analysis to NetApp or Hitachi’s RAID 6 implementation.
Again it’s pretty amazing to see, EMC’s claim with RAID 6 is not about performance, since performance can be achieved through RAID 1+0 configs, the idea is only reliability. For the Clariion platform the rebuild of RAID 6 devices can take 10% more time than a normal RAID 5 or RAID 1+0 device, the Clariion uses the same Even-Odd Algorithm.
With my previous Blog post on NetApp’s RAID-DP, HDS’s RAID 6, HP’s RAID 6 ADG, Sun StorageTek’s RAID 6 and this time around EMC Symmetrix DMX’s RAID 6, no one other than NetApp (98% performance efficiency) claims their version of RAID-6 as a performance enhancer. All the vendors are pretty much offering it as a standard Dual Parity technology for realibility and data protection.
Courteous Comments always welcome.
