It's not going to be this easy to bridge Fibre Channel and Ethernet!

Before the holidays, I posed a question on Google+ that generated quite a bit of interest and feedback. Now that it has settled down a bit I’d like to summarize the unresolved elements to make FCoE truly a world-class storage interconnect.

Setting the Stage

FCoE has been a controversial topic in both storage and networking, and for good reason. No one would deny that Ethernet is not an ideal transport mechanism for block storage I/O. “Porting” Fibre Channel to run on Ethernet networks has been a supreme technical challenge, and many companies and individuals have labored long and hard to make FCoE a reality.

Now that FCoE is specified in the standard and has been deployed in production environments, the question turns to its future. Will it take off and seize the mantle of dominance currently held by what I like retroactively to call “Fibre Channel over Fibre Channel?” Will they coexist for the next decade, with FCoE mainly deployed in “block” environments such as Cisco UCS? Or will FCoE ultimately fail to catch on, displaced by some other storage protocol like plain FC, iSCSI, NFS, or something entirely different?

The data center needs a flexible new protocol to meet the needs of virtual environments, and convergence of storage and data networking makes a great deal of sense in these environments. This was the root of my question, and I ask it in all earnestness.

My question: What elements remain unresolved to make FCoE truly world-class? What should the vendors be prioritizing? Here are the answers I received.

Technical Considerations

Link Aggregation on CNA’s

Converged network adapters (CNA’s) allow multiple protocols to access a single Ethernet connection, but some also include multiple ports that can be aggregated. In traditional Ethernet networks, link aggregation is a respectable approach for performance and availability. But storage networks have traditionally relied on host-based MPIO software, and these features are mutually exclusive. The zeitgeist seems to be a recommendation to avoid link aggregation on CNA’s that are used for storage networks.

How Do You Handle Virtual Machine Mobility?

As I described recently, virtual machine mobility is a major technical challenge for existing networks. The VMware proposal, the VXLAN, seems to be gaining traction right now. But this is only a solution for data networking. How will FCoE SANs handle virtual machine mobility? This remains unresolved as far as I can tell, though Ethernet switch vendors have come up with their own answers. Brocade demonstrated just such a solution at Networking Field Day 2, and I know that others have answers as well. But will there be an interoperable industry solution?

How Should FCoE Be Implemented Over Longer Distances?

Fibre Channel has traditionally relied on routers and other protocols (FCIP and iFCP) to span distances, but FCoE raises the possibility of native traversal. While it is certainly possible to span distances with FCoE, this is definitely not a recommended or supported idea. Without TCP/IP, or any routing mechanism, it’s just a bad idea. But I imagine that it won’t be long before vendors decide to give it a go anyway.

Implementation Considerations

Is TRILL Required for FCoE Networks?

This has been one of my own questions since the very beginning. Clearly, edge only FCoE works just fine without TRILL. But as networks become more complicated, and virtual machines move, it seems an awfully good idea to have some protocol to alleviate East-West routing concerns. I feel much better with TRILL (or some similar Ethernet fabric technology) in a complicated FCoE network.

Should All Switches Be Full FC Forwarders?

There are number of ways to implement FCoE on Ethernet network, and not all involve building a full Fibre Channel stack in each switch. While many (including myself) assumed that FCoE implied Fibre Channel forwarding in all switches, this is clearly not the direction taken by vendors, at least initially. Perhaps the current “Ethernet forwarding” approach is only a stepping stone, or perhaps it will emerge as the dominant FCoE standard.

How Will OpenFCoE and LoM Be Used?

OpenFCoE is a software solution allowing FCoE to be run without a CNA. If this became popular, it wouldn’t be long before data center architects began looking at LAN on Motherboard (LoM) and even 10GBASE-T as a potential SAN alternative. Will this be used in the long run? It could happen, but it’s certainly not something that’s here at the moment. But OpenFCoE is a real player, especially with Intel’s backing.

How Will Technologies like Zoning Interoperate?

Many networkers are just now beginning to see the true complexity of Fibre Channel SANs. Although interoperability of higher-level Fibre Channel functions between vendors has never been a priority in “FC over FC” SANs, Ethernet could change things. I would not be at all surprised to see a groundswell of customer support demanding greater levels of interoperability from FCoE than from FC, and zoning and VSAN is the likely first beachhead.

The Big Question: When Will We See the “Killer App” For FCoE

Just about everyone agreed that the real challenge for FCoE is market acceptance. Customers aren’t yet demanding FCoE, and vendors are finding it hard to articulate a compelling case to move from “tried-and-true” FC. Convergence, cost savings, and performance have all been put forth, but customers aren’t biting. Perhaps they just need a little time and a little more proof.

This post relies extensively on feedback from a number of people, including Ivan Pepelnjak, Tony Bourke, J Metz, Dmitri Kalintsev, Derick Winkworth, David Hardaker, Juan Lage, and Corey Hines.

© sfoskett for Stephen Foskett, Pack Rat, 2012. |
Eight Unresolved Questions About FCoE


This post was categorized as Enterprise storage, Everything, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2012. | Eight Unresolved Questions About FCoE
Read more posts categorized as Storage

A Compelling Product

At its core, Compellent is a storage software company. Their Storage Center software creates a “virtual storage array” from standard building blocks. The resulting storage product is one of the most flexible and (excuse the pun) compelling midrange storage systems on the market. Consider the following roundtable discussion from our recent Tech Field Day event in Seattle, which includes a discussion of Compellent’s products and technologies by independent voices in the industry:

Podcast: Play in new window | Download (Duration: 8:59 — 102.7MB)

Compellent’s Storage Center scales up, adding additional storage capacity to existing arrays without disruption and integrating this added bandwidth and capacity seamlessly. Compellent also has an impressive tiered storage and thin provisioning story, since data blocks are virtualized and can be moved between tiers dynamically. Full-featured integrated snapshot technology completes the picture, adding data protection credentials.

Compellent’s scale-out story is impressive as well, but is more about replication and virtualization than the clustering technology used by others, including Dell’s EqualLogic. Compellent’s newly-introduced Live Volume can be thought of as a volume manager for virtual servers, allowing their storage to exist simultaneously on two different storage arrays. Supporting VMware vSphere, Microsoft Hyper-V, and Citrix XenServer, Compellent claims that Live Volume enables a “grid of Compellent arrays”, but it’s not the sort of scale-out grid one would get from EMC’s recently purchased Isilon, HP’s LeftHand, Dell’s Compellent, or NAS specialist Panasas. It’s more of an active-active mirror of storage than a true cluster. Learn more about Live Volume in the following video by Rodos:

Let’s also take a moment to consider the hardware Compellent currently uses: They deploy and sell this software on SuperMicro servers with Xyratex disk enclosures, and Seagate hard disk drives, and connect everything with QLogic, Emulex, and Brocade storage networking gear. This is all pretty much standard-issue for storage vendors: Most no longer produce their own controllers or disk enclosures, and the standard components Compellent uses can be swapped from other sources if needed.

Compellent’s Mid-Market Strategy

Compellent has not traditionally competed in the stratosphere of enterprise storage dominated by EMC’s Symmetrix V-Max, the HDS/HP USP/VSP, and (to a lesser extend) the HP 3PAR InServ. Instead, the Minneapolis-based company has been happy to sell into the fat middle of the market, and has gained a loyal following in smaller-but-still-impressive enterprises. Nowhere is this more evident than at Compellent’s annual “C-Drive” customer conference: It’s amazing, really, for a smallish storage company to have such a devoted user base!

Although they offer two NAS front-ends, Compellent is a block storage player through and through. Their “Storage Center with NAS” offering is decidedly low-end, employing Windows Storage Server 2008 on a 1U server for SMB and NFS connectivity. The newer and more-impressive zNAS is based on the OpenSolaris operating system, including ZFS a friendly interface “co-developed” with Nexenta. Neither product is an in-house Compellent design, nor are they competitive in the high-end “big data” market touted by NAS leaders EMC, NetApp, and BlueArc.

Compellent’s Fit Within Dell

Assuming the acquisition goes through, attention will turn to the fit for Compellent’s Storage Center technology and marketing within Dell. What will Dell get for three quarters of a billion dollars? What will it mean to the enterprise storage market as a whole?

Dell’s EqualLogic acquisition, though expensive, is now widely seen as a success. The Round Rock management team allowed EqualLogic the freedom to develop and compete, while bolstering their hardware and manufacturing story with commodity components. The combination has become a dominant plater in the mid-market iSCSI storage space, and Dell is likely to attempt a repeat of that strategy here. It was widely rumored that 3PAR management preferred Dell’s to HP’s offering based on a promise of similar autonomy.

We assume Dell would replace the EMC-sourced CLARiiON with Compellent in their midrange storage arsenal, replacing the SuperMicro servers. Although Compellent would likely be allowed to develop and grow within the Dell umbrella, there is substantial overlap with the existing EqualLogic line. One would expect Compellent to focus more on Fibre Channel and FCoE (which they recently delivered), though iSCSI support is unlikely to be dropped. A longer-term strategy would see EqualLogic and Compellent combining their midrange storage IP and management team within Dell.

Considering their success selling CLARiiON block storage, Dell’s acquisition would strap a rocket to Compellent’s sales. Once the Dell hardware is grafted in, profits will grow substantially as well. This is a major win for Dell, which needs the kind of margins that only in-house IP can drive. The price is right, the market is ready, and the fit is solid.

What Comes Next

Some are suggesting that this is the end of the storage acquisition parade, but many holes remain. Dell still lacks enterprise NAS, data deduplication, and cloud storage success, and companies like Oracle have only just begun to bulk up their storage strategies. Dell’s acquisitions of Ocarina and Exanet have yet to pay dividends, likely driven by the immaturity of those products. One would not be at all surprised if they continued shopping in the storage space: Panasas, BlueArc, Sepaton, FalconStor, and Overland all look ripe for purchase. Although Compellent is a win for Dell, it is not likely to be their last acquisition.

Disclosure: Compellent sponsored and presented at Gestalt IT’s Tech Field Day 3.

© Stephen Foskett for Gestalt IT, 2010. | Thoughts On A Dell Acquisition Of Compellent
Read more posts categorized as Exclusive, Featured, Storage, Top Story

“Convergence” is a buzzword seen in the IT press constantly these days.  All convergence means is placing communications that used to ride on its own network onto one unified network; Ethernet’s cheapness, ubiquity, and ever-growing link speeds makes it the network everything is moving towards.  The first big convergence move was to combine voice networks with data networks, using IP telephony.  The challenges of a converged voice/data network include prioritizing voice traffic over pretty much anything else during times of link congestion, and keeping call quality high by delivering datagrams in a predictable time with a predictable gap in between those datagrams.

Frankly, these problems were and are a big pain in the collective backsides of network engineers everywhere.  Ethernet and IP are not transports intended to deliver traffic in a predictable, prioritized fashion.  Ethernet is a best-effort frame delivery system that’s only survived as long as it has by reducing collision domains down to one via switches.  IP uses higher-level protocols for reliability and underlying devices for prioritization.  Therefore, delivering VoIP frames and packets across a converged infrastructure means a carefully designed and deployed quality of service plan; the larger and more complex the network environment, the greater the potential pain.  Throw in congestion-prone wide-area links of varying transports with their own forwarding nuances (ATM is not frame-relay is not MPLS is not PPP), and the network engineer must know a lot about a lot to deliver an effective end-to-end QoS design.  Even worse, no one hears the engineer scream who is configuring QoS on multiple platforms, all of which might require unique configurations depending on hardware and software to net the same results, even within the same vendor product families.

Convergence Redux

Convergence has evolved with the increasing affordability and adoption of 10-gigabit Ethernet.  With the bandwidth and low-latency of 10G, the industry has pushed towards adding storage to the converged data center Ethernet, the idea being to eventually eliminate the unique Storage Area Network.  Fibre channel is the protocol of major concern here, in that other storage protocols like iSCSI are more tolerant of changing network throughput characteristics.  FC is not tolerant of a changing environment.  FC expects that frames will be delivered on-time, every time, and therein lies a significant challenge for the converged data center Ethernet.  While 10G is an awful lot of bandwidth, simply adding Even More Bandwidth (the tried-and-true method of capacity management for engineers who don’t want to think too hard) isn’t a safe answer.  No matter how much bandwidth is available, the Ethernet carrying Fibre Channel traffic (FCoE) must be able to guarantee a lossless path from host to disk and back.

Don’t Drop The Baby!

Enter Data Center Bridging, or DCB.  DCB comprises a set of proposed standards designed to extend Ethernet such that we can:

• Leverage flow control on up to 8 virtual links. (IEEE 802.1Qbb – Priority-based Flow Control) We can issue ethernet PAUSE frames on specific virtual links, and not interrupt forwarding for the entire physical link.  Practically speaking, we could tell everyone but the storage virtual link to shut up for a moment.
• Prioritize traffic classes within a virtual link. (IEEE 802.1Qaz – Enhanced Transmission Selection)  Here, we can use QoS techniques within a virtual link to prioritize certain kinds of traffic.  Voice – you rule!  Network engineer’s web surfing – go to the head of the line!  Bittorrent from Joe down the hall – tail drop.
• Exchange DCB information with other DCB devices. (also part of IEEE 802.1Qaz – DCBX, and expected to leverage LLDP)  Two devices can learn about each other’s DCB-related link characteristics.
• Optionally notify upstream senders of downstream congestion, allowing the sender to mitigate the congestion through rate-shaping.  (IEEE 802.1Qau – Congestion Notification)  In a backwards way, this reminds me of RSVP, where you can do an end-to-end bandwidth reservation across multiple network devices.  I’ll be interested to see how this is implemented.

All of this gives us the ability to guarantee that storage traffic has both the forwarding capacity and priority over other traffic flows when needed.  Done right, an FCoE frame should never hit the floor.  That said, I am interested to see how much hands-on engineering will be required to make this work as intended.  Legacy QoS is far from automatic in the real world, and it seems fair to assume that a well-designed DCB implementation will require rather a lot of whiteboarding, implementing, monitoring, and tweaking before it perfectly serves the environment it has been deployed in.

Hey, Pal – Ya Wanna Buy An Enhanced Ethernet?

DCB has a couple of implementations:  Data Center Ethernet (DCE) is Cisco’s flavor, and includes an implementation of TRILL.  Cisco markets DCE and related advanced data center technologies under the term “FabricPath“, which is baked into the Nexus 7000 product line.  In some Cisco documentation, they refer to DCE as a superset of DCB and another implementation of DCB, Converged Enhanced Ethernet (CEE).  CEE has a much broader group of networking companies behind it, including Broadcom, Brocade, Cisco, Emulex, HP, IBM, Juniper, and QLogic, and is helping to craft how the DCB standards will finally look when ratified.

The point of DCB then is to provide an interoperable set of standards whereby storage traffic can be guaranteed the bandwidth and flow characteristics it requires across an Ethernet transport, while co-existing with other traffic behaving quite differently.

The problem?  At this point, the jury is out on what form of converged storage the market will claim.  FCoE is seeing increased adoption, but uptake has been slow.  iSCSI has been around for a while, is generally well-understood by engineers who deploy it, and a popular choice.  Vendors are pushing various and usually proprietary fabric schemes.  Cisco, HP, Brocade, and IBM are all making acquisitions and creating product lines to sell you a single-vendor solution for storage, servers, and a converged network to run it on.

One thing’s for certain:  it’s a fun time to be a network engineer.

© ethan for Gestalt IT, 2010. | Don’t Drop The Baby: Data Center Bridging Wants Storage To Trust Ethernet
Read more posts categorized as All, Featured, Networking, Storage

Silentium AcoustiRACK

Israel-based Silentium brought along a server rack for small offices that was music to our ears. It incorporates a sound (if you pardon the pun) bottom-to-top and front-to-back airflow system, conventional sound-deadening baffles, and active noise cancellation to create a quiet sealed server rack. While this might not appeal to everyone, those of use who have to sit next to rack-mount servers, network gear, and storage will certainly appreciate it.

The active noise cancellation was very effective. It uses a microphone, CPU board, and speaker to inject out-of-phase sound waves to cancel out the sound of fans, servers, and anything else you might put in the rack. As you can hear in the video below, the effect was audible even on a noisy trade show floor.

QLogic

As we experiment with various SAN configurations, it becomes obvious that the industry needs a supplier of solid, inexpensive Fibre Channel switches that can grow as needs changed. That’s why the 12-port QLogic-sourced HP SN6000 switch was a nice find.

Clearly based on the QLogic 5600 product, it has a low price of entry but is easily expandable with a license unlocking four more ports, and also includes four dedicated 10 Gb FC ports for stacking multiple switches. It may not be as sexy as a Brocade 8000 or Cisco Nexus 5000, but small and cheap is welcome in the big-money world of Fibre Channel SANs.

© Stephen Foskett for Gestalt IT, 2010. | More From HP Tech Forum: Silentium and QLogic
Read more posts categorized as All, Exclusive, Featured, Server Virtualization, Storage

Do you know me?

Visitors to the Fibre Channel Industry Association (FCIA) booth at Storage Networking World in Orlando were greeted by a strange symbol (pictured above). It was included in the FCIA  Fibre Channel roadmap presentation as well. The group even made identifying the symbol a requirement to enter their door prize drawing.

But what is the symbol exactly? I was amused to get some puzzled looks (and no correct answers) when I polled a number of industry insiders. One even asked about the symbol when I included it in a blog post about Microsoft’s lack of FCoE support.

I guess everything needs a symbol nowadays, and Fibre Channel over Ethernet (FCoE) is no exception. Yes, this is the FCIA’s new FCoE symbol.

Fibre Channel plus Ethernet equals FCoE

Breaking it down, we might recognize the unofficial but widespread symbol of Ethernet (glance at the back of your PC or the box your switch or router came in) in the background with a double-ended single-bladed arrow superimposed. That arrow is apparently the new symbol for non-Ethernet Fibre Channel (FC).

Aren’t you glad you asked?

© Stephen Foskett for Gestalt IT, 2010. | FCoE Symbolism
Read more posts categorized as Storage

Brocade paid $3.4 billion for Foundry and rushed the acquisition using a lot of cash to close the deal quickly. HP just completed the 3Com purchase for $2.7 billion. To my mind, the 3Com assets are a lot more valuable since they also have firewalls, IDS and a wide range of networking products. Foundry only has Core Ethernet switching products and no access ethernet, no service provider, no wireless.

Of course, Brocade only makes high end FibreChannel switches which are high performance, high complexity and highly proprietary so Foundry makes a nice fit to nice little corner of the market for a tidy profit. But when Cisco announced FCoE and then threw a billion dollars behind it, Brocade’s core products around FibreChannel became obsolete. Oh, I know the FibreChannel will limp along for a few more years, but there is no five year plan for growth anymore. That’s over. Ethernet is the future.

And the Data Centre has become a war zone between HP, IBM, Cisco and now HDS. Where does Brocade fit in?

Foundry Command Line is a mess

I have also worked on the Foundry Ethernet equipment and find it a confusing mess of interface, syntax and technology differences. It needs a lot of support to keep that equipment running. As a result, it’s hard to like and use the Foundry / Brocade ethernet story. I recently worked on a site that has ServerIron, FastIron and BigIron switches. Each one has a slight, but significant difference in CLI that makes them frustrating to work with (and appreciate how Cisco’s IOS interface is reasonably consistent). I had to constantly refer to the manuals to validate and cross reference every command. That’s a but annoying.

The EtherealMind View

Mistake

So I reckon Brocade made a big mistake buying Foundry at all, and certainly overpaid for what they did get. Brocade’s share price has dropped a long way, and there has been VERY little information about what they are doing with their Ethernet products. They don’t share anything about future plans or get involved with customers from what I can see.

Where’s the plan?

I’m not a Brocade customer, and unlikely to ever consider them until I know more about what they are doing. And that doesn’t mean meeting with a sales team or marketing team. I should be able to clearly perceive the future from their public presence. I don’t want closed briefings or NDA roadmaps. Just show me a coherent plan, on your website, and then I can consider you as a serious supplier.

Or is it just me? Can you see a future for Brocade beyond five years?

© Etherealmind for Gestalt IT, 2010. | Brocade – What’s Their Direction?
Read more posts categorized as Networking

How fast can iSCSI get?

“Maximizing Hyper-V iSCSI Performance with Microsoft and Intel” might sound like another “blah blah” marketing piece, but a little birdy tells me that this webcast will drop a bombshell about iSCSI performance.

Lots of storage and networking folks don’t give iSCSI and Microsoft the credit they deserve. “iSCSI is cheap and easy,” they say, “but real performance requires Fibre Channel.” Those of us who have an open mind about such things know that this is simply not the case. The fastest SAN I ever saw was based on iSCSI, and Microsoft demonstrated wire-speed iSCSI over 10 Gb Ethernet in March. I never saw a Fibre Channel SAN (even an 8 Gb/s one) push over a gigabyte per second over a single link!

Still, ask the average sysadmin and they will tell you that iSCSI isn’t for high performance applications. That’s why folks should tune in to this webcast, as Microsoft and Intel knock down another iSCSI performance myth. Note that even though Hyper-V is called out in the title and description, this discussion is really about Windows Server 2008 R2 and applies equally regardless of whether or not you use Microsoft’s hypervisor.

Watch this space for a summary of the news immediately following the announcement.

• What: Maximizing Hyper-V iSCSI Performance with Microsoft and Intel webcast
• When: Thursday, January 14, 2010 8:00 AM Pacific Time
• Where: MSEvents.Microsoft.com
• Who: Anyone interested in high-performance storage and server I/O

© sfoskett for Stephen Foskett, Pack Rat, 2010. |
Microsoft and Intel Pushing iSCSI Performance Limits


This post was categorized as Enterprise storage, Gestalt IT, Virtual Storage. Each of my categories has its own feed if you’d like to filter out or focus on posts like this.

© Stephen Foskett for Gestalt IT, 2010. | Microsoft and Intel Pushing iSCSI Performance Limits
Read more posts categorized as Server Virtualization, Storage

So let’s say a customer decided to purchase a new V-Max system from EMC today. What would they get?

• They would get a freshly-designed cabinet, controllers, and firmware with more cache RAM, faster CPUs, larger disk drives, and support for all of their existing Symmetrix DMX features. Beta customers actually had both components late last year, and customer orders are proceeding. So it is safe to say that the V-Max clustered array itself is currently shipping. Note that the maximum configuration is 8 engines, each with a quad-core CPU, 128 GB of cache RAM, and 16 each of host and drive channels.
• Like the DMX before it, the V-Max supports a variety of disk drive types, from flash to Fibre Channel to Serial ATA. The larger so-called enterprise flash drives (EFDs) also appear to be shipping this quarter.
• Automatic provisioning with templates and management is available now as well.
• Virtual provisioning (thin provisioning and non-disruptive relocation) are enabled and shipping, but they’re mutually exclusive (see more below).
• PowerPath/VE and vSphere Storage plug-ins are shipping, even though VMware vSphere 4 was just released.

So our V-Max customer would be able to buy and deploy a solid, scalable, next-generation enterprise storage array that does everything they expect from existing systems and a little more. This is more than many new or re-engineered storage products can boast.

But our customer does not get everything that has been talked about regarding the V-Max architecture. What don’t they get?

• Future protocols like 10 GbE iSCSI and FCoE and 8 Gb Fibre Channel are not shipping yet. No surprise there.
• One of the most important components of V-Max is its fully-automated storage tiering, called FAST by EMC, which will not be fully realized for years. This technology is a major departure for the Symmetrix, which has long had the capability to manually move volumes but lacked the sub-LUN granularity needed to truly take advantage of the newest storage technologies like flash SSD. Although EMC plans to release “FAST v1″ during the second half of this year, this functionality merely automates the existing LUN relocation capability of all current Symmetrix arrays (for non-thin-provisioned LUNs) and is nowhere near the promise of FAST. Full FAST will not ship until 2010, and we suspect it might be well into that year.
• True Virtual Matrix mode, where multiple V-Max array “clusters” link together virtually to form a system larger than 8 engines. EMC has also talked about extending this mode over distances, creating a geographically dispersed array. None of this is coming any time soon.

So the answer to the core question of whether the EMC Symmetrix V-Max is shipping is yes. Customers can go out and buy one. But they will not get all of the functionality discussed and presented by EMC and its representatives last month.

Not So FAST

Fully-Automated Storage Tiering (FAST) is one of the most significant capabilities of the V-Max platform. As described, FAST will not be available for a year or more. This is not a secret or a scoop – EMC is the first to admit that FAST was pre-announced and is “Coming Next”!

V-Max's FAST technology "automates movement and placement of data based on changing needs" (illustration courtesy of EMC)

What is FAST? It is an automated tiered storage technology that can place data on different tiers of storage based on requirements. High-performance data on flash, bulk data on SATA, and so forth. EMC V-Max marketing materials, and the launch content, spent much time focusing on this technology, telling us that it automatically adjusts tiering of data, not just volumes, and reacts as needs change, rather than statically placing data. We’ve reported some confusion about just how FAST accomplishes this in the past, but we all agree that this would be an awfully nice way for an array to manage data.

At the core of FAST is EMC’s recently-introduced virtual provisioning technology. Why is FAST so much better than VP, and other competing automated storage tiering technologies? According to EMC blogger, Barry Burke, FAST has five key benefits:

1. Change RAID protection types as LUNs and volumes are relocated
2. Relocate a large number of LUNs and volumes concurrently
3. LUN and volume relocation is very quick
4. LUN and volume relocation has low performance impact on the array, and on the applications using the data being moved
5. LUN and volume relocation is done without disrupting replication

Let’s give EMC some credit for its FAST vision, as well as the pieces that have already been shipped. But we’re still quite a way off from seeing this FAST vision realized in production technology:

1. Current DMX and V-Max hardware allows customers to use multiple drive types, including flash SSD EFD, for over a year. Many other vendors are just now getting around to shipping flash disks. No one would claim that this is “FAST”.
2. Current DMX and V-Max also allows thin provisioning (a component of EMC’s Virtual Provisioning). This isn’t FAST either.
3. Current V-Max software allows customers to non-disruptively relocate volumes between disk and RAID types (another component of Virtual Provisioning). This is very nice to have. But this is not fully automated storage tiering. It’s just easier storage tiering. And it doesn’t apply to thinly-provisioned volumes.
4. FAST V1, which EMC plans to ship in the second half of 2009, will automate the selection and migration of volumes to different disk and RAID types. Although this is technically fully automated storage tiering, it does not quite live up to the promise of FAST as described by EMC at the V-Max launch. And it still won’t support thin (Virtual Provisioned) volumes!
5. Finally, EMC claims that they will fully realize the FAST vision with FAST V2, which will ship in 2010. This would enable granular (sub-volume) tiered storage for thinly-provisioned volumes.

Why does it matter that EMC won’t ship FAST V2 until next year? Simply put, anything less than FAST V2 will still not effectively utilize the expensive EFD capacity paid for by EMC customers. Even FAST V1 will take up expensive high-performance flash capacity with the entire contents of a LUN, including empty space! Customers wishing to conserve EFD capacity will use virtual provisioning to only store “used” data on flash. But these cannot be easily relocated if demand does not meet expectations, and not all of that used data requires high-speed flash storage. EMC also lacks thick-to-thin (or thin-to-thick) technology, though we suspect they’re working on that.

So customers have a choice: They can either use their EFDs more efficiently with thin volumes, or move data between EFDs and disks as needs change. They cannot have both.

An open question is how much EMC will charge for FAST V1 and V2. Assuming they are like Apple with the iPod Touch, and their history affirms this, EMC will charge for FAST as each major new capability is delivered. EMC has always charged for major new features. This is due to accounting rules which prohibit a company from charging customers ahead of releasing major features. The only way around this would be for EMC to recognize V-Max revenue as recurring over time, as in the case of the iPhone at Apple. Fat chance of that! Note that EMC is also talking about bringing FAST to CLARiiON and Celerra, but those timetables are not disclosed.

How Big Is Your Virtual Matrix?

Many have wondered about EMC’s choice of a name for this new storage system, some speculating that since the “V” stands for “virtual”, the V-Max was designed for server virtualization environments. EMC CEO Joe Tucci’s comments at the V-Max launch seemed to indicate this as well.

However, while it certainly is targeted at VMware vSphere and similar systems, the “V-Max” name actually has a very specific, and very different, meaning. “V-Max” is actually a reference to EMC’s new “Virtual Matrix” architecture, which allows V-Max engines to combine into a scale-out array with a virtual matrix of interconnects. Like FAST, the Virtual Matrix Architecture is the real vision of revolutionary storage from EMC. And like FAST, this vision is nowhere near being realized.

V-Max is so far being sold as a stand alone machine, not a scale-out cluster. At current GA, EMC is only offering 8 engines that are part of the same system cabinet (physically connected through MIBE and SIB), but there is nothing available that would connect 2 of these system cabinets, where instead of 8 engines you are now running 16 or 32 engines, forming a real V-Max. 

The V-Max name refers to the Virtual Matrix architecture of the array cluster (illustration courtesy of EMC)

EMC’s “vision” clearly extends well beyond 8 engines and a single V-Max.  EMC’s design allows for a federated set of V-Max machines that can all be managed as a single entity, and the company alluded to this in their product introduction.  They suggested that these separate V-Max machines wouldn’t even have to be in the same place, physically.  

Although the vision articulated at the product’s introduction talked about massive scaling in multiple locations, EMC’s maximum current configuration for the V-Max is 8 engines with up to 128 Xeon CPU cores, 1 TB of RAM, and 128 each front-end and back-end storage interfaces linked to 2,400 disk drives. This is no tiny system, to be sure, easily doubling the current DMX-4 array’s scalability as well as besting the competition. In fact, EMC can rightly claim that a V-Max can be configured today to be the largest integrated enterprise storage array available.

But 8 engines linked by local RapidIO connections in a single data center isn’t a revolution. The revolution happens when EMC realizes its vision of linking “dozens” of engines, thousands of ports, and tens of thousands of drives into a geographically dispersed system. Former EMC executive, Dave Donatelli, now controversially linked to HP, claimed that the V-Max can support up to 256 engines in a product launch video that has now been removed. IDC analyst, Benjamin Woo, talked about EMC’s ability to support a “single namespace” across geographies during his launch coverage. EMC’s Chuck Hollis says that “the overall architecture can eventually grow into literally thousands of ports, thousands of processing cores, tens of thousands of disks, and many terabytes of cache memory — all operating as a single, tightly-coupled array.” EMC’s Barry Burke also suggests that future revisions to the Virtual Matrix architecture could scale well beyond anything hinted here.

Again, in that same post, EMC’s Hollis tells us that the architecture delivers four key benefits in terms of scalability (bulleted for your pleasure):

1. “The difference between the two?  The V-Max can grow and grow from that modest starting point — still capturing the economics of mid-tier arrays and linear cost scaling.
2. Second, the V-Max array isn’t limited to a single cabinet with short wires.  Think in terms of multiple cabinets, separated by many meters or — in the future — longer distances using optical connections – all behaving as a single, giant array.
3. Third, the scaling is utterly linear and modular.  As more building blocks are added, the array gets bigger and bigger, and there’s no need to replace the frame, buy a bigger cabinet, etc. — nothing gets thrown away.
4. Fourth, there’s no assumption that all the building blocks are exactly the same.  Some may be big, some may be small, some may be old, some may be new.”

So a reader would look at this and think to himself, “this new V-Max starts as cheap as a competing midrange array and can scale out with different building blocks to span my entire data center, or even my entire enterprise.” This is a great promise, and awfully near to the holy grail of enterprise storage.

The problem is that none of this is possible today.

1. The V-Max starts at “about $250,000″ for a single engine. This is much, much more expensive than competing midrange arrays. It has to be, since a single V-Max engine has a lot more horsepower and componentry than any midrange array!
2. All V-Max connectivity is currently done with RapidIO connections in “a single cabinet with short wires.” Perhaps you could define “cabinet” differently and claim that the V-Max uses multiples, but no one (especially an EMCer who wants to keep his job) would suggest that the V-Max engines can be “separated by many meters or … longer distances.” When Chuck says this is “in the future” he means it!
3. V-Max scales to 8 engines, and initial reports are that scaling is indeed fairly linear. So this is a fair statement. But what if EMC replaces RapidIO with InfiniBand or datacenter Ethernet? Would there really be no need to “throw away” components?
4. There are no “big”, “small”, or “old” V-Max building blocks. They’re all new, and they’re all identical, and you can have up to eight of them. EMC says you will be able to mix and match once new building blocks come out, and we expect them to live up to this promise.

So even today’s V-Max system does use the Virtual Matrix architecture. It’s got a virtualized matrix interconnecting the engines, after all. But it’s nowhere near the scale-out promise of the Virtual Matrix concept. That’s quite a ways off still.

Conclusion: V-Max Is Not Overtaking The Future Yet

It is important to note that EMC sales literature does not claim capability beyond what’s currently on the truck. The V-Max data sheet, for example, makes no mention of FAST or scalability beyond 8 engines. And EMC’s Dave Graham is characteristically straightforward listing what’s in the box. In fact, EMC’s official web site and literature probably errs on the too-cautious side: A casual visitor might well ask himself, “where’s the beef?”

Why might an average storage buyer think there is more “beef” to be had? One only needs to watch EMC’s V-Max launch, and the surrounding coverage. Indeed, EMC’s own theme for the launch was “Overtake the Future!” EMC’s customers expected a revolution, and a revolution is what the company promised.  This isn’t anything new, unexpected, or even out of place. Companies often tout the theoretical future capabilities of their new products but don’t deliver on those promises for some time. Many have accused Microsoft of using this tactic to combat Windows competitors, for example. And let’s not forget the industry’s fascination with legendary vaporware products!

But the company’s boosters seem to want to have their cake and eat it, too. They’ve poured on the verbiage describing FAST, and used this future capability to declare victory in the battle against the likes of HDS, 3PAR, and Compellent. EMC’s Hollis tells us that “there’s nothing remotely like V-Max in the marketplace today”, and other EMC folks have made similar claims. As Wikibon points out, “EMC’s pre-announcement of Fully Automated Storage Tiering (FAST) is an admission that without FAST the answer is essentially no, EMC can’t compete on cost.” In other words, EMC was forced to prematurely unveil their FAST vision to keep their customer base from jumping ship to competitors already offering similar functionality. The same is true of the scale-out capability of the Virtual Matrix architecture.

For now, EMC has merely introduced a solid next-generation enterprise storage array. V-Max without FAST and scale-out is a nice-to-have refresh of the Symmetrix DMX-4. EMC should be especially respected for leaving all of the core Engenuity features like TimeFinder and SRDF intact, since this must have been no small feat. V-Max with FAST V1 is a more attractive proposition, but does not go that far beyond the capabilities of Symmetrix Optimizer. The same can be said of the 8-engine V-Max limit – it’s larger, but nothing to get too excited about.

V-Max with full-on FAST V2 and multi-site heterogeneous scale-out is the real revolution in EMC storage. If everything plays out, you could have several V-Max systems in the same data center, or perhaps even different data centers, all being managed as a single entity.  Then layer FAST on top of that (at the block level), and you have something unlike pretty much anything else on the market today. The question is, when can they achieve this, and will some of the other vendors catch up in the meantime?

This is what observers, analysts, and customers were looking for. And this is what they might think they saw. But in reality, we’ll all just have to wait another year or two and see what develops.

© Gestalt for Gestalt IT, 2009. | EMC Symmetrix V-Max: When Does It Get FAST and Virtual?
Read more posts categorized as Exclusive, Featured, Storage, Top Story

It seems like anytime someone talks about FCoE, they end up also talking about unified fabric. After having read a number of different articles and posts regarding FCoE, I can see where FCoE would be attractive to shops with significant FCP installations. In my mind, though, this doesn’t necessarily mean unified fabric. Given the political differences in organizations—think the “storage team” and the “networking team”—how likely is it that an organization may adopt FCoE, but not unified fabric? Or how likely is it that an organization may adopt FCoE, intending it to be a transitional technology leading to unified fabric, but never actually make it all the way?

So here’s my question: is unified fabric an inevitability?

And here’s a related question: Most people cite VoIP as proof that the unified fabric is inevitable. More so than anything else, I believe VoIP’s success was more a reflection of the rising importance of TCP/IP networking. If so, does that give iSCSI an edge over FCoE? Is iSCSI the “VoIP of the storage world”?

© Scott for Gestalt IT, 2009. | Is Unified Fabric an Inevitability?
Read more posts categorized as All, Storage

Fibre Channel over Ethernet is coming to Windows environments!

Although there is no word on a Fibre Channel over Ethernet (FCoE) initiator in the vein of their wildly successful iSCSI offering, Microsoft announced today that they will be creating a logo program with test requirements for the new protocol. This certification program will likely follow the company’s similar work with iSCSI and Fibre Channel adapters and a host of other products to improve functionality and interoperability in the Windows market.

This announcement shows that Microsoft recognizes the potential of FCoE as a major force in future data centers as well as the possibility of difficulty once these products are released. Reading between the lines of the release leads one to a few other conclusions:

• The Windows FCoE logo is still a year off, suggesting either that Microsoft has some work ahead of them to develop logo requirements or that FCoE implementation isn’t likely to come before then, or both
• There is no mention of a native FCoE initiator, suggesting that no such product is imminent despite calls for such a move
• Microsoft is drawing a sharp line between existing Fibre Channel products and drivers and new FCoE offerings, firmly separating the two markets and requiring that no FCoE products bear a FC logo and vice versa

The full release is in Microsoft’s Windows Logo newsletter for March 10, 2009, which has been mailed but has not yet appeared on the company’s web site.

© Stephen Foskett for Gestalt IT, 2009. | Microsoft Recognizes FCoE With Logo Program
Read more posts categorized as All, Exclusive, Featured, Networking, Storage, Top Story