Then they added some of their best sales people to the mix including engineers and then went knocking on the doors of every large company in the world in a “no large customer left untouched” blitzkrieg of one on one marketing.

So What is Acadia made of ?

Acadia is offering a package of actual products (not PowerPoint releases) from EMC, Cisco and VMware using a marketing strategy called Vblock. At Boston Gestalt IT Field Day, we received an excellent presentation on the details around the Vblock package from Ed Sai and Scott Lowe that sparked a heated response. I felt that the key issues were around two points:

• Why are the Vblocks so Large ?
• What happens if you exceed the operational parameters of a Vblock ?

Because there was much discussion about why the Vblock that have some strict limitations, I wanted to look into why these limitations exist and why people would object so vehemently to them. And also talk around the question for why Vblocks are big. Too big for most people to buy.

Comprehending the Limits on the joint venture

Acadia is not a merger (yet) of EMC and Cisco, but a partnership focused on the Data Centre. The partnership only promotes selected product groups from each company and even specific products within those group. For example, a Vblock does not include a Nexus 7000 switch, only the Nexus 5000 / 6000 / 2000 as the edge switches. ( You would use your own backbone to connect to your Ethernet network and legacy FibreChannel network ). VMware does not offer VDI solutions. EMC offers CLARiiON in Vblock1, and Symmetrix in Vblock2.

These limits may mean that your choices are limited. Some people regard this as as major concern. I regard this as a reasonable trade off – good service, easy marketing against lack of choice and flexibility.

Other Companies

It’s worth remembering that the joint venture is not exclusive. Both EMC and Cisco are free to pursue partnerships with other companies, and both are actively seen partnering in the market with other companies. Thus recent Cisco / NetApp and EMC/IBM announcements could create a confusing picture for some people who don’t understand these dynamics. That said, from what I’ve seen the Acadia partnership seems to be a “big deal” for both sides with a lot of passion, commitment and, most importantly, significant amounts of cash that fund expensive marketing campaigns. In this case, marketing that seems to actually be relevant and useful.

Selected Products

Acadia only works with selected products from both Cisco, EMC and VMWare. That is, they only offers products from the Data Centre Business Unit from Cisco, selected products Centerra and Celera at EMC and VMWare vSphere. The product is positioned in the market as a private cloud so any technology not relevant to that message isn’t going to be considered eg. No firewalls, no edge routers, no wireless, no backup software, no deduplication,.

My perception is that clear business goals drive this logic:

• Market clarity – limiting the product set allows Acadia to have a highly targeted product message, especially with the “cloud” buzzword attached.
• Technical capacity to deliver an effective sales force
• Leaves space for ‘other partners’ to have similar relationships with customers and not ‘upset’ the reseller marketplace
• minimise the business impact if the project fails
• customer want guarantees that it will work which should get better sales results.
• Acadia has to deliver the solution and deliver the support promise. Not easy when so many technologies are involved, with so many competing interests

I particularly want to focus on the Technical Capacity and Customer Guarantees elements of the argument. If you figure that EMC, Cisco and VMware are bringing a group of vendor engineers together, think about the problems you might have: engineers may have little or no expertise outside of their own company or worked in a cross-functional team. To overcome these problems I would go for a narrow product selection and then expand it over time.

Remember that large companies create significant opportunities for ineptitude and incapacity. Layers of management can create indirection and misalignment just as easily as they can create opportunity and focus. Bringing together teams from three large companies could easily go wrong (case study – HP and it’s many divisions that are rarely coherent).

There are other topics, such as upsetting the resellers, market clarity are all pretty obvious and shouldn’t need discussion.

What are Vblocks ?

The Vblock components are the cumulative permissions of what the marketing and business limitations will allow. There are no technical limits to what equipment could be in a Vblock except for the ability of Acadia to provide a good level of support. These limits are somewhat onerous but not entirely unreasonable. There are currently two Vblocks defined (and more are planned) – known as Vblock 1 and Vblock Two, they are designed, tested and (as is the fashion) certified to within certain, rigidly defined performance levels.

Basic Components

The Vblock architecture is reduced to five layers of physical infrastructure and roughly shown in this diagram taken from “Vblock Infrastructure Packages Reference Architecture” released in 2010.


Outline Bill of Materials

From this, you can see there is a fair amount of variation within the Vblock parts list to cater for your specific requirements. To quote Acadia :

A Vblock consists of a minimum and maximum amount of components that offer balanced I/O, bandwidth, and storage capacity relative to the compute and storage arrays offered. Each Vblock is a fully redundant autonomous system that has 1+1 or N+1 redundancy enabled by default.

In practice, Vblock 1 storage is scaled to deliver from ~42 TB/41000 IOPS (Min) to ~68 TB/ 50000 IOPS (Max). Vblock 2 storage is scaled to deliver from ~140 TB/92000 IOPS (Min) to ~211 TB/140000 IOPS (Max). The hardware is selected to match your performance requirements.

Other flexibilities

Some other quotes from the Cisco / EMC / VMware reference architecture guide:

For Vblock 1, there are no hard disks on the B-200 series blades as all boot services and storage are provided by the SAN. Hawever a snall hard drive may be installed if local page memory is required for vSphere.

For Vblock 2, each B-200 series blade module has 72GB SATA drive for page memory purposes. If required, these can be removed to reduce power and cooling overhead, increase MTBF or save costs.

The MDS9506 switches are recommended, but may optionally be changed for 9509 or 9513 to scale capacity or reduced to an MDS 9222i is less density is required

Inflexibilties

For Vblock 1, there are no hard disks on the B-200 series blades as all boot services and storage are provided by the SAN. Hawever a snall hard drive may be installed if local page memory is required for vSphere. If the local disk is user for main storage or operating system storage, it is not considered a Vblock and is a custom implementation at this point.

It is also acceptable for operating system and applications to be run directly on the B-200 series blades. It should be noted that other hypervisors are not supported by Vblocks and invalidate the Vblock support agreement (my emphasis)

The Limits are arbitrary, practical and mainly marketing

At the Boston Tech Field Day, we saw a number of questions on why there are limits, and what happens if you exceed the solution parameters. I believe it is a simple marketing reason – simple products, simple factors, easy to comprehend makes for a product that is easy for sales to sell, and even easier for customers to buy. Pricing is clear and unambiguous, and sales peoples will attempt to focus the customers away from price onto other topics. Like low power, special UCS features, ease of operation, etc etc.

The key factor is that the Cisco UCS server will be perceived by many as unproven. You can be sure that HP / IBM / Dell are going to work that Fear / Uncertainty / Doubt in a competitive situation.

Exceeding the Limits – You Certainly SHOULD

If you choose to exceed the offered the solution, then that is a simple choice. It’s not something to be particularly concerned about. Currently, we all build stacks of servers, storage and networks and provide our own support. Why would buying a guaranteed stack make any difference ?

These limits are to ensure that service guarantees can be met. The service guarantee exists to help customer to buy these products., it does not exist to make the products work. However, when the CIO is about to approve a purchase order for half million pounds they will be looking for a comfort level that it’s going to work. Those Cisco UCS servers are new and you want some assurance.

So once the system is working, feel free to head out in your own direction. Meeting the requirements of your business is more important that having a an end-to-end guarantee in overall picture. Remember, Cisco / EMC / VMware is still going to support each product using the same process that we have today.

The Fear of Lock In

My concern with the Vblock bundle is fear of lock in. The Cisco UCS product is a closed solution within the racks. Across the backbone of the Data Centre will also be Cisco Nexus 7000 switches for “best compatibility”. All server components must be purchased from Cisco who will be able to arbitrarily set the price. All and any server upgrades, and extra server must come from Cisco.

All storage vendors are an effective lock in today. The so-called “certification programs” ensure that EMC and other “chosen few” are restricted to the supply of drives and accessories.

If Cisco / EMC achieves a dominant position then they have the ability to control pricing. Since both companies are already dominant in their respective markets, and tend to “reassuringly expensive” price models, this is a well founded concern.

What does a Vblock physically look like ?

While every Data Centre is different, every rack looks pretty much the same. During my research, I have realised that a Vblock takes very little space in the data centre. In fact, it’s not until you see the amount of space that you realise the impact to

Vblock 1 – Racked

This is a front view of the Vblock 1 with storage, servers and switches.

Caption Text.(Click for a full size image)

And this is the rear view. Notice how few cables are used for physical connections here.

Caption Text.(Click for a full size image)

Vblock 2 Racked

A Vblock 2 is quite a bit larger and needs more storage. The server blades are larger (more CPU, more RAM per blade). I believe that the increased IOPS for the storage subsystem means that bigger storage units are needed. I think five racks is at the smaller side of the Vblock 2 possibilities.

Caption Text.(Click for a full size image)



Realising the dream – no cables

Caption Text.(Click for a full size image)

Vblock expansion

Even though the Vblock is modular, you can bet that Acadia has a nice little pitch around expansion. Sure enough, it is spelled out in the Architecture Reference:

Caption Text.(Click for a full size image)

As Vblocks are adde, the capacity of the Vblock scales either as an aggregated pool, whereby any ICS blade can access any storage disks n the SAN or as an isolated silo. For example, it is perfectly acceptable to aggragte to Vblock 1′s to provide capacity for 6000 VMs that can share common storage capacity… As long as storage capacity is added in conjunction with compute capacity to maintain balanced performance as published within the Vblock, the system does not require any additional validation .

Heh, you bet they want to sell you lots more so of course there is an answer for this. Once you have made the first purchase, you are likely to come back for more. Lets make it easy for the customer to do that.

So, Why are vBlocks so Large ?

A Vblock 1 is expected to support from 800 to 3000 VMs, and a VBlock 2 is expected to support 3000 to 6000 virtual machines. Since I started writing this post, a Vblock 0 has been announced that supports 300 to 800 VMs.

You might think that 800 is a lot of VMs. You might think that the price of a Vblock is too much. In which case, you aren’t someone who is going to buy it. If you can’t scratch together the several hundred thousand in single purchase order, you do’t understand the question. Today, many companies buy infrastructure as part of the project. If a project needs a server and a switch then that is funded. If a project needs a data centre, then a data centre is built. If a project needs fifty servers, then they MIGHT consider using VMware. Mostly we wait a few years and then have a big project to clear up the mess created by project funding of infrastructure.

Cisco and EMC don’t want to sell this product to small business. Cisco in particular wants to establish themselves in the marketplace as quickly as possible. Therefore selling to big companies, who are doing the “big cleanup project”, who are taking the time to evaluate the whole cost of ownership cycle, implementing with a full cycle approach is what Cisco wants. This is important.

Why ? Because Cisco wants to be able to produce white papers claiming the “huge cost reductions”, and “radical operations alignment” and “enabling technologies” etc etc. These white papers have real impact on the analysts and customers who figure that if they buy the product they will get those benefits. All these great stories, so loved by ‘journalists’ convince the middle sized market to buy Cisco servers.

What does EMC get ? EMC gets to move into the mid-sized market right alongside Cisco. EMC has been targeting the high end of the market, and they need new markets. A move down into the mid-sized market must not damage the high end products and branding (since this could affect profits), and attaching to a “differentiated” product will keep the premium marketing message intact.

EtherealMind View

Basically, I view the Acadia partnership as an upmarket fancy marketing campaign with extensive tie-ins. It’s “dressed up with a little black dress and killer set of heels” and well funded to boot. This type of thing normally makes me distrustful as many of these initiatives have turned out to be one night stands with an uncomfortable and unhappy morning after.

On the other hand, both Cisco and EMC have attached their corporate goodwill to this program in a major way. If Acadia fails, both companies will suffer significant image loss. It’s not marriage, but they are definitely living together.

The Challenge to the Status Quo

One of the most exciting (to my mind) aspects of the Vblock concept, is the potential to change the purchasing cycle of IT Projects. Many companies fund new purchases from projects only. This tends to create an inherently failed infrastructure buildout where each project buys what it needs. Project “cost controls” mean that infrastructure is rarely purchased to outside the project requirements. That is, buying additional disk drives or edge switches is easy, but buying additional arrays or core switches is almost impossible.

A Vblock could represent a “once a year” purchase of server infrastructure. Much easier than endlessly integrating, and reintegrating a hodge podge of servers, memory, disk drives and accessories into some sort of complete system. Compare this with the HP C-Class chassis concept which requires endless upgrades to scale into new and more useful configurations.

Most importantly, once engineers are freed from mundane tasks of installing memory and disk drives, they can move to focussing on the management platforms and dramatically enhancing the business value of computing. Tasks such as automating software and OS deployment, automated failure responses, performance analysis and

Cisco needs to buy EMC

Ultimately I remain convinced that Cisco must and will buy EMC to compete with HP and IBM. The Acadia joint venture looks a lot like a trial or temporary position while the market reaction is gauged. EMC has a marcap of USD$34 billion and revenue of USD$14 billion while Cisco has marcap of USD$154 billion on revenue of USD$35.5 billing. Cisco appears capable of acquiring EMC and is believed to have sufficient cash reserves to make a substantial cash rich offer. EMC probably has pretentions to be a lot bigger and don’t want to be bought right now. EMC egos are well known for their ‘capacity’ and they work hard at that. Combine that with a Wall Street that is anti-acquisition at the moment, and Cisco can’t make a move right now.

Public Clouds are not ready for the Enterprise

Public Clouds are fine for people that are not conducting serious business today. Dot Com startups, marketing pushes, publicity campaigns and other low vitality requirements could be considered for cloud infrastructure due to their trivial nature, but serious business applications are unlikely to move into the cloud with the current level of capability.

However, private clouds are practical today. To me, Vblock looks like an early mover in the Private Cloud space for IaaS. The option for a corporate IT department to build a three rack system that hosts between 300 to 700 servers is an exciting development. If I can reduce the amount of time my operations teams spend doing trivial and repetitive tasks and move them into systems automation on the management platforms, then we are moving to better place.

Technology Acceptance

There are no technology issues in this stack. In general, people perceive that Cisco and EMC are market leaders in their respective spaces, and the corporately the two companies fit well together. Unlike other stacks, there doesn’t appear to be a product that is less than excellent. In this sense, few people will dismiss the Acadia solution on technology grounds.

Conclusion

Convincing IT Management to take on the change put forward by Cisco / EMC is going to require a lot of marketing and sales. And that’s exactly what Acadia is all about. The technology takes care of itself here, Acadia is just marketing exercise, but a good one.

© Etherealmind for Gestalt IT, 2010. | Market positioning Acadia, EMC, Cisco and the whole vBlock Idea
Read more posts categorized as Cloud Computing, Featured, Networking, Server Virtualization, Storage, Top Story

• Connecting Nexus 5000 to Older Gigabit Ethernet Switches
• Setting Up FCoE on a Nexus 5000
• FCoE and VLAN Trunking on Nexus 5000

One thing I hadn’t yet documented was how to enable jumbo frames on a Nexus 5000. Since jumbo frames are now officially supported for VMkernel traffic with VMware vSphere, the combination of jumbo frames and 10Gb Ethernet is an attractive one. I’ve covered the ESX/ESXi side (ordinary vSwitches here and distributed vSwitches here), but here’s the Nexus side.

The commands are pretty straightforward, and I’ve included the commands for both NX-OS 4.0 and NX-OS 4.1 (they are different between versions). Important note: if you enabled jumbo frames under NX-OS 4.0 and then upgraded the switch to version 4.1, you’ll need to re-do your jumbo frame configuration.

For NX-OS 4.1, the commands to enable jumbo frames are:

switch(config)# policy-map type network-qos jumbo
switch(config-pmap-nq)# class type network-qos class-default
switch(config-pmap-c-nq)# mtu 9216
switch(config-pmap-c-nq)# exit
switch(config-pmap-nq)# exit
switch(config)# system qos
switch(config-sys-qos)# service-policy type network-qos jumbo

Now, contrast the commands above with the following commands, which you would have used to enable jumbo frames on NX-OS 4.0:

switch(config)# policy-map jumbo
switch(config-pmap)# class class-default
switch(config-pmap-c)# mtu 9216
switch(config-pmap-c)# exit
switch(config)# system qos
switch(config-system)# service-policy jumbo

The end result of these differences is this: if you upgrade NX-OS from 4.0 to 4.1, then your jumbo frames configuration will go away, and you’ll need to enter the commands for version 4.1 in order to enable jumbo frame support again. This little gotcha caused me quite a headache when my NFS-based datastores suddenly went offline after the NX-OS upgrade.

More information on the necessary commands can be found here for version 4.0 and here for version 4.1.

© Scott for Gestalt IT, 2009. | Enabling Jumbo Frames on a Nexus 5000
Read more posts categorized as Networking, Server Virtualization

Acadia is interesting, but it really isn’t the meat of the announcement, in my opinion. The real substance of the matter is the nature of the coalition. There are many interesting questions/thoughts circling in my head right at the moment:

• What impact will this have on VMware’s relationship(s) with HP, IBM, and Dell? “Throwing their hat in the ring” with Cisco’s UCS, so to speak, may greatly endanger VMware’s much larger (with respect to revenue) relationships with other OEMs. What will happen to VMware if those OEMs “throw their hat in the ring” with Microsoft and Hyper-V? This is not a good place to be.
• The acrimonious Cisco-HP relationship adds further fuel to the concerns over VMware’s close alliance with Cisco’s computing platform.
• Does this new coalition signal a move away from the “arms-length” relationship between EMC and VMware, a move that some (competitors, notably) have been talking about for some time? If so, what danger does that put VMware in with regards to storage relationships?
• It seems to me that VMware has the most to lose here. What does EMC lose if this doesn’t go well? Nothing, really. What about Cisco? Nothing, really. VMware, on the other hand…well, it could be ugly.
• What does this coalition offer that the three companies couldn’t deliver without the coalition? Why risk important relationships? This is a big question in my mind. Lots of technology companies have delivered validated designs without any sort of formal coalition. Why is one necessary in this case?
• On the other end of the spectrum—keeping Acadia out of the picture for the moment—is this “new coalition” really anything more than what the three companies have already been doing? Is this really anything more than each of the companies dedicating resources to this effort? I know from my own direct interaction with at least one of these vendors that resources had already been dedicated to the VCE technology intersection before any sort of formal announcement. So, does this formal announcement really mean anything at all?

I don’t have any answers (yet), but you can at least read my thoughts—and contribute back to them via the comments—without having to pay $499 to some analyst firm.

By the way, if you’d like some other viewpoints on this matter, here are a couple from opposing viewpoints:

NetApp – Jay’s Blog: The Importance of Being Open
Chuck’s Blog: Announcing the VCE Coalition

Feel free to speak up in the comments below (courteous comments only, please, and be sure to include full vendor disclosure where appropriate). Thanks!

© Scott for Gestalt IT, 2009. | A Few Quick Thoughts on the VCE Coalition Announcement
Read more posts categorized as Networking, Server Virtualization, Storage

1. Enable FCoE on the switch.
2. Map a VSAN for FCoE traffic onto a VLAN.
3. Create virtual Fibre Channel interfaces to carry the FCoE traffic.

The first step is incredibly easy. To enable FCoE on the switch, just use this command:

switch(config)# feature fcoe

The next part of the FCoE configuration is mapping a VSAN to a VLAN. What VSAN should you use? Well, if you are connecting to an existing Fibre Channel fabric, perhaps on a Cisco MDS switch, you’ll need to make sure that the VSANs between the Nexus and the MDS are appropriately matched. Otherwise, traffic on one VSAN on the Nexus won’t be able to reach devices on another VSAN on the MDS. If there’s enough demand, I’ll post a quick piece on this step as well.

Note that this FCoE VSAN-to-VLAN mapping is a required step; if you don’t do this, the FCoE side of the interfaces won’t come up (as you’ll see later in this post). Assuming the VSAN is already defined, perform these steps to map the VSAN to a VLAN:

switch(config)# vlan XXX
switch(config-vlan)# fcoe vsan YYY
switch(config-vlan)# exit

Obviously, you’ll want to substitute XXX and YYY for the correct VLAN and VSAN numbers, respectively.

After you’ve enabled FCoE and mapped FCoE VSANs onto VLANs, then you are ready to create virtual Fibre Channel (vfc) interfaces. Each physical Nexus port that will carry FCoE traffic must have a corresponding vfc interface. Generally, you will want to create the vfc interface with the same number as the physical interface, although as far as I know you are not required to do so. It just makes management of the interfaces easier. The commands to create a vfc interface look like this:

switch(config)# interface vfc ZZ
switch(config-if)# bind interface ethernet 1/ZZ
switch(config-if)# no shutdown
switch(config-if)# exit

At this point the vfc interface is created, but it won’t work yet; you’ll need to place it into an VSAN that is mapped to an FCoE enabled VLAN. If you don’t, the show interface vfc <number> command will report this (emphasis mine):

vfc13 is down (VSAN not mapped to an FCoE enabled VLAN)

As I mentioned earlier, if you haven’t mapped the FCoE VSAN onto a VLAN, you won’t be able to fix this problem. If you have mapped the FCoE VSAN onto a VLAN, then you only need to assign the vfc interface to the appropriate VSAN with these commands:

switch(config)# vsan database
switch(config-vsan-db)# vsan <number> interface vfc <number>
switch(config-vsan-db)# exit

At this point, the vfc interface will report up, and you should be able to see the host’s connection information with the show flogi database command.

From this point—assuming that your storage is attached to a traditional Fibre Channel fabric, which is likely to be the case in the near future—you only need to create zones with the WWNs of the FCoE-attached hosts in order to grant them access to the storage. Refer to my posts on creating zones and managing zones on a Cisco MDS for more information on this task.

In my own experience, once FCoE was properly configured on the Nexus 5000 switch, then creating zones and zonesets on the Cisco MDS Fibre Channel switch and creating and masking LUNs on the Fibre Channel-attached storage is very straightforward. This, as has been stated on several previous occasions, is one of the strengths of FCoE: it’s compatibility with existing Fibre Channel installations is outstanding.

© Scott for Gestalt IT, 2009. | Setting Up FCoE on a Nexus 5000
Read more posts categorized as Networking, Server Virtualization, Storage

Multi-hop FCoE falls into two basic scenarios:

• FCoE initiators and/or FCoE targets separated from an FCF (fibre channel forwarder) by multiple hops through IEEE DCB-compliant Ethernet switches
• Multiple FCFs chained together to connect FCoE initiators and FCoE targets

There are additional scenarios, but for now let’s discuss just these two.

In the first scenario, FCoE initiators and FCoE targets might be separated from an FCF by one or more IEEE DCB-compliant Ethernet switches (also known as an “FCoE passthrough”). In this situation, FCoE Initialization Protocol (FIP) would be required in order for the FCoE initiators and FCoE targets to communicate. Now that FIP support is beginning to emerge following the ratification of the FC-BB-5 standard in early June, this sort of scenario becomes more possible.

If you think like me (and if you do, I’m very sorry to hear that!), your next question is, “OK, what is an IEEE DCB-capable switch?” As it turns out, the Nexus 5000 can be an IEEE DCB-capable switch (or an FCoE passthrough). Cisco doesn’t advertise that fact because they don’t feel that building a solution out of a bunch of Nexus 5000 switches is the best approach. OK, fair enough, so the Nexus 5000 isn’t really designed to be used that way. So what other options are there? None of which I’m aware, at this point, so that makes it impossible to build multi-hop FCoE solutions today. When a valid IEEE DCB-capable switch or FCoE forwarder does appear then you’ll be able to build these sorts of designs—assuming that you have FIP support in both the FCoE initiators and the targets. (Note that you could mix pre-FIP components in here, but all such components would have to be connected directly to the FCF, and would only be able to communicate with other components connected directly to the FCF.)

In the second scenario, FCoE initiators and targets are connected directly to an FCF—like a Nexus 5000—but you’ve got multiple FCFs chained together to create a larger fabric. You might consider this analogous to linking multiple MDS 9000 series switches together with inter-switch links (ISLs). In this case, FIP support would still be necessary for initiators to connect to targets on a different FCF, but now there’s another wrinkle. You see, Cisco has the concept of a VSAN (think of it like a VLAN for Fibre Channel—this is a simplistic definition but reasonable enough to use). In the MDS world (keeping in mind that NX-OS, the software running on Nexus switches, has its roots in SAN-OS, the software that runs on MDS Fibre Channel switches), there is the concept of trunking E_ports, where multiple VSANs are carried on a single E_port between two MDS switches. Continuing the VSAN/VLAN analogy, a trunking E_port is analogous to an 802.1q VLAN trunk.

Bear with me, there’s a reason I’m telling you all this.

When you use FCoE on a Nexus 5000, you end up mapping each VSAN to a VLAN. When you need to move from one FCF to another FCF—i.e., from one Nexus 5000 to another Nexus 5000—how should the VSAN information be presented? Should the VSAN information reside in the 802.1q VLAN tag, so that an 802.1q VLAN trunk is considered a trunking E_port with regard to VSANs? Or should the VSAN information remain embedded in the FC commands that are encapsulated by Ethernet? This fundamental question has not yet been answered. There are advantages and disadvantages to each approach, and as the T.11 group responsible for FC-BB-5 and other FCoE standards hasn’t yet come to agreement yet on how to handle this, then it’s currently not possible to create the FCoE equivalent of trunking E_ports (I believe these will be referred to as VE_ports). Since you can’t create VE_ports, you can’t connect multiple FCFs together, and you can’t build a multi-hop FCoE fabric composed of multiple FCFs.

As you can see, then, that even with FIP present in all components, neither definition of multi-hop FCoE is possible today. Although a Nexus 5000 can function as an FCoE passthrough, Cisco doesn’t recommend that architecture. Without any other IEEE DCB-capable Ethernet switches available to use as an FCoE passthrough, that makes the first scenario impossible to build. Likewise, the inability to create VE_ports and trunk VSANs across multiple Nexus 5000 switches means that it’s impossible to build the second scenario today. While multi-hop FCoE is the ultimate goal, it’s just not possible right now.

Here’s some food for thought while you digest this information: how would a fabric extender change things? That’s a topic I’ll delve into in a future post, so stay tuned!

Of course, FCoE experts and wizards are encouraged to add your corrections, clarifications, and thoughts in the comments below.

© Scott for Gestalt IT, 2009. | Why No Multi-Hop FCoE?
Read more posts categorized as Networking, Server Virtualization, Storage

The recent surge by storage networking vendors (Cisco and Brocade et al) to develop and roll out Ethernet based storage networks means that Storage and Networking teams are going to need to work together. The most common reaction from Storage professionals has been “I wouldn’t trust my storage data on the Data Network, I can’t rely on it!”.

Which is ridiculous and, ultimately, futile. As a Data Networking designer, I can assure you that is what the Voice Networking people said as the IP Telephony/VoIP technologies dismantled their careers. This isn’t any different.

FibreChannel is a single function network. Data Networks are multifunctional. And because of that, they are cheaper.

Why is Brocade walking away from FibreChannel ?

Brocade basically invented FibreChannel so that they could create a market that suited themselves very nicely. Which is fine because a number of other vendors joined in which validated the market that Brocade domainated.

But since Cisco bought the MDS its been clear that they were not going to settle for being number two and have been aggressively attacking the storage networking marketplace. Cisco knows that its strength is its existing customers and data networking, so the obvious progression is to move storage to the Data Network.

Enter FCoE………

Cisco picked a team of engineers and created a startup, Nuova Systems, with a brief to develop Ethernet Storage networking. They developed Fibre Channel over Ethernet and series of hardware products that had the silicon to support the requirements.

And they are serious requirements. The largest and most critical storage networks need low latency, low jitter and lossless network to ensure good operation. (Lets not talk about the fact that the storage marketplace has been so completely oversold on the story of lossless and low latency networks with FibreChannel that they now believe it is actually true).

So Nouva Systems built an Ethernet switch that has the necessary capabilities. It doesn’t seem to have been hugely difficult since it only took them a couple of years, and their are competitive products from companies like Woven Systems, that already have most of these features before they started.

They also started the process for a number of new Ethernet standards that allowed the storage technologies to exist.

So Cisco then bought Nuova back in in a buyout and positively threw the product into the marketplace with a blaze of hyperbole and whizzbang marketing. Three months later the products began to ship to selected companies. Yep, its the Nexus 5000 which has FibreChannel AND 10 Gigabit Ethernet in a single chassis. Hosts can now use FCoE to connect to existing FC networks and storage.

Storage Networks are actually quite small

One thing that Storage People seem to miss is that Storage Networks are really small. A few dozen FC switches is a big storage network. For Data Networks, several hundred switches is a big network. Its the importance of the data that “makes appear bigger in the mirror” and the lack of tolerance to interruption.

New Technology, New Designs

There are three missing pieces of the puzzle for Storage over Ethernet. They are:

• Design & Architecture
• Operational Skills and Management
• Technology Standards

Design & Architecture

Network Architects need time to learn the new technologies and design methods. Historically, Data Networks have not been sensitive to loss and new designs and thinking are need to handle this. Fortunately, the storage network is strictly limited to the Data Centre so this is relatively straight forward. Architects, generally, are able to adopt new technologies once the need is clear. (Enter the Account Manager……)

Operation and Support

This will require new procedures and some training, but it won’t be much different from what we do today. Some of the monitoring and analysis tools need upgrades to enhance their performance analysis, and the reporting tools that we use for monitoring bandwidth will also need upgrades. Today, this work is performed by the Storage team, and this work will move the Network Operations team.

Data Networking will need to improve it performance to match the service levels that Storage needs. But that is not a technology problem, it’s a business problem.

Standards

This is the current weakness in the Storage over Ethernet story. The key standards that control how the HOST interacts with the Network in QoS signalling and negotiation are not near complete. FCoE itself is nearing completion, and a number of new 802.1 standards that allow Ethernet networks to have new features that support much faster convergence (sub-second) and higher available bandwidth’s (SPBB) will be ready in the next year or so.

But I can’t trust it ? Can I ?

As it stands today, I can only agree that it would be a very brave decision to invest in FCoE. Lack of standards, limited experience and insufficient resources for knowledge and expertise mean that you will be doing a lot of hard work.

I personally think that Storage over Ethernet is a stupid idea and believe that Storage over IP will be far more successful for more than 90% of the market. However, I also recognise that:

• Cisco has spent more than a billion dollars bringing FCoE to market
• Brocade bought Foundry to get their own Ethernet products at a cost of $3 billion
• Companies like NetAPP and EMC have announced and are shipping FCoE enabled devices
• Emulex and QLogic have chipsets and adapters in the market. I believe Intel will have something shortly.

and the final push from Cisco is their Unified Computing Strategy for which FCoE is absolutely fundamental. And that is getting a lot of mindshare since it seems to move into markets dominated by HP & IBM.

So its time to accept the inevitable….

If there is one thing I have learned in twenty odd years of technology, it is this: Cheap Always Wins.

In the end, FibreChannel will never be big enough to be cheap like Ethernet and there is no chance that FibreChannel will continue for long now that a cheaper option exists. You can argue about technical superiority all you like, whine about feature this and feature that. but it won’t matter, in the end, cheap wins, and the Data Network is cheaper to buy, cheaper to run, and cheaper than Fibrechannel.

It’s time to start getting friendly with your Network Architect and get working on the new, cheaper, future. See if you can build a Storage Data Network that you can live with, not one that gets forced on you.

© Etherealmind for Gestalt IT, 2009. | Monofunctional or Multifunctional – Cheap always WINS
Read more posts categorized as Featured, Networking, Storage

Cisco today introduced their B-Series UCS blade server

Did you feel the earth move today? No? Despite months of hyperbolic warnings, Cisco’s release today of their Unified Computing System blade servers (code named Project California) is gentle and evolutionary. The networking giant is challenging HP, IBM, and Dell, to be sure, but not with a slap in the face. Cisco is easing into the server pool, unlike Sun who is reportedly preparing to take a cannonball leap into the networking pool.

Cisco will add blade servers to their lineup, that much is true. But these new servers are meant for virtualization only, using VMware and Microsoft Hyper-V. Suddenly the integration of Cisco’s technology into VMware in the form of the Nexus vSwitch makes much more sense.

They’ll be a solid alternative in the VMware server market, to be sure, but what will the long-term effect be? Certainly, they reveal Cisco’s approach to the new virtual datacenter world, “Data Center 3.0“. Rather than bothering with a soup-to-nuts line of servers, Cisco will focus where the most value is for integrated systems: high-end virtual servers. This fits well into their traditional network-focused strategy, as well as VMware’s vision of data center evolution, vSphere.

So is this introduction just a bunch of hot air? Not at all! The UCS includes some innovative technology that separates it from the pack:

• I/O is converged using 10 GbE with (pre-standard) Data Center Bridging technologies
• The blade servers are beefy, and the 7-server plus 1-vSwitch architecture is scalable
• A new management platform (UCS Manager) integrates blade server hardware and network provisioning and configuration
• It leverages enterprise storage (EMC and NetApp are happy!) with fabric extenders

Is it a slam dunk? No way! Cisco faces some serious challenges:

• Do companies want to integrate networking and servers politically and organizationally?
• It looks solid (with backing from VMware, Intel, and Accenture), but how well will it be supported at the customer side?
• How will Cisco partners and new partial-competitors, HP, Dell, and IBM react to this challenge?
• Will the 10 GbE DCB technologies that Cisco is leveraging really scale in the real world?

We shall see…

© Stephen Foskett for Gestalt IT, 2009. | Cisco Enters the Virtual Server Hardware Market
Read more posts categorized as Featured, Server Virtualization

You see, Cisco bought Nuova Systems to rush FCoE to market as the Nexus 5000 switch. But someone forgot to complete the standards work, and now this is causing problems. Customers are obviously concerned that FCoE has NOT been standardised.

Both the ANSI and IEEE are moving slowly on completing the standards work for FCoE and its supporting CEE etherent technologies. Cisco must have hoped that they would have been finished by now and might be getting nervous that things are moving slowly.

I think Omar is wishing that people would stop worrying about standards and just put their money down. Nervous are we ?

There is a lovely quote here on Cisco’s Data Centre blog

“Its also important to note, as Silvano Gai, who literally wrote the book on FCoE, pointed out, standards are organic things. If they are going to be useful and stay relevant, standards need to remain organic, so waiting for a standard to be “done” is kinda pointless.”

Huh, what? Standards don’t matter ?

Is that a used car you are trying to sell?

I’m thinking that if Omar (chief FCoE standard bearer for the entire world) tell us that standards don’t matter then sales are slow. If sales are slow, then FCoE might not cross the adoption gap…..

PS. I read the book. Review: Cisco’s documentation is better.

© Etherealmind for Gestalt IT, 2009. | Cisco is Wishfully Wishing for FCoE Sales?
Read more posts categorized as Networking