Is this the year of 10 Gigabit Ethernet in the LAN?

The Problems with 10GB Ethernet

Power

If you are using copper patch leads for 10GbE, you are going to a need a lot of power. Using standard copper can use up to 45W per port (although 10GBaseCX-4 apparently uses 4.5W per port).

The IEEE is working on Energy Efficient Ethernet (802.3az) technology that will allow links to auto-negotiate down to lower speeds or go to “sleep’ during periods of inactivity which will further reduce power consumption.

Cabling

10GBaseT copper uses 650Mhz frequency spectrum and needs high quality cabling to work reliably. This means that you need to test, properly, your existing Cat5 or replace it with Cat6A or better. If you use Cat6A or even Cat6, the cable is physically much larger (and you may not have the space in your computer room). In this case, you will have up to 100 metre cable length. If you use Cat5 or Cat5e, the distance is much shorter depending on the quality of your cable, typically less than 40 metres and would probably need testing for assured reliability.

10GBaseSR uses multimode cabling but has different cable lengths depending on type of cabling and certain combinations will require mode conditioning patch leads.

The current list of 10GB XENPAK / X2 interfaces from Cisco show the confusion that the different types of cabling causes. For example consider the following table showing the LAN options (I’ve removed the WAN units) and the variation in cabling types:

X2 Product ID	XENPAK Product ID	Transceiver Type	Wavelength	IEEE Standard	Maximum Distance/Cable Type
X2-10GB-LRM	XENPAK-10GB-LRM	10GBASE-LRM	1310 nm serial	802.3aq	220m over multimode fiber
X2-10GB-SR	XENPAK-10GB-SR	10GBASE-SR	850 nm serial	802.3ae	26m over 62.5-micron FDDI grade multimode fiber 33m over 62.5-micron 200 MHz x km multimode fiber 66m over 50-micron 400 MHz x km multimode fiber 82m over 50-micron 500 MHz x km multimode fiber 300m over 50-micron 2000 MHz x km multimode fiber
X2-10GB-LR	XENPAK-10GB-LR+	10GBASE-LR	1310 nm serial	802.3ae	10 km over single-mode fiber
X2-10GB-ER	XENPAK-10GB-ER+	10GBASE-ER	1550 nm serial	802.3ae	40 km over single-mode fiber
X2-10GB-LX4	XENPAK-10GB-LX4	10GBASE-LX4	WWDM 1310 nm	802.3ae	300m over 62.5-micron FDDI grade multimode fiber 240m over 50-micron 400 MHz x km multimode fiber 300m over 50-micron 500 MHz x km multimode fiber
X2-10GB-CX4	XENPAK-10GB-CX4	10GBASE-CX4	Copper	802.3ak	15m over 8 pair 100-Ohm InfiniBand cable

The impact of cabling

In a recent project to plan a refit of an existing data centre, the 10GbE cabling needs was a major problem. Because of constraints in the change control and risk management, we eventually decided to use 1Gb ethernet because the time needed to get long change windows exceeded the length of the project.

And in other projects, the cost of recabling the fibre optic to meet the new requirement for 10GbE was prohibitive for smaller works. That is, we couldn’t just add a “patch of green” to an existing facility and extend the new switch as funds became available.

Which is weird, because it reminds me of the Token Ring / FDDI / Ethernet wars back in 1995 or so.

High Cost

If you take the time build budgetary pricing around a Cisco Nexus 7000 you will quickly realise that the cost of 10GbE capable switch is really expensive. I found that a typically configured Nexus 7018 with a good number of 10GbE and some 1GbE was around GBP £500K / USD$800K. Admittedly, this was a fully loaded model but forms the basis for a cost analysis against our existing Cat6500 choices. Frankly, I couldn’t convince anyone that this was a good idea.

Sure, the Nexus 7000 is good product (not a great product in my opinion) and offers some 10GbE capability but the lack of features and high cost means that 10GbE is still not a part of our short term strategy. I wonder how many other people have a similar problem ?

Which Year was that ?

It seems that every year is the year of 10 Gigabit Ethernet.

2009

In March 2009 The Register posted an article
Intel dubs 2009 ‘The Year of 10Gb Ethernet’.

2008

Silvano Gai, in his book Data Center Networks and Fibre Channel over Ethernet, Section 1.5.2 claimed “2008 is the year of 10GE”.

2004

Amd as far back as 2004, people have been claiming that “this is the year ….”

The market for 10-Gigabit Ethernet might start to show some signs of life in 2004. An economy gone cold, combined with its comparatively high price tag, kept down shipments of 10-GigE ports to a few thousand last year. But vendors predict that the 10-Gbps version of Ethernet will continue along a familiar trajectory. “This will be the year of 10-Gigabit Ethernet,” declared Richard Brand, director of network architecture for Nortel’s advanced technology group

Highbeam – Note this is behind a registration wall

And this article on CNET (really) 10-Gigabit Ethernet comes alive

The market for 10-gigabit-per-second Ethernet switching got off to a slow start, but now that corporate customers are looking for more speed on their networks, the technology seems to be hitting its stride.

And the fact ?

10GbE hasn’t really happened has it ? The standards took a long time to finish, and the prices have been very high for both the cabling and switching equipment. Server manufacturers aren’t putting the chips on their motherboards because of high power consumption. But most importantly, almost no-one need the bandwidth except for certain niche applications.

My Prediction

There is no question that 10Gigabit Ethernet is going to happen. Eventually. But there isn’t enough money or momentum to make 2010 a watershed year. There still isn’t enough demand for bandwidth in most parts of the networks to require the upgrade and CIO’s are investing in Virtualisation this year, not Networking.

That said, areas that require long investment cycles may buy 10Gb Ethernet for preemption against future upgrades (and thus downtime) will drive a surge in 10GbE purchases this year. For example, upgrades to data centres and storage networks (for those using iSCSI and FCoE) may purchase 10GbE switches and routers to build high performance backbones but continue to connect servers and edge switches at 1GbE. These are high visibility, high value purchases that will create a lot of marketing noise and management attention. The reality, however, is that 10GbE will be adopted small scale, and will not be used in the distribution switches, or the wiring closet, or the WAN. ((Note that it’s different for Service Providers whom I expect will have a lot more interest in 10GbE for their WAN backbones and may actually make investments in their backbones soon.))

I’m expecting 10GbE to get slow, progressive adoption over the next three years. It’s not an industry revolution, and not enough people need to increase bandwidth to drive rapid. The only use for 10GbE in the Enterprise is for Data Centres where Storage and Virtualization/Blade Servers are driving adoption.

No one else cares.