Windows 8 server editions will include a filter driver for NTFS for data deduplication

The next version of Microsoft Windows Server includes integrated data deduplication technology. Microsoft is positioning this as a boon for server virtualization and claims it has very little performance impact. But how exactly does Microsoft’s de-duplication technology work?

Introducing Windows 8 Deduplication

Let’s make one thing clear right from the start: Microsoft started from a clean sheet and invented their own deduplication technology. This is not a licensed, cloned, or copied feature as far as I can tell. There are some clever aspects to it, along with a few head scratchers for folks like me who’ve seen lots of different deduplication approaches.

Microsoft’s deduplication is layered onto NTFS in Windows 8, and will be a feature add-on for Server users. It is implemented as a filter driver on a per volume basis, with each volume a complete, self describing unit. It is cluster aware, and fully crash consistent on all operations. This is a pretty neat trick: As is typical for Microsoft, deduplication will be a simple, transparent feature.

Now let’s talk for a moment about what Windows 8 deduplication is not.

• It is a server-only feature, like so many of Microsoft’s storage developments. But perhaps we might see it deployed in low-end or home servers in the future.
• It is not supported on boot or system volumes.
• Although it should work just fine on removable drives, deduplication requires NTFS so you can forget about FAT or exFAT. And of course the connected system must be running a server edition of Windows 8.
• Although deduplication does not work with clustered shared volumes, it is supported in Hyper-V configurations that do not use CSV.
• Finally, deduplication does not function on encrypted files, files with extended attributes, tiny (less than 64 kB) files, or re-parse points.

Some Technical Details on Deduplication in Windows 8

Microsoft Research spent 2 years experimenting with algorithms to find the “cheapest” in terms of overhead. They select a chunk size  for each data set. This is typically between 32 KB and 128 KB, but smaller chunks can be created as well. Microsoft claims that most real-world use cases are about 80 KB. The system processes all the data looking for “fingerprints” of split points and selects the “best” on the fly for each file.

After data is de-duplicated, Microsoft compresses the chunks and stores them in a special “chunk store” within NTFS. This is actually  part of the System Volume store in the root of the volume, so dedupe is volume-level. The entire setup is self describing, so a deduplication NTFS volume can be read by another server without any external data.

There is some redundancy in the system as well. Any chunk that is referenced more than x times (100 by default) will be kept in a second location. All data in the filesystem is checksummed and will be proactively repaired. The same is done for the metadata. The deduplication service includes a scrubbing job as well as a file system optimization task to keep everything running smoothly.

Windows 8 deduplication cooperates with other elements of the operating system. The Windows caching layer is dedupe-aware, and this will greatly accelerate overall performance. Windows 8 also includes a new “express” library that makes compression “20 times faster”. Compressed files are not re-compressed based on filetype, so zip files, Office 2007+ files, etc will be skipped and just deduped.

New writes are not deduped – this is a post-process technology. The data deduplication service can be scheduled or can run in “background mode” and wait for idle time. Therefore, I/O impact is between “none and 2x” depending on type. Opening a file is less than 3% greater I/O and can be faster if it’s cached. Copying a large file can make some difference (e.g. 10 GB VHD) since it adds additional disk seeks, but multiple concurrent copies that share data can actually improve performance.

Stephen’s Stance

Although I am intrigued by Microsoft’s new deduplication technology in Windows 8 server, I still have many questions about its usefulness and impact on performance. Concentrating duplicate data in the system volume makes sense from a technical perspective, but could lead to an I/O hotspot on the disk. This is especially true for external caching storage systems, since there is no integration between Microsoft deduplication and storage array features. I am particularly concerned about the use of deduplication with VHD files in Hyper-V, since it could eat up valuable system RAM and impact I/O performance.

If you would like to try Microsoft deduplication for yourself, I am happy to report that it is included in the developer preview of Windows 8 that is available on Dev Center. Here are a few commands to get you started, and read Rick Vanover’s post too!

Import-Module ServerManager
Add-WindowsFeature -name FS-Data-Deduplication
Import-Module Deduplication
Enable-DedupVolume E:
get-dedupvolume

Note: I am a Microsoft MVP and Microsoft briefs me on upcoming technologies under NDA. This post is based on a Microsoft briefing from November which was said at the time not to be covered by any NDA. All of this information could be gleaned by experimenting with the Windows 8 developer preview, but it’s much easier to just go to the source.

© Stephen Foskett for Gestalt IT, 2012. | Microsoft Adds Data Deduplication to NTFS in Windows 8
Read more posts categorized as Server Virtualization, Storage

Ekahau and MetaGeek are working together to bring Wi-Fi site surveying and spectrum analysis to the masses in a portable format

The most interesting products and companies at Interop Las Vegas 2011 were found around the edges of the show floor. Companies like NEC, Synology, Ciphertex, and Endace may have gone unnoticed in the shadows of towering booths of the industry titans but deserve attention. One such pairing was two Wi-Fi analysis companies, MetaGeek and Ekahau. Both work together to enable spectrum analysis and site surveying on portable devices – smart phones and tablets.

MetaGeek Makes Spectrum Accessible

One of my personal favorite presentations at Wireless Field Day in March was the awesome startup tale told by Ryan Woodings, founder of MetaGeek. While working on a wireless mouse dongle, Ryan and company noticed RF interference and turned lemons into lemonade: They repurposed the USB dongle as a spectrum analyzer and started a company to make this formerly esoteric technology accessible.

Although MetaGeek’s Wi-Spy is nowhere near as full-featured as the big guys, it’s far more accessible at 1/10 the cost. It’s cheap enough that a home hobbyist could pick one up as a way to learn about Wi-Fi and 2.4 and 5 GHz wireless spectrum. And the Windows software is easy enough that even a storage guy like me could figure out that my home Wi-Fi router was on the wrong channel.

At Tech Field Day, Ryan wowed the crowd with an early peek at an iPad app to interact with Wi-Spy captures. I ran into Ryan at Interop, and he showed me a more-polished version of that app, promising it would hit the App Store soon. He also hinted that an iPhone version would follow, and showed off email and Dropbox integration.

Ekahau Site Survey: Laptop to Android

MetaGeek had a guest in their booth from another company I’d heard of from my Wi-Fi engineer friends: Ekahau makes site survey products for laptops and was talking about an Android version as well. Ekahau’s Mobile Survey supports a variety of Android devices, including the popular Samsung Galaxy Tab and Motorola Droid.

Stephen’s Stance

Wireless Field Day exposed me to a whole new world of enterprise IT. The wireless engineers there spend days roaming through corporate facilities doing site surveys, planning Wi-Fi access point installations, and troubleshooting connectivity and interference. Most use Windows laptops, but it’s exciting to think that Apple iPhones and iPads and Android phones and tablets may also be used in the future. These devices are much more portable, with great battery life and interactive screens. And it looks like MetaGeek and Ekahau are leading the way. I can’t wait to get an update from these companies at Wireless Field Day 2 in early 2012!

© Stephen Foskett for Gestalt IT, 2011. | MetaGeek and Ekahau: Wi-Fi Analysis To Go
Read more posts categorized as Featured, Networking

I’ve got a new video podcast up and running: Raising the Floor is a series of discussions about the future of enterprise IT. I kicked the series off talking about one of my favorite topics: Cloud storage.

I invited two excellent guests to join me for this conversation:

1. Josh Goldstein, VP of Marketing, Cirtas
2. Andres Rodriguez, Founder and CEO, Nasuni

It was a pretty broad discussion, all packed into less than half an hour. I urge you to check out the podcast (and subscribe in iTunes, rss, or email) but I wanted to share a few excerpts. You can also read the entire transcript over at Foskett Services: I’ll be posting it as a series of articles over the next week!

Andres Rodriguez: I think one of the things that hurt StorageNetworks at the time was the fact that they couldn’t deploy the equipment in an efficient, multi-tenant way. And so, if you look at the new cloud architectures, (places like Nirvanix, places like Amazon) those systems are designed from the get-go to be shared among many, many users, and make very efficient use of the equipment and the software running it across that user base.

…

Josh Goldstein: Today, the cloud is built on top of process that are very difficult for most companies to replicate on their on. So, the price you’re paying to your cloud provider includes not just storing your data but also keeping multiple replicas of that data spread across different geographical sites.

You’re highly protected against not only a disk drive failure, but also an entire array failure or even an entire site failure where your information’s still is survived those kinds of events and is remaining accessible to your when you need it.

That’s something that for most organizations to engineer that level of reliability is extremely expensive and difficult for them. The cloud providers have been able to do that at scale and still deliver the capacity to you with that type of protection at a price point that’s really pretty amazing.

…

Andres Rodriguez: I’ll start there and I said anything that traditional storage world, we have file based systems. We have block based systems and that is exactly where Cirtas and Nasuni stand. We are the equivalent of say a company named EMC and NetApp for the cloud world. The approaches are complimentary. And they are both trying to solve the same problem. I’ll start on the file side but Josh can take on the block. But on the file level is really, look, if you want to have something that behaves very much like a file server, say like a NetApp box. It means you are going to have a file system, you want to have a protocol to export it locally on so something like CIFS, you are now going to have access to directory integration so that you can have access control. This is what makes a file server useful in the datacenter.

…

Andres Rodriguez: One of the really interesting things about that is that Josh is essentially describing thin provisioning in the classic sense. But unlike doing thin provisioning and this is the cloud is automatically provisioned. So he is essentially promising the volume. But then the volume is growing gradually, smoothly without any administrative interference. Without you having to worry about it; the volume is filling in its data as it goes. And that is one of the things we talked about in the beginning.

The cloud really allows you to build a different type of storage system, because automatic provisioning is really thin provisioning should be but isn’t. When you’re talking about physical linked arrays that are limited by actual hard drives, you know, running, spindles running in your datacenter.

Again, you can watch the video of the whole conversation right now over at Foskett Services, or you can read the transcripts that will be posted there over the next few days.

Disclaimer: I worked at StorageNetworks, and Nasuni is a client of Foskett Services.

© Stephen Foskett for Gestalt IT, 2011. | Talking Cloud Storage Gateways With Nasuni and Cirtas
Read more posts categorized as All, Storage, Tech Field Day Roundtable Podcast

W. Curtis Preston launched his own series of Backup Central Live! seminars for 2011

Last week, after the Exec Event in Palo Alto, I joined my friend W. Curtis Preston for his first Backup Central Live! event. Curtis has spent years educating IT pros about data protection, this was the first week of a new series of self-produced events. And let me tell you, although I’ve seen him present dozens of times, Curtis was really in his element here. He held the packed room enthralled, and the vendor sponsors I talked to were very pleased about the event!

Introducing Backup Central Live!

The Backup Central Live! series are day-long seminars across the USA in 2011. Each event includes over 3 hours of content from “Mr. Backup”, W. Curtis Preston, as well as presentations from Jacob Farmer and the sponsoring vendors. The seminars are free for qualified end-users, which includes most of the readers of this blog!

Curtis and company will cover the challenges of backing up and recovering data in a variety of settings:

• Virtualized servers (e.g. VMware, Hyper-V, Xen)
• Very large servers and data centers
• Remote offices and laptops
• Data retained for multiple years

The session also includes technical detail about key products and technologies:

• Cloud Backup Services
• Deduplication
• Continuous data protection (CDP) and near-CDP
• Archive software
• Tape and its proper role

Attendees even get free breakfast and lunch, which was of a good hotel caterer quality in my opinion.

Stephen’s Stance

The Backup Central Live! crew does a great job putting together a professional event

I knew Curtis could put together quality backup content, but the crew deserves credit for such a professional and successful event. They attracted some great sponsors, too, including AppAsure, Aptare, FalconStor, NEC, Quantum, Spectra Logic, and Cirtas. And Jacob Farmer’s involvement was a pleasant surprise, too: I’ve always enjoyed the deep technical conversations I’ve had with him!

If you enjoyed my own backup, archiving, and storage seminars in the past, I know you’ll love this event. The next Backup Central Live! cities are as follows. If you’ll be around, you really ought to attend!

• Orlando, FL Feb 1 Register here
• Houston, TX Feb 8 Register here
• Chicago, IL Feb 22 Register here

My only suggestion for the crew is that they get a bigger room next time!

© Stephen Foskett for Gestalt IT, 2011. | See W. Curtis Preston’s Backup Central Live!
Read more posts categorized as Desktop, Events, Featured, Server Virtualization, Storage

Although it’s fun to speculate about blockbuster purchases, smaller acquisitions make much more sense

The industry is buzzing with rumors that Isilon will be the next big storage acquisition now that 3PAR is safely in HP’s hands. The alleged buyer, EMC, is itself surrounded by rumors of becoming a shared feast for Oracle and Cisco. But there are much more accessible and tasty items on the enterprise IT acquisition menu, and the lower cost of entry makes them much more likely to come about!

Buying Talent, Technology, and Market Share

There are three good reasons companies look to buy others:

1. They lack the talent to play in a certain market. Although engineers are often touted as the prize in tech acquisitions, other kinds of talent (sales, marketing, and leadership) often make more of the difference. But buying talent is perilous because (especially in California), talent can walk out the door.
2. The next gap to be filled is technology itself. Startups are often more innovative and creative, filling new product niches and taking risks on new concepts. This manifests itself as product, of course, but IP and roadmap are equally important.
3. There is also market share to consider. Companies love to crow about their sales leadership, and very little matches the messaging that comes from overtaking a competitor in sales. Buying up a smaller, but not insignificant, competitor is a great way to achieve this!

There are a few more reasons companies get bought, of course. Companies sometimes build up vast cash reserves, making them ripe for a buyout. Others bring together assets that are more valuable split apart and sold off than together. Still other acquisitions are made to keep a competitor from having a key component of the market.

There are many great companies in enterprise IT, often packed to the gills with talent, technology, and market share. NetApp, EMC, Juniper, Symantec, and so many others are loaded with value, yet not all are about to be acquired.

What’s Stopping You?

It’s not easy for a company to buy another. The most-obvious roadblock is financial: As I discussed regarding Oracle, HP, and NetApp, companies can use cash, debt, or stock to make the purchase. Cash is clean and tidy, but each has its own trade-offs. At the end of the day, the purchase price can be too big a hurdle to overcome.

NetApp and Juniper have market caps around $17 billion, making them a very expensive purchase. Symantec is a little smaller at $12 billion, but it’s still large enough to be out of play for many companies. And EMC, at $42 billion, would be a historically-large acquisition, probably requiring more than one company at the table.

Although far smaller, Isilon begins to look expensive once one looks at the prospects for return on investment. Currently priced just under $2 billion, Isilon has solid clustered NAS technology and is a major player in high-performance computing. But they are not even ranked in the top external storage vendors according to IDC, and EMC and NetApp rule the mainstream NAS market. Their annual revenue grew respectably from $61 million in 2006 to $120 million in 2009, but that’s small-potatoes in enterprise IT terms. And with just $79 million in cash and short-term investments, they won’t be bought for plunder either.

Ripe Fruit

Isilon is just too small to justify a multi-billion dollar acquisition, but there are other tastier possibilities in play. Three in particular come to mind:

1. FalconStor has excellent technology, proven over a decade and used by thousands of customers. They have excellent OEM connections, too. And their recent CEO ousting makes them a killer bargain at under $150 million. Seriously – they could make that much revenue next year given a solid sales re-launch. at the very least, their stock (NASDAQ:FALC) is a solid buy!
2. Overland (NASDAQ:OVRL) is in the midst of a re-birth, but their purchases of Snap and MaxiScale make them an acquisition target in my mind. Their astonishingly-small market cap of just under $17 million is less than their quarterly revenue. Yes, they’re losing money, but this would change quickly in the hands of a strong benefactor.
3. Finally there’s big-data protection specialist, Sepaton. It’s hard to say exactly what a purchase would entail, since Sepaton isn’t a public company, but it’s probably safe to assume that the price would be small and the acquired debt somewhat larger. But their technology is just waiting to be blasted prime-time by some big-data gorilla.

See the common theme here? Great technology, huge potential, and a miniscule price. Although none of these would be much of a market share win initially, all could be serious threats to the established players with some sales effort. Any of the big equipment companies (EMC, Dell, IBM, Oracle, or Cisco) would make hay with Overland, and any big software player (Oracle, CA, IBM, HP, or Symantec) would find much to love in FalconStor or Sepaton.

Stephen’s Stance

Although multi-billion dollar blockbuster deals are more fun to speculate about, smaller technology purchases like these can have much greater leverage. I expect at least one of these companies to be snapped up in the next year, and wouldn’t be surprised to see all three gone before Isilon, EMC, or Symantec. Indeed, I just don’t see the business case for an Isilon acquisition today.

Image credit: Cash Register 70 by zizzybaloobah

© Stephen Foskett for Gestalt IT, 2010. | Why FalconStor, Overland, and Sepaton Ought To Be Acquired Before Isilon
Read more posts categorized as Storage

Every technician develops a set of tools and practices that serve them well. In this post, I present my best practices for enterprise IT.

Although many people are cynical about the whole idea of best practices, I’m a believer. I think that such beasts do exist, just that too many companies, analysts and especially consultants spend too much time applying the label to whatever works in their best interest at the time. To counteract this cesspool of non-best practices, I thought it best to put down a few ideas of my own. Following are four fundamental best practices I have distilled from almost 20 years in enterprise IT. I wonder if you agree with them.

Before proceeding, I suggest that you take a look at the three principle tests that I use to define a best practice.

Best practice 1: Minimize Complexity

IT has a tough position in business. We are ignored while everything is working well, and in trouble when things start falling apart. We are held accountable, but generally cannot control our own destiny. The only way we have a fighting chance is to minimize the complexity of the solutions we must support.

As I said in Sailing the Titanic (Why We Need ILM and Then Some!): “To make a timely (and tired) Harry Potter analogy, IT are the house-elves of the business – powerful but subservient, with little input into what happens above and around them.”

This is where the current trends of virtualization, consolidation, and standardization comes in. A single administrator can manage a far-larger environment if it’s standardized, and these systems tend to be far more reliable as well. Sadly, minimizing complexity is the antithesis of a true hacker’s heart. Who wouldn’t want to play with the latest and greatest toys? Why not build an intricate and interconnected system? But these solutions fail my “best practices” test: They’re risky, unusual and just plain imprudent.

Best Practice 2: Use the Right Tool For the Job

Any woodworker or mechanic will tell you that using the right tool makes any job easier. Even if it works, it’s just bad practice to masquerade network-attached storage (NAS) as a block device or a storage area network (SAN) as a collection of shared files. I am always skeptical of so many oddball innovations trumpeted by vendors large and small because they seem to fail the “right tool” practice.

I wrote more about this in Use Process Solutions For Process Problems, Technical Solutions For Technical Ones and Why Do I Ignore NAS?

But sometimes they work out just fine. You see, best practices can’t predict the future or what works in a certain situation. It’s better to minimize complexity and stick to what you know than to press for the perfect solution every time. It wouldn’t be right to bring SAN storage into an all-NAS environment just for a single database application, for example. Although we generally shouldn’t try to drive a nail with a screwdriver, it can be the best option when a hammer can’t be found!

Best Practice 3: Prepare For Failure

The hacker in me is tempted to stop tinkering as soon as a system is functional, but the realist in me knows that I have to make it twice as good. You can’t prepare for every eventuality, but you can seek out probable points of failure and harden them.

For example, see Chris Evans’ discussion of Choosing Between Monolithic and Modular Architectures

Many of the fundamental best practices in enterprise IT come down to this. Redundancy from RAID and N+1 components, the widespread use of multi-path technology, and high-availability design are all examples of preparing for the worst. We would not bother with all of this if we didn’t know that errors occur even in well-engineered systems, and that everything we build is sensitive to failure.

Best Practice 4: Align Expectations with Reality

We all face conflicting demands, and we are tempted to offer best-case responses to keep everyone happy. But excessive optimism leaves IT folks stressed and end users frustrated when things don’t go right. It’s time to adjust expectations.

See The Techie/Business Schism, where I talk about the looming disconnect even within IT

Concepts like service level agreements (SLAs) and business impact analyses come from an innate knowledge that the real world doesn’t always conform to the best-case assumption. Although we don’t have real control over user demands, we can try to set correct expectations by admitting what we can and can’t do realistically.  Don’t let those who rely on us labor under misconceptions of our capabilities.

Stephen’s Stance

We can’t let others lead us astray with talk of “best practices”, but we can define some of our own. My three-part best practices test, and these four general practices, have served me well over the years. What are your best practices? Image credits: Dave’s Bike Tools by bre pettis

© Stephen Foskett for Gestalt IT, 2010. | Four Fundamental Best Practices for Enterprise IT
Read more posts categorized as Featured, Storage, Tech

Donate your conference backpacks to school kids in need!

Trade shows are a veritable swag-fest, some with great loot and some with junk. I’ve been critical of the booth babe and chotchkie phenomenon, but my friend Kevin Houston has a better suggestion: Donate your (useful) swag like backpacks and pens to school kids in need.

Kevin first suggested this to me at HP Tech Forum after I refused the giveaway backpack I wouldn’t have used. It turns out that he’s been doing this for a while, with a focus on the backpacks so many vendors distribute. It’s funny – my own son uses my VMworld 2009 backpack and I have to admit it’s a well-built and useful bag. But I hadn’t thought of the underprivileged kids out there who could use it even more.

Did you know that 41% of kids in the United States come from “low-income” households? According to the NCCP, over half of the kids in Arkansas, DC, and Mississippi are under financial stress. I’m sure back-to-school time is a serious disappointment for many kids whose parents can’t afford a new backpack, pencils, and other school supplies. Many don’t realize that school districts can’t afford enough supplies either. In fact, it’s so common that the IRS allows teachers to deduct up to $250 in out-of-pocket expenses to equip their classrooms with basic supplies.

So I’d like to echo Kevin’s call: It’s time for us techies to lend a hand and support education. The next generation needs us! When you come home from VMworld and similar events, load that backpack up with school supplies (notebooks, pencils, scissors, and the like) and donate it to a local school. They’ll be glad for the help!

Photo by Kevin Houston

© Stephen Foskett for Gestalt IT, 2010. | Donate Your Swag to School Kids In Need
Read more posts categorized as Events, Featured

The Internet is responsible for the demise of many “meatspace” industries, from magazine publishing to CD sales, but there are some things it’ll never do: No matter how much time you spend interacting online, meeting in person is something else entirely. I’m not trying to advocate a return to the old world of tech conferences and user groups. Rather, I envision a new breed of connected events that reinforce online communities with real face-to-face communication.

The Human Element

People used to read trade magazines and scour conference halls to uncover product and technology gems, but this kind of research is better done online today. The other day, a friend-of-a-friend tweeted a request to suggest “NAS storage in the $3k to $5k range.” Knowing I would have some suggestions, my friend retweeted the request and I fired back a list of suggestions. Since we were on Twitter, my reply included links to the companies that made the products in question. Back in the old days, this product search might have lasted weeks or months, but I bet jjx had her research done in hours.

EMC World airport bus interview: Karl Lewis of UofM from Stephen Foskett on Vimeo.

So why go to a trade show if not to discover products? It’s all about the people. Increasingly, trade events like EMC World and Storage Networking World are becoming venues to meet people who were previously mere online avatars. They are for building friendships, business development, and cementing real connection and community. A look at my recent trade-show videos on Vimeo will reveal that most of my own interaction is personal, and the relationships built there could never be forged online.

David Scammell talks SNW and SpiceWorks from Stephen Foskett on Vimeo.

The new people I have met, like Karl Lewis and David Scammell, will become part of my own community, and I will become part of theirs. We would have never met without the Internet (Karl recognized me from my blog, and David spotted me on Twitter), but we would never have become friends without real human interaction.

Devolving the Conference

Tech Field Day Overview (Short) from Stephen Foskett on Vimeo.

The same can be said of the Gestalt IT community. We were friendly before we met, but our interactions at Tech Field Day cemented a surprisingly strong bond, especially between the delegates and presenting vendor staff. We are much more a community now than before these events. This is the main reason we will continue the Field Day event series: It brings a new kind of community to life! This kind of interaction might have happened at old-school conferences, but the specific (some might say peculiar) nature of the Field Day (invite-only, independent, and blogger-centric) makes it unique.

Another event I really look forward to is Greg and VaNessa Duplessie’s BD Event. It’s like the Field Day in that it distills a single element from the old trade show days to its purest form. In this case it is the business-to-business back-channel discussion that gets the focus. The BD Event is all about business development, and every part of it is designed to foster the sharing of ideas and opportunities between business people. I love the concept, and I hope I will see a tremendous crowd at the Boston event next month!

Watch live video from The Cube LIVE from EMC World 2010 on Justin.tv

Even traditional conferences are changing. EMC World had a tremendous blogger lounge set up, including an amazing live video setup. SiliconAngle brought us “The Cube“, a live webcast that brought the social elements of the blogger lounge to a massive online audience. My hat is off to NiceFishFilms for their technical expertise, yet it was the human element, getting such diverse people together in the same place, that made it all work.

Stephen’s Stance

The Internet will certainly change many elements of the traditional trade show, but it will never kill the need for real interaction. I look forward to the next clever twist on the conference, complementing Gestalt IT’s Field Day, the BD Event, and The Cube, but do not worry that we will all some day remain in our secluded caves. We are social people, and nothing beats meeting someone face to face.

© Stephen Foskett for Gestalt IT, 2010. | Nothing Beats Meeting In Person
Read more posts categorized as Events, Featured

Networking may be straightforward, but the world of networking terminology is not. I’ve been steeped in the strange lingo of Ethernet for many years, but I still get confused by some of the terms. What’s the difference between 1000BASE-CX, 1000BASE-SX, and 1000BASE-T? In this post, I’m going to tackle this Ethernet network naming convention.

Networking Basics

Let’s get the first two components of the network name out of the way:

The first part is the signaling rate in megabits per second. In layman’s terms, this is the speed of the network at hand. You are likely to come across one of the following:

• 10 megabits
• 100 megabits – Fast Ethernet
• 1000 megabits – Gigabit Ethernet, GbE, or 1000BASE-X
• 10,000 megabits – 10 Gigabit Ethernet, 10GbE, or 10GBASE-X
• 40,000 megabits – 40 Gigabit Ethernet, 40GbE, or 40GBASE-X
• 100,000 megabits – 100 Gigabit Ethernet, 100GbE, or 100GBASE-X

It may strike you as odd that the next part is always the word, “BASE”. But there is a reason for this, too.BASE refers “baseband”, meaning that this is an unfiltered line not requiring a digital modulation scheme. Back in the day, there was a 10PASS-TS version of Ethernet that used a signaling scheme similar to a modem, but baseband is dominant today.

So 100BASE refers to a Fast Ethernet connection that uses the unfiltered cable for transmission.

BASE-What?

The third part of an Ethernet network type refers to the cabling used to carry the signals. The earliest forms of Ethernet used coaxial cable, but thin twisted-pair cabling became popular in the mid-1990s. Faster versions of Ethernet also often use fiber optics rather than electrical signals.

There are a bewildering assortment of physical interconnects for Ethernet. But the naming system isn’t as complex as it might appear:

• The first letter tells us which kind of wire we are talking about:
  • “T” means twisted-pair cable (e.g. the common Cat5 in use today)
  • “K” means a copper backplane
  • “C” means balanced copper cable
  • “F” means optical cable
  • “B” uses two wavelengths over a single optical cable
  • “S” means short-range multi-mode optical cable (less than 100 m)
  • “L” means long-range single- or multi-mode optical cable (100 m to 10 km)
  • “E” means extended-range optical cable (10 km to 40 km)
  • “Z” means long-range single-mode cable at a higher wavelength
• Next is the coding scheme for data on the wire
  • “X” means 4B/5B block coding for Fast Ethernet or 8B/10B block coding for Gigabit Ethernet
  • “R” means 64B/66B block coding
• Finally, we have a number representing the number of parallel “lanes” for data
  • “1″ would mean serial (non-parallel) but is omitted instead
  • “4″ or “10″ are available for copper wire
  • Just about any other number could be used for optical lanes or wavelengths

Examples

Now let’s look at some examples:

• Back in the day, 10BASE-T became more common than coaxial 10BASE2. It was a simple 10 megabit baseband signal over common twisted-pair.
• When Fast Ethernet first rolled out, there was some concern that traditional (usually Cat3) cabling couldn’t handle 100 megabits. Some early implementations used four copper pairs (100BASE-T4) or fiber optics (100BASE-FX), but nearly every 100 megabit Ethernet connection today is 100BASE-TX, using plain two pairs on plain Cat5 cable.
• Gigabit Ethernet had a similar history. Many were concerned that two pairs on unshielded Cat5 wiring could not handle 1000 megabits per second, so optical (1000BASE-SX) and balanced shielded wiring (1000BASE-CX) were specified. Although an unshielded 2-pair standard was developed (1000BASE-TX), it never really caught on. Therefore, today’s predominant gigabit LAN connection, 1000BASE-T, uses all four pairs of unshielded twister-pair wiring on a Cat5 cable (see note 1).
• The 10 Gigabit Ethernet world has mostly shifted to the block coding scheme from Fibre Channel, 64B/66B, which is denoted by the letter “R”. This gives us a family of fiber optic cables (10GBASE-SR, LR, ER, etc), and a copper backplane interconnect (10GBASE-KR). The earlier copper wiring standard (10GBASE-CX4) used InfiniBand-like 4-lane cables and 8B/10B signaling, as did 10GBASE-KX4 on the backplane. A backwards-compatible twisted-pair 10GBASE-T has also been developed, but work continues to make it power-efficient enough to be practical (see note 2).
• Looking ahead, we see Higher-Speed Ethernet emerging: 40GBASE-KR4 for backplane use, multi-mode optical 40GBASE-SR4 and 100GBASE-SR10, and long-range single-mode optical 40GBASE-LR4 and 100GBASE-LR10.

As you can see, all this alphabet soup does have some consistency. Common unshielded twisted pair wiring is all “BASE-T”, optics are denoted according to their range (“S”, “L”, “E”), and backplanes use “K” copper. Clear as mud?

Note 1: Lots of people (and even equipment makers) incorrectly refer to common Gigabit Ethernet as “1000BASE-TX”, but this really should be called “1000BASE-T”.

Note 2: We will probably never see a 10GBASE-TX, which would use just 2 pairs of unshielded twisted pair copper wiring.

© Stephen Foskett for Gestalt IT, 2010. | 1000Base-What?
Read more posts categorized as Networking, Server Virtualization, Storage

My presence at Storage Expo NL may have been cursed, but my presentation went off without a hitch thanks to the creativity and flexibility of the VNU staff and Expo volunteers like Ilja Coolen. In my session, I talked about the shortcomings of traditional tiered storage as a way to advance the Noble Goals of Storage Management and proposed a solution: Although merely using different disk types will never deliver the goods, adding flash and cloud to an integrated, automated solution will be truly revolutionary. I look forward to the day when all of today’s buzz-worthy technologies (flash, cloud, thin provisioning, automated tiering, post-RAID) are mixed together to form a really revolutionary storage system.

I went ahead and recorded the entire presentation and posted it on Vimeo for anyone to see. But I just realized I never posted it here to my blog. So without further ado, I give you “Extreme Tiered Storage: Flash, Disk, and Cloud!”

Stephen Foskett on Extreme Tiered Storage from Stephen Foskett on Vimeo.

By the way, in case you were wondering, I used BoinxTV to create this video.

© Stephen Foskett for Gestalt IT, 2010. | Extreme Tiered Storage: Flash, Disk, and Cloud
Read more posts categorized as Cloud Computing, Storage