• Skip to main content
  • Skip to primary sidebar
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Computer History / Faster Ethernet Gets Weird

Faster Ethernet Gets Weird

June 19, 2015 By Stephen 6 Comments

Once upon a time there was Ethernet. Every half decade or so, the industry got together and worked out a faster version. Sometimes they didn’t totally agree, but a standard emerged at 10x the speed of the previous version. Throw all that out the window: Faster Ethernet is coming, and it’s going to be weird!

Get ready for 2.5, 5, 10, 25, 40, 50, and 100 Gb Ethernet, all at once!
Get ready for 2.5, 5, 10, 25, 40, 50, and 100 Gb Ethernet, all at once!

Although it looks like a simple network Channel, Ethernet is anything but. Ever since the development of “Fast Ethernet” (that’s 100 Mb Ethernet to you and me), the IEEE has considered using multi-lane communications in various Ethernet standards. Today’s common 1000BASE-T standard multiplexes 125 Mbps signals, and 10GBASE-CX4 uses four separate 3.125 Gbps lanes.

Faster Than 10 Gb Ethernet in the Datacenter

This multi-lane weirdness is really amped up when we look beyond 10 Gb Ethernet! Essentially, there are two popular lane or channel rates:

  • 10.3125 Gbps used alone (as 10 Gb Ethernet) or in 40 Gb or 100 Gb Ethernet (and 1G and NBASE-T, which I’ll get to in a moment). Intel and Cisco seem to like this set of speeds.
  • 25.78125 Gbps is also used alone (as 25 Gb Ethernet) or in 50 Gb or 100 Gb Ethernet. This is popular with NIC and switching vendors like QLogic and Avago (via acquisitions of Broadcom and Emulex).

Already, the datacenter server market, looking beyond 10 Gb Ethernet, is facing a choice between the Intel-led 40/100 roadmap or the alternative path of 25/50/100 Gb. Avago, having acquired Broadcom and Emulex, seems to be the lead cheerleader behind the latter, though QLogic and Mellanox are also pushing it. And of course everyone is acting very friendly in public, knowing that customer adoption will ultimately drive which standard is adopted.

So far, 100 Gb Ethernet is touted as an inter-switch link, while 40 Gb or 25/50 Gb Ethernet is the focus for server interconnects. Available PCI Express bandwidth is one reason for this segmentation, since it is very difficult for any server to saturate a 100 Gb link. But PCIe Gen 4 will double per-lane throughput, opening the door to 100 Gb servers sooner rather than later. And the challenge of InfiniBand, already pushing past 50 Gb, means greater interest in 100 Gb Ethernet in high-performance computing.

The question comes down to the two roadmaps for speed between 10 Gb and 100 Gb: Do you hold out for (potentially more expensive) 40 Gb or jump on 25/50 Gb? Are you with Intel or the other chipmakers? What kind of cabling do you have or prefer?

Less Than Ten, More Than One

Many servers and network devices need more than 1 Gb of throughput but don’t want or need 10 Gb and the potential expense of fiber optic connections. That’s where NBASE-T comes in. This new standard would allow 2.5 Gb or 5 Gb Ethernet over existing copper cabling. Although the NIC and switch would need to be changed, NBASE-T has wonderful potential beyond the datacenter to provide more performance with little additional investment.

This is especially true for Wi-Fi access points, which might need more than 1 Gb of backhaul but have to make do with existing cabling. The next generation of Cisco AP’s and campus switches will likely support NBASE-T out of the box and customers will see much improved performance. Another benefit of NBASE-T is Power over Ethernet: Wi-Fi AP’s need PoE but power requires copper, making fiber-based 10 Gb Ethernet use even more difficult.

NBASE-T is a compromise between the hardware vendors, with the Intel side seeming to win out. It’s essentially 10 Gb Ethernet “slowed down” for 2.5 and 5 Gbps rather than a totally new standard. And it can run at 1 Gb too, an intriguing possibility as an alternative to today’s aged 1000BASE-T.

Here’s a great presentation on NBASE-T from Cisco at Tech Field Day

Ideally, future Ethernet chips would automatically detect all these speed combinations and negotiate the best-possible performance for any given connection. Indeed, it’s nice to see the IEEE talking about 2.5 and 5 in the same breath with 10, 25, and 40. But this isn’t a reality today. Buyers have to make a choice and ensure that both ends of the wire support whichever speed they select.

Stephen’s Stance

“One size fits all” doesn’t work for Ethernet, but this proliferation of speed options sounds like trouble without automatic capability negotiation. It’s nice to have options, but the IEEE must remain focused on interoperability and rein in the interests of the various companies proposing next-generation Ethernet technologies.

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying…
  • Ranting and Raving About the 2018 iPad Pro
  • GPS Time Rollover Failures Keep Happening (But…
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Tortoise or Hare? Nvidia Jetson TK1

Filed Under: Computer History, Features Tagged With: 10 GBASE-T, 10 GbE, 10 gigabit Ethernet, 100 GbE, 25 GbE, 40 GbE, 50 GbE, Avago, Broadcom, Cisco, Emulex, Ethernet, InfiniBand, Intel, NBASE-T, poe, QLogic, Tech Field Day

Primary Sidebar

Spreadsheets never disappoint

Warren Buffett

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!

May 29, 2020

What You See and What You Get When You Follow Me

May 28, 2019

GPS Time Rollover Failures Keep Happening (But They’re Almost Done)

April 6, 2019

Symbolic Links

    Featured Posts

    A Watch Guy’s Review of the Apple Watch

    April 27, 2015

    Marketers: Fudging the Meaning of Buzzwords Matters (To You!)

    December 2, 2015

    How Will Cisco Recover From The Consumer Strategy Blunder?

    January 2, 2013

    The Terrifying True Story Of Virtual Machine Mobility

    December 22, 2011

    A Complete List of VMware VAAI Primitives

    November 10, 2011

    Microsoft’s Big Chance to Change

    August 23, 2013

    How to Get Me to Write about Your Company or Product

    March 15, 2012

    Review: Blue Snowball USB Microphone

    March 31, 2010

    Co-Processors, GPGPU, and Heterogeneous Computing

    June 26, 2017

    Frequent Flier Kung Fu for Novices

    March 12, 2012

    Copyright © 2021 · Log in