Unless you’re “in the know”, terms like “layer 2” can seem mysterious, making it all the more plausible when someone touts the benefits. It seems logical: “Bare-metal” communication must be better, faster, and cheaper than higher-level “everything over IP” approaches, right? But it’s not quite that simple.
This piece assumes you know something about the OSI model.
Ivan Pepelnjak posted a great overview of the “bridging versus routing” debate (Bridging and Routing: is there a difference?), and Marko Milivojevic posed the question in response: “I’m one of those who doesn’t understand the whole L2 obsession of the modern networking world, but…”
It really is an obsession: Data communications folks continually argue about the merits and trade-offs between high-level network protocols and low-level communications. We hear it in storage all the time: FCoE proponents assume performance benefits, and AoE fans add cost advantages to the mix. But many of these claims are unsubstantiated, and iSCSI and NAS protocols like SMB and NFS just keep rolling forward. If storage over IP is so bad, why does iSCSI work (and perform) so well?
One thing often missing in the “layer 2” arguments is what’s missing when you skip the network layer. There’s a reason IP is so widespread: It may not be the best protocol ever, but it works really well in a huge variety of situations and there is a vast pool of associated technologies that can be drawn upon when using it.
IP can run over just about anything, from FireWire to SONET, so any protocol using IP can (theoretically) run there, too. I’ve run iSCSI over Wi-Fi and WAN links, and it works great out of the box with 10 Gb Ethernet. Protocols that are tightly linked to a layer-2 protocol face stiff challenges when moving to different data links. Witness the difficulty moving Fibre Channel to 10 Gb Ethernet, including all those data center bridging technologies. In fact, when faced with the challenge of long-distance Fibre Channel SAN communication, encapsulation over IP was a natural choice.
IP also has a myriad of wonderful technologies to choose from. The creators of iSCSI were able to pull authentication, encryption, lossless communication, and many other features straight from the existing toy box. Developers of new non-IP protocols have to invent their own solutions to these problems, often with disastrous results. Why reinvent the wheel? Just apply a little CHAP, some IPsec, and roll it in TCP and you’re done!
Photo credit: “Akashi Kaikyo Bridge 明石海峽大橋” by Shenghung Lin