VMware has one awesome hardware compatibility list. It’s not the breadth of it (in fact, ESX’s range of hardware support is astonishingly small) but the thorough, public way in which VMware shares this information. It’s really worth a look, and I regularly visit the list to see what’s changed.
But the thoroughness and detail of the VMware HCL can be daunting. It’s fairly easy to search for a specific piece of hardware, but it’s difficult to tell what’s supported in a general sense. This causes trouble for people who are shopping for hardware, since it’s hard to know what’s supported without an exhaustive search.
As part of the preparation for my Storage for Virtual Servers seminar series this year, I decided to do some research. I’ve boiled down certain key hardware categories into a general plain-english list of what’s in and out of the ESX HCL. In the spirit of openness, I’m presenting this data here for all to see, and I welcome corrections and updates. Indeed, I’ll try to keep this page up to date as new hardware is added!
Minimum Requirements
No one should run unsupported hardware in a production environment. So the very minimum requirement for every buyer should be a list in the VMware ESX HCL with the proper firmware, driver, and software.
FCoE also requires a flow control mechanism, preferably 802.1Qbb, and the DCBX protocol to enable communication between ports. I strongly suggest running 802.1Qaz bandwidth management as well, since it will allow you to get more performance from your CNAs than plain PFC.
Congestion management (802.1Qau) isn’t required, and I’m not sure about the state of affairs for interoperability. So that’s a “nice to have” at this point, though it will certainly become more desirable in the future.
You might also look for a CNA that supports other protocols over DCB, since iSCSI is an attractive alternative to FCoE in the data center, as is NFS. But that’s just my opinion!
FCoE CNAs for VMware ESX
Let’s kick things off with FCoE CNAs. I’m not a huge fan of FCoE in general, but I do see it as an increasingly-viable protocol for large-scale enterprise virtualization products.
It’s critical to have supported hardware, and I suggest researching specific models on the HCL before buying. But here’s the general state of affairs with regard to FCoE CNAs:
| Manufacturer | Model or Series | Supports 802.1Qaz Bandwidth Management (ETS) | Supports 802.1Qaz Data Center Bridging Exchange Protocol (DCBX) | Supports 802.1Qbb Priority Flow Control (PFC) | Supports 802.1Qau Congestion Management (QCN) |
|---|---|---|---|---|---|
| Brocade | 1007 (IBM blade) | yes | yes | yes | no |
| 1010/1020 | yes | yes | yes | yes | |
| Emulex | LP21000 | yes | yes | yes | no |
| OneConnect OCe10102 | yes | yes | yes | no | |
| QLogic | QLE8042 | no? | yes | yes | no |
| 8140/8142, 8150/8152 | yes | yes | yes | no | |
| 8240/8242 | yes | yes | yes | yes |
I was surprised to see that Brocade and QLogic list 802.1Qau Congestion Management (QCN) as supported. I didn’t realize anyone supported this spec yet, or indeed that there was such a spec!
It’s also puzzling to see that QLogic does not list 802.1Qaz Bandwidth Management (ETS) as supported for the (admittedly older) QLE8042 card. Perhaps that’s a error? But then again, Emulex doesn’t list 802.1Qbb Priority Flow Control (PFC) for the LP21000, and although FCoE doesn’t require Qbb, it’s a very good idea…
Update: Emulex tells me the LP21000 does indeed support PFC (Qbb).
Stephen’s Stance
If you’re shopping for networking gear and want to move to FCoE eventually, I hope this list will help you get started. Most OEMs supply the latest Emulex, QLogic, and Brocade CNAs with their own part number, and the long ESX HCL is testament to this fact. But, generally speaking, if you’re using a latest-generation CNA like the QLogic 8200 or Emulex OneConnect series, you ought to be fine. The Brocade CNAs look good, too, but are a little harder to find.
Emulex tells me they will add QCN support once it’s ratified, and I look forward to learning more about the Brocade and QLogic implementations. I’d also love some feedback on how well these features work, and if they’re all they’re cracked up to be. Is one vendor’s PFC or ETS as good as another’s? And does DCBX really work in practice? I believe most folks aren’t mixing HBAs widely and are instead sticking to one manufacturer. That’s probably a good idea out here on the cutting edge!
I was surprised that no Intel or Broadcom cards are listed as supporting FCoE in the HCL. The Intel X520 is impressive, and I’d taken their “open FCoE” claims to include VMware. But I guess they’re not there yet, and this seems to be a glaring omission for a major player like Intel! Similarly, Broadcom is only talking about iSCSI with VMware. Where is their CNA with VMware support?
Stephen – for the Intel/Broadcom solutions, see slide 17 of my FCoE presentation from last year http://www.slideshare.net/stuminiman/converged-networks-fcoe-iscsi-and-the-future-of-storage-networking-4013274
Emulex and QLogic solutions deliver a FC packet to the OS, Intel and Broadcom deliver Ethernet packets. Until VMware’s hypervisor can recognize and process an FCoE packet, it’s not a solution that can work.
Cheers,
Stu