Comment The temperature in the 10GbitE networking technology pot is rising as vendors sprinkle 10gigE magic into their products.
InfiniBand developer Mellanox and Fibre Channel HBA supplier Emulex are both becoming 10gigE enthusiasts. The cosy HBA duopoly of Emulex and QLogic is being blown wide open as InfiniBand and Ethernet switch vendors join in the Converged Network Adapter (CNA) fray.
InfiniBand developer Mellanox is extending its 10gigE activities by using its low-latency InfiniBand expertise to develop loss-less and low latency Ethernet products, including CNAs, the cards used by servers to send Fibre Channel over Ethernet (FCoE) messages across Ethernet links. These cards are the follow-ons from Fibre Channel Host Bus Adapters (HBAs) linking servers to Fibre Channel-based Storage Area Networks, and they converge separate Fibre Channel and Ethernet networks onto Ethernet.
The two dominant HBA vendors, Emulex and QLogic, are being joined by SAN fabric switch vendor Brocade in developing CNAs, and now InfiniBand vendor Mellanox has thrown its hat into the 10gig CNA ring as well. Brocade has bought Foundry, an Ethernet switch vendor. Voltaire, another InfiniBand developer, is also embracing 10gigE.
It appears that the data centre networking industry is in full agreement; 10gig Ethernet is going to be the dominant data centre connect technology and all other data centre networking product and technology suppliers better get on board the Ethernet train before it leaves the station.
This is prompting consolidation as Ethernet, Fibre Channel and InfiniBand product and technology vendors adopt Ethernet technology and broaden their markets to include server interfaces - the NICs and CNAs, switches - which now need FCoE, and FCoE interfaces to storage arrays. The basic Ethernet technology itself needs uprating to become lossless and low-latency, and so be suitable for carry traffic between Fibre Channel protocol-using devices which don't like losing packets or waiting for transmissions too long.
A low-latency and lossless Ethernet, a DataCentre Class Ethernet (DCE) is also a candidate for linking servers in a cluster and carrying messages between them using the RDMAoE (Remote Direct Memory Access over Ethernet) protocol, reducing the cluster burden on server CPUs.
Against this background Emulex is being pursued by Ethernet switch supplier Broadcom as it tries to build a combined CNA-switch business. The Emulex CNA uses 10gigE technology from a supplier called ServerEngines with whom Emulex has a multi-year joint development and supply agreement. ServerEngines supplies 10gigE and iSCSI, meaning TCP/IP Offload (TOE) technology, and the Emulex CNA converges an Ethernet NIC, iSCSI TOE, and FCoE capabilities into a single ASIC.
ServerEngines supplies 10gigE technology for use in HP and IBM servers, and other customers include Fujitsu, Intel and NEC. It also has an I/O controller that offloads RDMA processing as well as iSCSI TOE and FCoE capabilities, leading to the prospect of Emulex offering cluster interconnect features on its CNA products.
Jeff Benck, Emulex's chief operations officer, said: "Our jointly developed converged networking solutions are fully compatible with the seven million installed ports Emulex has shipped to date" - signalling the installed base upgrade possibilities in prospect here.
The idea is that by converging many network protocols onto a single adapter than you reduce the number of server I/O ports needed. By having the adapter construct data packets in its hardware then the host server's CPU doesn't have to use its cycles to do the job and can do more application work. In a bladed and virtualised server environment, optimising I/O port efficiency and the ability of the server blades to run virtual machines are both good things to do.
The latest market data shows that QLogic, Emulex's main HBA competitor, has been taking market share away from Emulex. The two have in excess of 80 percent market share in the HBA market. Dell'Oro, a research agency, released figures showing that QLogic had a 50 percent market share in the first quarter of 2008, a 54 percent share in the last 2008 quarter, and had grown to a 56 percent share in the first quarter of this year.
This Emulex weakness is something that gives Broadcom its takeover opportunity, and which it is aiming to change should it succeed.
QLogic has bought NetXen, a 10gigE product supplier. Its CNA does not include an iSCSI TOE capability. It may well need to do so in future if the Emulex strategy of adding an iSCSI TOE to its CNA works. Both QLogic and Emulex intend to reduce the size of their CNA to a mezzanine card, suitable for bladed server racks, and to a LAN-On-Motherboard form factor, obviating the need for an interface card at all.
The FCoE and iSCSI TOE chip can be used by switch and by storage array vendors to provide end-to-end FCoE links with no need for physical Fibre Channel. Adding RDMAoE capabilities collapses a cluster interconnect cable onto the Ethernet wire as well.
Mellanox, which supplies InfiniBand products for clustering and networking in high-performance computing and dedicated database and filer clusters, such as Oracle's Exadata machine and Isilon's products, is developing 10gigE technology products. It has introduced a 10gigE CNA, the ConnectX ENt, using Teranetics 10GBASE-T PHY technology. This is a dual-port card with virtualisation acceleration features and FCoE support.
Mellanox has also announced its own PHY, the PhyX, a physical interface transceiver that implements the base physical Ethernet networking layer. It's designed for 10GigE switches and pass-through modules, useful for server blades. PhyX is a 6-port device and supports all 10 Gigabit Ethernet physical layer functionality. It can be field upgraded to support FCoE and 2/4/8Gbit/s Fibre Channel gateway functionality with no change in hardware.
John Monson, Mellanox' marketing VP, is seeing a mainstream market here, much bigger than InfiniBand, and one that Mellanox can supply. He's seeing the coming into existence of a 10gigE ecosystem of base technology, adapter, cable and switch suppliers. Mellanox is demonstrating its new CNA at Interop Vegas in Las Vegas, Nevada in the next few days, connected to an Arista Ethernet switch, and hoping to accelerate the ecosystem's development.
It's likely that Ethernet switch vendors of all sizes will see a growing need to provide converged server interface products using combined TCP/IP, FCoE and RDMAoE, as well as switches, and it could be that we are seeing a consolidation of adapter and switch vendors underway. For storage networking vendors it is perhaps a rude awakening to the robust and busy Ethernet adapter and switch market with all three sets of vendors, and the InfiniBand suppliers, hoping to add the others' capabilities to their products to secure their installed base and grow their addressable market.
This networking convergence, this application of 10gigE magic pixie dust, seems sure to lead to a round of supplier consolidation over the next few years as Ethernet economics propel waves of change across the market. ®