Ethernet throughput has constantly increased. Data rates have multiplied from the initial specification of 10 Mbps...
in 1983 to 40 Gbps and 100 Gbps in 2010. While both 40 Gbps and 100 Gbps have been available since 2010, why did work on a 25 Gigabit Ethernet (GbE) standard not begin until 2014?
Increasing server performance and the cost of server-to-switch connections are the reasons for the 25 GbE effort. Currently, many cloud and data center servers support 10 GbE network interfaces and are connected to top-of-rack switches via either copper or fiber cables. New generations of processor chips, combined with higher levels of CPU utilization due to virtualization, are threatening to turn the 10 GbE link into a bottleneck.
Eliminating such bottlenecks was just one of the goals of the 25 Gigabit Ethernet Consortium, which began operations to create a new Ethernet standard in July 2014. The group, backed by Arista Networks, Broadcom, Google, Mellanox and Microsoft, said it wanted to enable "cost-efficient scaling of network bandwidth delivered to server and storage endpoints in next-generation cloud infrastructure, where workloads are expected to surpass the capacity of 10 or 40 Gbps Ethernet links."
The group developed a specification for 25 GbE and 50 GbE, and urged other vendors to adopt it. Later, in July 2014, the Institute of Electrical and Electronics Engineers (IEEE) formed a study group to explore 25 GbE standardization. In December, the study group transitioned into the 802.3by task force charged with developing the standard.
While the server to switch connection is expected to be the primary use of 25 GbE technology, access switches now supporting 10 GbE uplinks are expected to evolve to 25 GbE as data rates from end-user devices increase. The 25 Gigabit Ethernet technology, which is expected to be formally approved later this year is also engineered to support storage traffic.
Single-lane 25 Gigabit Ethernet will anchor the new Ethernet standard
Increasing server performance could have been supported by transitioning from 10 GbE directly to the existing 40 GbE standard rather than developing a new 25 GbE specification. Yet, while 40 GbE would have provided more throughput than needed, the problem was cost. Server and switch interfaces would have become more expensive and network cabling would have to be replaced.
An Ethernet interface uses a serializer/deserializer (SerDes) component to generate 64-bit units of serial data. The 64-bit units are then encoded with an additional 2 bits to support clock synchronization. The 66 bits are transmitted via the physical coding sublayer through a cable to the receiver's PCS and on to a SerDes that transfers the data to higher interface layers. An Ethernet lane includes a single SerDes on each end and a single transmission path.
At the time that 10 GbE was standardized, SerDes components were limited to 10 GbE. These components are now capable of 25 GbE, making it possible to transmit 25 Gbps over a single lane. A 25 GbE interface is currently more expensive than 10 GbE, but the increase isn't proportional: It's not two-and-a-half times more expensive. The cost is expected to drop as volumes ramp up. No new cables are required to upgrade from 10 to 25, GbE since both can transfer data through the same twinax or twisted pair copper cable or through single mode or multimode fiber.
Four 10 Gbps lanes are required for 40 GbE. Each lane requires a separate SerDes. The PCS then distributes the four 66-bit units in a round-robin fashion across the four lanes. On the receive side, the units are placed back in proper order and sent to four SerDes components.
There are two ways to transmit four lanes over fiber. Each lane can be sent over a separate fiber pair or all four lanes can be combined on a single pair by using wave division multiplexing with each lane transmitted at a different wavelength. Transmitting four lanes over copper is defined by 40GBASE-CR4. Each lane is transmitted over a separate twinax cable. In either case, fiber or copper, four lanes increase cost and energy consumption.
The 25 Gigabit Ethernet Consortium released a 50 Gbps specification at the same time it released the 25 Gbps spec. It uses two 25 Gbps lanes to deliver higher throughput than the 40 GbE standard while requiring only two lanes rather than four.
Vendor support for a new Ethernet standard
The IEEE is not expected to finalize the 802.3by standard until this fall, but network equipment vendors have begun testing their products. The Ethernet Alliance conducted tests of 25 GbE technology at the University of New Hampshire (UNH) Interoperability Lab last June. A long list of vendors participated, including switch and server vendors Arista, Cisco, Dell and Hitachi; network connector vendors Amphenol and Molex; and Ethernet adaptor vendor Intel and QLogic. Mellanox, which produces switches, connectors and adaptors, also tested its products, while Ixia, Spirent, and Xilinx supplied test equipment.
The reported success rate of 86% was considered to be extremely encouraging, especially considering that, in June, the new Ethernet standard was a year from completion. Results of the tests were fed back to the IEEE committee to aid them as they resolved final details in the standard.
The successful UNH tests indicate that vendors are well along in their 25 Gigabit Ethernet product design and implementation efforts. At this point in time, Arista, Cisco, Huawei and Mellanox have announced products supporting the new spec and Hewlett Packard Enterprise announced it intends to develop both servers and switches supporting 25 GbE interfaces.
Server performance monitoring and virtualization
Can 25 Gb Ethernet cure data center oversubscription?
Security and 25 GbE: What you need to know