News Stay informed about the latest enterprise technology news and product updates.

40 GbE uptake will ride 10 Gigabit server adoption wave

40 GbE will make its way into data center networks as enterprises adopt large numbers of racks with 10 GbE server connectivity. 10 Gigabit servers will drive up port density and eventually force enterprises to take the 40 GbE plunge.

With enterprises just beginning to deploy 10 Gigabit Ethernet in their data center networks, why are vendors already pushing costly 40 Gigabit Ethernet?

This week Force10 Networks detailed its core-to-edge 40 Gigabit Ethernet (GbE) strategy beginning with the S4810 top-of-rack switches featuring 48 10 GbE ports and four 40 GbE uplink ports scheduled for release later this month. In early 2011, Force10 will launch a 40 GbE line card for its ExaScale core switch/router chassis. The line cards each have six 40 GbE ports, four of which are usable at one time. With the line cards, the ExaScale can support up to 56 active ports of 40 GbE in a half-rack chassis.


Force10's 40 GbE move follows two top-of-rack announcements from competitors Blade Network Technologies and Extreme Networks. Two weeks ago Blade announced its RackSwitch G8264 top-of-rack switch with 64 10 GbE ports and up to four 40 GbE uplink ports. Last spring Extreme Networks unveiled its VM3-40G4X, a four-port 40 GbE module that plugs into its Summit series of top-of-rack switches, giving them 40 GbE up-link capabilities.

This is just the beginning of the 40 GbE wave. Alan Weckel, director at Dell'Oro Group, said that by next spring's Interop every networking vendor will have announced top-of-rack switches with 40 GbE uplinks, if not also core switches with 40 GbE ports.

Large-scale 10 Gigabit Ethernet servers will drive 40 Gigabit Ethernet adoption

Most network engineers are still upgrading from 1 GbE to 10 GbE in their data center networks, so they don't even want to think about 40 GbE switches for now. But this growth in 10 GbE connectivity at the server edge will ultimately drive 40 GbE uptake, said Kevin Wade, senior director of product marketing at Force10

Enterprises that have deployed servers with 10 GbE connections integrated into their motherboards require server edge switches with 40 GbE of uplink bandwidth into the aggregation and core layers of the data center network, Wade explained. Today most enterprises achieve this bandwidth by combining four 10 GbE uplinks at each top-of-rack switch. It's cheaper, but ultimately it won't be as scalable. As more and more 10 GbE servers are installed into data centers, 10 GbE port density will become a challenge.

"Eventually there [will not be] enough [10 GbE] ports on aggregation and core switches to support racks and racks of 10 Gigabit switches," Weckel said. "Enterprises would have to add another layer of switches to support that, and those switches are very expensive.

In order to avoid adding another tier of switching to their data center networks, and also to reduce the amount of complexity involved in managing multiple 10 GbE uplinks from every server rack, enterprises will have to move to 40 GbE technology. That port density crunch is just around the corner because enterprises are poised for a tsunami of 10 GbE server adoption.

"If you look at the market today, less than 5% of servers are doing 10 Gigabit," Weckel said. "But you're guaranteed that 100% of servers by 2014 will be 10 Gigabit."

While the cost of 40 GbE technology is too great for most enterprise IT budgets today, prices will begin to drop when enterprises turn to 40 GbE switches in large numbers. For now only large hosting companies and Internet powerhouses like Facebook and Google consider 40 GbE worth the money.

The wild card in the cost of deploying 40 GbE technology is the optics, Weckel said. None of the vendors that have announced 40 GbE products have said what they will charge for the optical transceivers that enterprises will need to connect 40 GbE ports to fiber optic cables. Vendors are still calculating a price that enterprises will be able to stomach, and early adopters will find themselves paying a premium, he said.

Network management and security must keep pace with 40 Gigabit Ethernet speeds

The South Carolina Department of Probation, Parole and Pardon Services, hasn't yet deployed 10 GbE servers in its data center, so 40 GbE technology isn't yet on the radar, according to David O'Berry, director of IT systems.

Before O'Berry would even consider 40 GbE in his data center, he wants to see network management, monitoring and security technologies hit the market that can keep up with bandwidth that high.

"You have to make sure you're getting good visibility into what the communications look like," he said. "People say, 'Oh you need 40 gig.' Well what are you monitoring it with? They say it's just server-to-server so it should be fine. And I'm like, 'No.' I assume there are management tools coming down the pipe, but I don't think there is anything out there now, even on a consolidated 10 Gigabit link. It's not about speeds and feeds right now. It's about not crippling yourself when you get the rubber to the road."

O'Berry conceded that that when he does move to 10 GbE at the server edge, he might use 40 GbE to uplink into his core, but he said he's learned his lesson from the VMware adoption wave. The industry lauded the operational expense reductions enterprises could realize from server consolidation through virtualization, but vendors neglected to develop management and security controls to coincide with that consolidation. Enterprises have struggled to regain that control, which in turn has created a lot of additional overhead for IT organizations. O'Berry worries the same thing could happen if enterprises adopt 40 GbE technology without putting the right management and security technologies in place.

40 Gigabit Ethernet uplinks will be required on top of rack switches

Although the majority of enterprises aren't quite ready to dive into 40 GbE networks, 40 GbE uplink capabilities will soon become a standard requirement for enterprise shopping for top-of-rack switches, Weckel said. Most enterprises want to see that their data center switches will be capable of supporting 40 GbE when it's needed. The 40 GbE uplinks on Force10's S4810, for instance, are QSFP+ (Quad Small Form-factor Pluggable) ports which enterprises can connect to a splitter cable that converts each 40 GbE connection into four 10 GbE connections. Enterprises can use the uplinks at 10 GbE speeds today and convert them back to 40 GbE when they are ready for an upgrade.

"It's future-proofing, and [40 GbE] is going to become an important check box," he said.

Let us know what you think about the story; email:

Dig Deeper on Data Center Networking

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.