News Stay informed about the latest enterprise technology news and product updates.

New VMS feature enables big spines of Mellanox Ethernet ToR switches

A new Layer 3-based Virtual Modular Switch feature on top-of-rack Ethernet switches from Mellanox enables large clusters for leaf-spine architecture.

InfiniBand switching specialist Mellanox, which has been expanding its Ethernet switching business, has enabled a Virtual Modular Switch feature on its top-of-rack switches using standard routing protocols for interconnections.

The Mellanox Ethernet interconnect technology is a free software upgrade that allows data center operators to build leaf-spine architectures with a spine composed of top-of-rack Mellanox switches. Virtual Modular Switch (VMS) uses Equal Cost Multi-path Routing (ECMP) over Open Shortest Path First, or OSPF, for multi-pathing and load balancing among the top-of-rack switches in the spine layer. Arista Networks recently announced that customers could interconnect its 7500E modular switches with ECMP, too.

Mellanox also has added software plug-ins on its switches for Puppet and Chef, the open source DevOps system management tools. The plug-ins will allow users of these tools to manage their Mellanox Ethernet networks alongside the rest of their infrastructure.

In the past, networking vendors have offered various other methods for virtualizing multiple switches together. The most widespread approach is switch stacking, a method of daisy-chaining multiple switches together with a single pipe. This architecture introduces significant blocking, according to Amit Katz, senior director of product management at Mellanox. Vendors tout the manageability of stacking, because an administrator can manage the whole stack as a single switch, but there is a performance tradeoff since there's one linear backbone running through the stack, he said.

More on Mellanox Ethernet technology

With Open Ethernet, Mellanox customers can run open source network OS on its hardware

Mellanox tries to capitalize on push for converged I/O

Mellanox's acquisition of Voltaire combines two InfiniBand companies trying to break into Ethernet

In more recent years, vendors have pushed further with Layer 2 link aggregation technologies, like Cisco's Virtual PortChannel, or VPC; HP's Intelligent Resilient Framework, or IRF,; and various implementations of multi-link aggregation (MLAG). "This creates a bunch of switches behaving like a single switch, and since you're running link aggregation, the links will be active-active, rather than active-passive with spanning tree," Katz said. These Layer 2-based interconnections are limited in size, however. "Most vendors can only connect two switches (with MLAG). All these solutions are typically answering the need for a larger Layer 2 network, and they are typically implementing it with modular switches to enable two times the bandwidth of a modular switch," he said.

Mellanox's VMS is based on Layer 3 protocols that offer better hashing than Layer 2 approaches do, and "the distribution of traffic is well-separated between the different switches," thus allowing much larger groups of interconnected switches, Katz said. "What we are talking about here is in line with very large networks using Layer 3 standard technology, not vendor-specific features."

VMS is available on switches based on Mellanox's Switch X2 silicon. These top-of-rack switches feature 36x40 Gigabit Ethernet (GbE) port densities. Mellanox also can use the InfiniBand features on its switches to ramp up Ethernet bandwidth to what the company calls "56 GbE." At this higher bandwidth, engineers can build virtual modular switches with higher aggregate throughput but fewer switches.

Mellanox introduced new Puppet and Chef plug-ins to enable management and automation of this new VMS feature and other network management functions, including software upgrades, VLAN configurations, port operations and Open Shortest Path First configuration.

Mellanox Ethernet solutions strong but face stiff headwinds

"[Mellanox] has taken what they've done with InfiniBand fabric management and they are applying that to the Ethernet world," said Eric Hanselman, research director at New York-based 451 Research. "In InfiniBand, you always had to have active management capabilities that could do the carving up of all those interconnects. Bringing that capability into Ethernet allows you to do some interesting interconnection topologies with controls that give you flexibility in terms of how you manage the interconnect."

Although Mellanox is delivering some powerful technology with VMS, the company faces a tough market, Hanselman said. Mellanox is targeting companies that are transitioning from InfiniBand to Ethernet in data centers by offering mixed-mode switches. Only so many companies are interested in that transition, he said.

Mellanox is also trying to compete with data center fabric products like Juniper's QFabric, Brocade's VCS and Cisco's FabricPath.

"While they have a whole set of interesting capabilities for someone who wants to leverage those kinds of control capabilities for overall topology in the interconnect, the customer has to make a decision to move toward Mellanox as a single-vendor solution to leverage those capabilities, just as they have to do with any fabric," Hanselman said. "In this case, while Mellanox is a single-vendor decision that involves InfiniBand-based sophistication, the company does not have the breadth in its Ethernet portfolio that other vendors can offer."

Let us know what you think about the story; email Shamus McGillicuddy, News Director.

Next Steps

AI platform competition heats up as Mellanox is acquired by Nvidia

Dig Deeper on Network Infrastructure

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.