Increasingly complex infrastructure demands and evolving changes in application delivery are revealing the fragile state of legacy data center networks. Fabrics offer one alternative for meeting the demands of the modern data center.
What is an Ethernet fabric?
An Ethernet fabric is a type of network topology that collapses traditional three-tier data center switching architectures into one or two tiers, ensures network traffic is never more than two hops away from its destination and operationally behaves like one big switch. The goal of an Ethernet fabric is to increase the scalability, performance and resilience of highly virtualized and cloud-ready data centers.
"It allows multiple conversations to happen across the fabric simultaneously but still maintains this expectation that all of the systems still look like they're adjacent to each other," said Eric Hanselman, chief analyst at 451 Research. "In pure networking terms, it's going to be entirely done in a Layer 2 environment."
Fabrics also support equal-cost multipath forwarding at Layer 2 and Layer 3 with "fine-grained, flow-based load balancing," according to Gartner analysts Caio Misticone and Evan Zeng in a research note, "Technology Overview for Ethernet Switching Fabric."
Vendors typically sell commercial Ethernet fabrics as a product suite comprising specialized hardware and software. Fabrics can be implemented using various architectures, the most common among them leaf-spine and mesh.
Several terms -- including data center fabric, switch fabric, network fabric and Ethernet fabric, or some amalgamation of all four -- are often used interchangeably to refer to this topology. This has caused some confusion in the market, as many of these terms have alternate meanings. While some experts favor switch fabric, the phrase may also refer to the switching mechanism in a physical switch. Data center fabric is sometimes used to refer to fabric architectures for converged data and storage networks. Network fabric is a broad term that is sometimes used to mean network architecture. Ethernet fabric is more precise than the first three, although it should be noted Brocade also uses Ethernet Fabric as a commercial product name. For the purposes of this primer, we will use the term Ethernet fabric to refer to flatter, highly scalable topologies that enable low latency via one- or two-hop connectivity in data center switching.
Why would you need an Ethernet fabric?
Applications -- and by extension, the networks that delivered them -- used to be a lot simpler. Application traffic primarily flowed north and south between three tiers of switching that connected servers to clients.
The widespread adoption of server virtualization, however, transformed these devices, enabling these once-static architectures to become virtual machines capable of exploiting a data center's entire capacity by moving traffic among multiple physical servers. Applications were also becoming more complex, with various functions being broken off into different systems and components that would need to communicate across servers.
This influx of east-west traffic -- that is, traffic between and among servers -- has strained traditional three-tier switching architectures and limited scalability, said Hanselman of 451 Research. Data would now have to pass through more hops to reach its destination, adding latency and consequently degrading performance.
Meanwhile, the performance and resilience of data center networks were further hamstrung by the pervasive use of Spanning Tree Protocol (STP), an algorithm that prevents bridge loops by shutting down redundant paths in favor of a single active link that transmits data. While STP is sufficient for conventional traffic flows and application architectures, it is an imperfect and fragile approach that uses bandwidth inefficiently.
Ethernet fabrics -- along with complementary technologies such as TRILL and Shortest Path Bridging (SPB) -- offer an alternative to the complexity and inefficiencies of three-tiered networks and Spanning Tree. An interconnected fabric combines the visibility of Layer 2 with the operational benefits of Layer 3.
"The idea is you can see everything," Hanselman said. "It simplifies the way an application needs to think about what it needs to do [in that] it doesn't have to figure out [the right path]," he said. "[Nodes] can simply all send off a packet into the fabric, and the fabric takes care of getting it to that person."
What are some of the limitations of an Ethernet fabric?
While a fabric offers many benefits, there is one major challenge that can be a deal-breaker for some network engineers: It almost always requires a single-vendor network.
"Fabrics give you the tools to operate much more flexibly, but you have to make a commitment to a particular vendor," Hanselman said. "The big question for most [enterprises] is about the willingness to commit. If you want to be able to keep multivendor networks, it's really hard to go with any fabric capabilities -- and for some folks, that's a big problem."
With a few exceptions, most vendors have created proprietary enhancements to standard protocols, such as TRILL and SPB, he said. This has rendered most vendors' fabrics incompatible with their competitors' infrastructure.
"The other piece to this is fabrics … are not infinite [in capacity], and there are scaling limitations," Hanselman said. "When you get into the multi-thousand-port range, you're starting to get to an area where the management headaches [are such that] you should consider segmenting."
Does everyone need an Ethernet fabric?
Fabrics were once seen as the decisive path to the next-generation data center. But the emergence of software-defined networking (SDN) and the growing adoption of cloud computing has given network engineers and data center pros more factors to consider. Although Ethernet fabrics and SDN aren't mutually exclusive, the former is more mature and currently has greater commercial availability.
More on Ethernet fabrics
Learn more about the data center fabric wars in the Network Evolution e-zine
Get one expert's take on why TRILL won't work for data center architecture
"People are looking for ways to simplify provisioning and simplify deployments. Some of them are doing it with relatively simple network architectures and automation tools, other folks are doing it with more radical approaches like SDN, and other folks are … saying the conventional Ethernet fabric approach is good enough and 'We don't care about the long term,'" said Joe Skorupa, vice president and distinguished analyst at Gartner. "And there are other folks saying Ethernet fabrics look transitional. The [question] is, how soon do you need [to upgrade]?"
For his part, Hanselman contends that fabrics are a must-have for large enterprises that are looking to remove the brittleness from their data center networks.
"Realistically today, everybody should be running fabric. There are so many good reasons to have the flexibility, the capacity and the ease of management," Hanselman said. "Questions start to come in on how you make that transition. If you're building a brand-new network today, the question is, 'Which fabric do you pick?'
"If you have an existing network and you want to be able to transition to fabric, the issue is operational," he added. "Networks that have been running Spanning Tree have always been so fragile that they are typically treated with kid gloves and babied into any extension or change."
Although some vendors have developed some stackable designs for the midmarket, many smaller companies don't have the scalability needs to justify the investment in fabrics.
"For some companies, a single switch is so bloody big you don't need a multiswitch network," Skorupa said. "If you can get several hundred 10-gig ports on a single chassis, for a lot of companies, that's big enough."
This was first published in November 2013