Carrier Ethernet essentials

In this guide, service providers will find helpful tips for delivering Ethernet as a service and gain an understanding of the key issues in Ethernet backbone deployments. Explore how service provider certification can give you the leg up in enterprise negotiations with the assurance that the services they're buying are compliant and interoperable.

Carrier Ethernet is on its way to becoming the service providers' technology of choice due to its combination of improved revenues, better cost management and opportunity to lead customers into the managed services space. Even better, enterprise customers have been working with Ethernet for years in their local area networks (LANs), and it holds great promise for delivering metro-area-based IPTV and video content delivery for the mass market, as well.

In this guide, service providers will find the essential information they need for delivering Carrier Ethernet as a service and gain an understanding of the key issues in backbone deployment. It also explores how service provider certification can give you a leg up in enterprise negotiations because potential customers will know the services they're buying are compliant and interoperable.

Ethernet as a service: Best practices for delivery

Ethernet services can offer providers a combination of improved revenues, better cost management, and an opportunity to lead customers into the managed services space. Achieving these goals will require careful planning, designing and deployment processes, as well as effective network and services management practices once customer services are actually deployed.

Ethernet as a service is distinct from Ethernet as the basis for a multi-service metro infrastructure, and this distinction is important for ensuring that best-efforts consumer Internet services and high-bandwidth video services (especially video on demand) do not create resource competition and performance/stability problems. On the other hand, consumer services may create considerable metro network economies of scale that, if properly exploited, will lower the cost base for Ethernet services to business customers.

The best approach to providing Ethernet as a service is to plan to create multiple Ethernet overlays on a common optical (probably WDM/DWDM) layer to achieve the greatest economies in basic cost per bit. At the minimum, Ethernet services to businesses should be separated from consumer services, and many providers would further separate consumer Internet from IPTV and video on demand.

An Ethernet services network must be designed and deployed in four logical layers:

  1. Access -- which provides not only for Ethernet access but if necessary also for Ethernet-over-DSL or Ethernet-over-SONET for small and large sites, respectively. Access strategies that cover the range of business locations expected in a service geography are important for ensuring that multipoint Ethernet services can be deployed to the full site population of target customers to improve the customer benefit case.
  2. Aggregation -- which concentrates traffic onto the metro core network for transport to points where Ethernet service switching can be performed economically.
  3. Switching/Service -- which provides the actual multipoint Ethernet service capabilities and also metro transport of traffic.
  4. Interconnect -- to allow for connection between sites in different metro areas, using Ethernet/fiber, IP/MPLS or SONET trunks.

The access layer of an Ethernet services network would consist of direct Ethernet connections (usually GigE) to larger customer sites, and connection to both the DSL and SONET networks to obtain connection to small sites or to very large sites that have multi-service SONET access. Care should be taken in designing the Ethernet component of this network to ensure adequate levels of redundancy; it is difficult to provide full redundancy in the access layer without increasing costs considerably, which will reduce both profit and service acceptance rates.

The purpose of aggregation is to combine traffic from multiple sources to the point where it is economical to transport it. For this reason, the aggregation layer of the network must be highly redundant in order to prevent single outages from affecting large numbers of customers. This can be achieved at the Ethernet level through enhanced topology management (RSTP, MSTP) and through the IEEE 802.3ad link aggregation protocol for distributing traffic on multiple trunks. Packet-over-SONET or resilient packet ring may also be effective in some applications, though generally more costly. 

Carrier Ethernet -- Chapter download

In this chapter, you'll find in-depth information on:  

-- Carrier Ethernet standardization

-- Defining Carrier Ethernet

-- Ethernet in the LAN vs.

-- Ethernet in service provider networks

-- Attributes of Carrier

-- Ethernet Carrier Ethernet service reliability and scalability

-- MEF certification program

-- The Ethernet Service Model Performance parameters

Download Chapter 2, Carrier Ethernet.

Excerpted with permission from the McGraw-Hill Companies.

The switching/service layer of an Ethernet service network creates the virtual LAN structure needed to connect the users' sites into networks. Since this layer and the aggregation layer may employ Ethernet switches, the boundaries between these layers may be blurred, particularly where traffic densities are high, such as in a major city. In that case, aggregation and switching/service may actually combine. In the switching/service layer, redundancy at the equipment level is very important since traffic concentrations are very high, but it is also necessary to provide the same level of connection redundancy that was present in the aggregation layer. In some networks, the switching/service layer may be created using IP technology, and this is particularly true where MPLS is used for metro interconnect. Where IP/MPLS is used, it may be advisable to create virtual LANs using IP; this will also facilitate the extension of the service to other cities.

The interconnect layer provides linkage between metro networks. Most providers use something other than Ethernet technology for these links: IP/MPLS, Packet-over-SONET, or even direct optical trunking (WDM/DWDM). The best interconnection option will depend on the amount of traffic moving out of the metro service area.

New standards are creating additional options in the critical aggregation and switching/service layers. The Metro Ethernet Forum's MEF 2 framework is based on the service level specification for the Ethernet offering and uses Extended Ethernet Protection Switching (EAPSv2), a superior option to RPR. Enhanced metro Ethernet features such as PBB and PBT may also be used to create virtual LANs and paths for business services. PBT, and an MPLS version with a similar multi-protocol goal (T-MPLS), can use the GMPLS control plane for superior route control and also improved operations stability.

For all carrier Ethernet services, it is good design practice to ensure that congestion is held very low, not only to improve the delay and loss characteristics of the service but also to ensure that there is spare capacity for fail-over. The "rule of 10," where each layer of switching or aggregation has a trunk capacity equal to ten times the port capacity, is a simple way to ensure that utilization levels remain low and services require minimal traffic engineering and management to maintain. (Written by Tom Nolle, President, CIMI Corporation)

Using Ethernet as carrier backbone transport

As network traffic has evolved from time-division multiplexed (TDM) to packet, providers have become more committed to backbone or "core" networks built on packet technology. In the 1990s, the growth of the Internet created a wave of interest in "convergence" on IP as the universal network technology. But IP networks have proved more costly than expected to operate, and in any event, packet protocols are multi-layered and IP is a Layer 3 protocol. Below IP, at the "data link layer" of the OSI model, are options like Packet over SONET (PoS) and Ethernet.

SONET technology has evolved since the days when OC-3 (155 Mbps) was considered fast, and now OC-768 (about 40 Gbps) is common. In fact, SONET standards have consistently kept ahead of Ethernet in terms of speed, and this has helped maintain SONET and PoS as preferred options. Recently, the combination of WDM/DWDM and faster Ethernet (10 Gbps Ethernet is now available, and 100 Gbps is being worked on by standards bodies) has raised interest in using Ethernet as the backbone in large packet networks.

Ethernet backbone applications can be broadly divided into interface and network applications. In the former, Ethernet is used simply as a point-to-point link layer protocol between devices, usually IP routers. In this application, the benefit of Ethernet lies in its lower cost relative to SONET. In network Ethernet applications, there is an actual Ethernet network built, over which IP and other higher-layer protocols travel.

Ethernet LAN technology is not suitable for network core deployment and, in fact, is unsuitable for carrier deployment in any application. The standards for LANs, particularly those relating to the size of Ethernet subnetworks and the bridging between subnets (spanning tree), are not scalable to carrier levels. The IEEE and Metro Ethernet Forum (MEF) have been working on a series of standards to create carrier-scalable Ethernet.

The key issues in Ethernet backbone deployment are maintaining scalability with very large Ethernet networks, providing QoS to applications that need it, and improving resiliency to ensure that failures in the core do not generate tens of thousands of customer complaints. How each of these is best accommodated depends on whether the provider intends to offer Ethernet services or simply use Ethernet as a path protocol under a service layer like IP.

Scalability in Ethernet backbone applications means better spanning tree. Rapid Spanning Tree Protocol (RSTP) offers better convergence in large Ethernet networks, and the Multiple Spanning Tree Protocol (MSTP) provides for VLAN-specific bridging needed for Ethernet services. Both may still create challenges, however, and current work on Provider Backbone Bridging with Traffic Engineering (PBB-TE), often called Provider Backbone Transport (PBT), eliminates spanning trees completely, allowing Ethernet to use a separate control plane such as GMPLS to create the bridging tables.

Where VLAN services are to be offered seamlessly from customer premises through the core network, most providers have determined that the "stacked VLAN" approach of IEEE 802.1ad will not support sufficient customers and flexible VLAN tag assignments to be commercially optimal. Instead, they opt for the 802.1ah approach (PBB). Some network architects believe that very large VLAN spaces are better served by using a hybrid of Ethernet in the metro network and an IP-based core. 

Bokmark "Uncommon Wisdom

Providing QoS at the network transport level, in today's thinking, is best accomplished through the PBB-TE/PBT mechanisms at the Ethernet level because PBT paths can be put into place and maintained statically. If Ethernet services are also to be offered, then it will be necessary to maintain the 802.1p class-of-services from VLANs and map them correctly to PBT trunks. Some vendors recommend that core VLAN interconnect be done via MPLS rather than through Ethernet, but PBT trunking should permit QoS control entirely within the Ethernet level. Still, when large scale and stringent QoS control are both requirements, an MPLS core may be a better approach.

Resiliency issues in Ethernet today are handled in the Metro Ethernet Forum's MEF 2 specification, and this is an excellent practices guide even for Ethernet backbone applications, particularly if Ethernet services are also offered directly from the backbone.

Another issue to be addressed in Ethernet backbone design is the relationship between Ethernet and other layers, and this is of particular interest in the area of managing availability and resiliency. If strong Ethernet path recovery exists, then higher-layer protocols like IP will not "see" failures, and recovery processes there can be less stringent. Similarly, if Ethernet is used over recoverable optical trunks, it is important that the Ethernet design accommodate the optical reconfiguration below (Reconfigurable Optical Add-Drop Multiplexers, or ROADMs) to ensure that there is no "hunting" of topology discovery created by multiple-layer fault recovery events.

All of these constraints must be considered within the context of a strong network and operations management plan. The MEF 2 specification addresses end-to-end Ethernet monitoring for OAM&P, and where PBT is used, the GMPLS or other control plane may be able to provide some detailed operations management and control links for a network management or OSS application. There is considerable standards work under way in Ethernet management, however, and it is advisable for backbone designers to check the state of the standards and the specific support offered for each by the vendors available before making a final vendor choice and finalizing their Ethernet backbone design. (Written by Tom Nolle, President, CIMI Corporation)

Carrier Ethernet certificiation

Carrier Ethernet services are seeing some serious traction among enterprise customers, and to make sure businesses feel comfortable buying this familiar technology from a telecom service provider, the Metro Ethernet Forum is offering certification not only for vendors, but for service providers, too. Service provider certification marks a revolutionary change in the industry. Since the program began in April 2005, more than 80% of the major carriers offering Carrier Ethernet in the U.S. have gone through the MEF's program designed to guarantee service functionality and performance. MEF-certified services help provide enterprises with guarantees on real-time applications like VoIP and videoconferencing.

The third phase of the MEF's Carrier Ethernet is now underway – educating enterprises around the world about the advantages of buying certified equipment and services.

In this Q&A, Bob Mandeville, president and founder of Iometrix talked to site editor Kate Gerwig about the purpose of Carrier Ethernet certification for service providers. Read the full Q&A on Carrier Ethernet certifications.

This was last published in November 2007

Dig Deeper on Telecommunication networking