Best practices for optical network design

Fiber-optic technology -- not long ago used only in long-haul networks -- has become the transmission medium of choice not only in the core, but in metro and access networks. The game-changer is global growth in consumer broadband and the need to distribute enormous amounts of content without hauling it halfway around the world. This Telecom Insights guide to best practices for optical network design looks at access, metro and core network issues affecting fiber deployment.

Fiber-optic technology -- not long ago used only in long-haul networks -- has become the transmission medium of...

choice not only in the core, but in metro and access networks. The game-changer is global growth in consumer broadband and the need to distribute enormous amounts of content without hauling it halfway around the world.

This Telecom Insights guide to best practices for optical network design looks at access, metro and core network issues affecting fiber deployment including:

  • Recent developments in FTTX ("fiber to the whatever") deployment and passive optical networking (PON) technology in access networks
  • Factors affecting metro network use of SONET, WDM or Ethernet, and the services that will be supported by fiber deployment
  • Specific core optical network design considerations including aggregation, geography and reconfigurability.

In this series:

  Fiber-optic networks: Access network design
  Optical networks: Metro network design best practices
  Optical networks: Core network design best practices

Fiber-optic networks: Access network design
by Tom Nolle

The rapid growth in consumer broadband seen worldwide today would not be possible without a major shift in the practices for provisioning access infrastructure. Copper loop and CATV cable were once the only means of transporting information from a provider central office or head end to the customer. Today, both these media are being "shortened" or even eliminated by the use of fiber optics.

Fiber is not a new development in access networks. Not only has it been used for almost two decades in the provisioning of high-speed commercial/enterprise customers, service providers in the 1990s found that replacing large bundles of copper by a few fiber strands could improve service reliability and lower craft cost. BellSouth took the lead in deployment of access fiber in that period, and the move was justified completely on cost savings.

The traditional access fiber architecture has been the fiber remote, which is a high-speed fiber trunk (SONET or Ethernet) that terminates in an electro-optical multiplexer. In analog phone days, these were called "digital loop carriers" (DLCs), and the term "new generation DLC" was used for a time, but most such devices today deliver DSL services and so are usually called "remote DSLAMs." A remote DSLAM's primary benefit is to shorten the access copper to allow higher DSL speeds and improve reliability. Most providers would counsel against offering premium DSL on loops over 8,000 feet, and the highest DSL speeds may be achievable only on loops 1,000 feet or less in length.

Pushing fiber close to the customer is generically called "deep fiber," and various acronyms are used to indicate just how deep the fiber is. FTTH means "fiber to the home," which is the extreme of giving every user an optical-electrical termination. FTTC takes "fiber to the curb," serving a group of homes, while FTTN means "fiber to the node" or "neighborhood," and allows each fiber remote to serve a larger population.

The problem with all deep fiber strategies, and the reason why providers don't simply run fiber to every home, is cost. If loops are kept to a length of 5,000 feet, a single remote can serve customers in an area of almost 2,000 acres. Shorten the loop to 1,000 feet and it serves only a little over 70 acres. Since the user population is generally proportional to the geography, this reduction means the cost per user could rise 50 times or more. Shorter loops mean higher speeds, however, and for video over IP, most operators would require at least 24 Mbps (ADSL2) connections. In Asia and some other areas, VDSL is used with speeds of 50 Mbps or more. Both these require much shorter loops (8,000 feet is optimum for ADSL2, according to reports, and 500-800 feet for 50+Mbps VDSL).

Balancing cost and performance is the goal of the various passive optical networking (PON) systems. PON creates a "tree" structure of fiber connections using optical splices without electrical termination or handling. PON typically supports 32 branches, and each of these can in theory be a remote or a home. A single PON tree supporting 32 branches has 33 electrical devices, counting the head end. Serving 32 locations with point-to-point fiber would require 64 electrical devices and generate higher costs and greater reliability risk.

PON systems use a common fiber architecture but a variety of opto-electric approaches. The original broadband PON (BPON) and the successor Gigabit PON (GPON) are both based on ATM. The new Ethernet PON (EPON) standard has been ratified, and most operators contemplating major new PON deployments are conducting assessments and procurements of EPON. GPON and EPON have sufficient capacity for video delivery and high-speed Internet. Some providers like the ATM framework of GPON for its ability to create multiple independent service channels to the user via virtual circuits. Others prefer EPON because it matches better with Ethernet-based metro architectures.

Planning for access network fiber deployment demands a careful consideration of the following:

  1. The demographics of the area to be served, including household income, family size, and age distribution. This data is critical in establishing the service market opportunity. In general, favorable demographics justify deeper fiber deployment.

  2. The geography and topology of the service area, including the household density (average lot size), the rights of way available, and whether cabling is underground or above ground. This data is critical to set the cost points for each approach. Obviously, poor characteristics here will create profit margin challenges if not taken into account. Studies in Japan, where fiber deployment is high, indicate that even whether the ground is flat or hilly has an impact on deployment cost.

  3. The service mix to be provided, over at least a five-year period, considering both trends in demand and in competition. The worst possible outcome in an access fiber deployment is a new set of requirements that the fiber architecture deployed cannot effectively support.

In the installation and maintenance phase, access networks present special problems because of the high cost of rolling a truck to fix a problem. A broadband consumer may require three years to pay back the cost of a single service call. This means that it is absolutely critical that each fiber strand be properly installed and that, in particular, the splicing used in PON installations be carefully done and verified. Fiber should also be tested end-to-end prior to committing it to customers. Unlike copper, whose problems tend to develop over time, operators report that most fiber problems are uncovered shortly after installation and result from improper practices.

Optical networks: Metro network design best practices by Tom Nolle

The trends in telecommunications today show clearly that the largest incremental amount of fiber deployed in the next decade will not be in the network core but in the access network and metro network. Content, the fuel of consumer broadband traffic growth, is an application that delivers a relatively small number of movies or programs to a large population of users. In most cases, this means that content will be cached at a metro level and that the greatest traffic growth created by content will be in the metro area.

Metro fiber today is based largely on SONET, which is 1310nm single-wavelength deployment. SONET networks are usually constructed as a series of protected rings that allow fast failover to the alternate "rotation" in the event of a fiber cut. Rings are connected via optical add/drop multiplexers (ADMs).

The advent of wavelength division multiplexing (WDM) -- coarse or dense -- deployed in the 1550nm range has added versatility to metro optics by providing multiple lightpaths per fiber and greatly increasing the capacity of a given fiber strand. At the same time, the increased volume of packet traffic, which does not require SONET's synchronous delivery behavior, has changed the traffic profile for the metro network of the future. Today, Ethernet is more likely to be the planned electrical layer of metro networks, and WDM the optical. This shift is changing the balance of tasks between electrical and optical components and the best practices for deployment.

SONET rings can be replicated in metro Ethernet and dense wavelength division multiplexing (DWDM) networks by simply using the same fiber and relying on wavelength separation or by running multiple SONET paths over WDM. Since there are probably no major metro networks worldwide without any traditional synchronous TDM traffic, planners should expect to use a hybrid of SONET and Ethernet technology. Where there is a large installed base of SONET equipment, no plans to eliminate PSTN switches, and major customers with direct SONET access, it may be advisable to plan a transition in the metro optical network from SONET-over-1310 to SONET/WDM and then to begin to integrate Ethernet-over-SONET, finally moving portions of the network to Ethernet-over-WDM.

The changing economics of WDM appear to be defusing the "SONET replacement" issue. Most operators now expect to maintain SONET for PSTN transport for as long as those services are offered, moving to non-SONET architectures only as packet voice displaces TDM voice. However, the gradual evolution is most likely to be compromised by exploding consumer broadband use, particularly by IPTV plans. Operators report voice traffic is stable while data traffic is growing at often triple-digit rates. The faster packet traffic grows relative to "circuit" or TDM traffic, the more likely it is that hybrids of SONET and Ethernet (Ethernet over SONET) will have too small a window of value to justify investment. This is probably the reason why more and more optical vendors are offering hybrid products with reconfigurable add-drop multiplexing (ROADM) and Ethernet.

A WDM issue that receives considerable media play is the way in which transit optical connections are handled. Most products have converted between optical and electrical (O-E-O) to perform a wavelength transit connection because pure optical cross-connect (O-O-O) has been expensive. While O-O-O products have been available for five years or so, most vendors still use O-E-O technology.

The primary issue today is still cost; service providers believe that future ROADM products will provide all-optical transit connections. The key issue for designers, regardless of the mechanism used, is that wavelengths can be "transcoded" to a different wavelength across the switch; a system that fails to provide this is too complex to manage because wavelength assignment on various fibers becomes interdependent, and some reconfiguration modes may not be available because of collisions.

Metro optical deployment is affected by the service mix to be supported, but the service topology has an equal or greater impact. A primary question to be addressed is the amount of intra-metro traffic to be carried relative to the volume of traffic that will simply be connected to a metroPOP for transport outside the metro area.

In areas where consumer broadband traffic makes up the bulk of total traffic, most fiber deployment will focus on linking serving offices to a POP for core network interconnect. While these connections have to provide resiliency, they will rarely require the SONET standard of failover, 50ms, since they will support Ethernet traffic. The introduction of IPTV may change this picture because loss of connectivity will cause pixelization that may produce viewer dissatisfaction, particularly on pay-per-view systems. If user buffering is available, the failover time should be no more than about two-thirds of the buffer interval. Often, consumer broadband failover is best accomplished at the Ethernet level.

Where there is significant synchronous (TDM) traffic and significant corporate packet traffic, it may be necessary to provide optical failover at SONET 50ms levels, in which case SONET or resilient packet ring (RPR) may be required. As noted, WDM may allow metro optical designers to separate traffic according to optical failover requirements and provide improved failover only where needed.

Reconfigurability, meaning the ability to create variable metro optical topologies by interconnecting wavelengths in various ways, is most likely to be needed either to accommodate a large amount of business traffic (metro Ethernet services) or to support alternate routing between serving offices and metroPOPs where the core network connection is made. Where IPTV is delivered, this multi-homing may also be needed for content service points.

At the optical layer, reconfigurability and fast failover are very different things. ROADMs offer a great amount of topology flexibility to adapt to changes in traffic demands, to the point where wavelength services can be offered to metro customers and where even Gigabit Ethernet customers can be quickly accommodated. Adding rapid and multiple spanning trees to Ethernet can provide resiliency at the electrical layer for everything but the most stringent failover requirements.

Many believe that metro optics will, over time, migrate away from the 50ms failover standard of SONET as circuit-switched and TDM traffic become a smaller portion of network load. If this is true, then a pure ROADM-and-Ethernet solution, particularly one based on one-box optical/electrical approaches, may be the best long-term solution.

Optical networks: Core network design best practices
by Tom Nolle

The "core" of a network is a place where aggregated traffic moves among on- and off-ramps. Because the traffic is highly aggregated and thus represents thousands or millions of user relationships, core network nodes are likely to have traffic destined for virtually all other core nodes, meaning that the nodes are highly interconnected. This may contrast sharply with metro networks, where "preferred topologies" often involve simply connecting serving or edge offices with points of presence (POPs) for connection to the core -- a star topology instead of a mesh.

The aggregation of traffic onto the core creates another difference in core optical networks: It is unlikely that a single network user will contribute a large portion of total traffic, and thus adding new users will generally not change the core significantly. In contrast, a large metro user may require reconfiguration of network bandwidth to accommodate the traffic. For this reason, and because of the "mesh" factor above, reconfigurability is likely to be less of an issue in core networks.

Core networks are also typically immune from the need for fast failover, the 50 ms optical alternate routing available with SONET. Ring configurations using fully redundant fiber paths are harder to create and more expensive to maintain in core networks, and so resilience is typically left to the electrical layer.

The final difference is that of geographic scope. A large metro network might span 50 to 100 km; a large core optical network can circle the globe. This long reach necessarily means that core fiber may have to span great distances without intermediate repeaters, including submarine environments, deserts, etc. Thus, ultra-long-haul fiber technology is often critical in core networks. The greater geographic scope of the core network also means getting craft personnel to an area to fix a problem may require days or weeks, and so it is critical to have some form of backup plan and to reduce outages as much as possible through design.

One issue that core and metro networks share is the issue of synchronous or circuit-switched traffic. Where PSTN calls and T1/E1 lines are to be supported over the core, it will likely be essential to utilize SONET/SDH transport for at least some of the optical paths to provide for synchronous end-to-end delivery. SONET/SDH services over global distances also require very accurate clocking to insure that bit errors are not created through "clock slips." These SONET/SDH trunks can either use the standard 1310 nm wavelength or one of the 1550 nm WDM wavelengths. Packet traffic does not require SONET/SDH, but many core network operators continue to use some SONET/SDH ADMs and switching in the core to preserve the option of circuit switched services.

A "pure packet core" can be made up of single-wavelength or WDM fiber, and thus it may be possible to create a virtual optical topology that approaches a mesh to avoid electrical handling. However, routers often have "adjacency problems" when installed in a full physical mesh, creating very long convergence times in the case of a failure. This, combined with the fact that reconfigurability in the core is often not a major requirement, means that core networks are more likely to use very high-speed fiber paths (OC-768, or 40 Gbps, for example) if the economies of these single electrical interfaces are better than the sum of the cost of WDM and a larger number of slower interfaces (4x10Gbps Ethernet).

The router adjacency issue is an example of an important point in core fiber design, which is that the needs of the electrical layer and even the service goals must be considered. Current trends in service provider Ethernet, spearheaded by work in the IEEE and the Metro Ethernet Forum are making Ethernet a strong candidate for core network deployment both to provide flexible virtual routes for higher-layer protocols like IP and to serve as the basis for actual customer services. This approach allows operators to create meshed optical networks for resiliency and add packet routing and even multicasting without creating additional router adjacencies.

A major consideration in optical core networking is the location of major points of service interconnection. The larger a provider core network is in terms of geographic scope, the more likely it will interconnect with networks of other providers, especially for local access in other geographies. These interconnection points are obviously both major traffic points requiring special capacity planning and points of major vulnerability. No interconnection point with another operator should be single-homed in fiber connection, nor of course should metro connections with the core provider's own metro infrastructure be single-homed.

The final point in core optical design is the management framework. Core networks carry aggregated traffic from millions of users, and failures will result in a flood of customer complaints. In addition, optical failures will trigger an avalanche of faults at the higher protocol layers, generating so many alerts that the network operations personnel may be overwhelmed. Many operators have insufficient integration between packet and optical layer management, and this increases vulnerability to alert storms and also makes customer support personnel less likely to have ready answers to complaints. The best optical core is no better than the operator's ability to manage it properly.

Tom Nolle is president of CIMI Corporation, a strategic consulting firm specializing in telecommunications and data communications since 1982. He is a member of the IEEE, ACM, Telemanagement Forum, and the IPsphere Forum, and the publisher of Netwatcher, a journal in advanced telecommunications strategy issues. Tom is actively involved in LAN, MAN and WAN issues for both enterprises and service providers and also provides technical consultation to equipment vendors on standards, markets and emerging technologies. Check out his SearchTelecom networking blog Uncommon Wisdom

Oprical networking is the only relevant Layer 1 technology today in the network's core except in very unusual markets or geographic conditions where terrestrial microwave may still be deployed. While core optical network deployments in some areas may be literally indistinguishable from metro fiber, there are other networks where the key requirements are totally different. Thus, the first question in optical network design and deployment for core networks is the nature of the network itself, and how core and metro requirements might differ.
This was last published in November 2008

Dig Deeper on Telecommunication networking