CenturionStudio.it - Fotolia
A massive fire near a branch office prompted engineering firm MWH Global to stop relying solely on MPLS, and transform much of its worldwide WAN into a fully orchestrated Internet-based infrastructure.
MWH does massive water projects all over the world, from recycling to preserving wetlands. To run the company it takes 180 remote sites, 8,000 users and the ability to turn up about three offices a month.
Until recently, the company relied on a global MPLS WAN with service from a Virtual Network Operator that was very reliable, but also expensive. Beyond cost, the service was also not agile enough for quick provisioning and de-provisioning of offices. It also couldn't support a growing number of applications MWH was using the cloud. Turning up new connections basically meant waiting on the carrier to drop lines.
"The MPLS cost-per-meg was beyond the cost tolerance of most local offices," said Eric Williams, MWH network services manager, who presented a case study at the Open Networking User Group conference in NYC last week.
Around 2012, the company began piloting broadband Internet links orchestrated by Glue Networks in about 14 sites. The WAN orchestration technology allowed automated, swift provisioning of broadband connections with total visibility.
It was during that pilot that a fire broke out in a suite adjacent to MWH"s San Diego office.
"We had to relocate the entire office and everybody in it, and most equipment had been damaged," said Williams.
To get the office back online, Williams turned to Glue. Glue immediately shipped a router, which was used to reestablish Internet connectivity and spin up VPN services. Employees returned to work remotely via VPN connections within 24 hours.
Glue"s technology worked so well that MWH decided when its VNP contract lapsed it would replace pieces of the existing network with a Cisco iWAN architecture for regional connectivity, using the Gluware Orchestration Engine for automation and management.
How MWH built an automated broadband WAN
Two years later, MWH runs Cisco's newly released ISR 4451-AX routers that enable secure VPN domain provisioning with intelligent path control. They also have an API for Glue, so MHW can automate deployment and management.
The overall architecture is based on Cisco iWAN Dynamic Multipoint VPN technology, which lets branch offices connect directly to each other over the Internet or public WAN with secure IPsec VPNs that are not static. Using DPVPN and intelligent path control, Cisco iWAN can load balance for best path routing over aggregated connections.
The VPLS network core is connected to four regional points of presence (POPs) in Chicago, Sydney, London and Mumbai with "copious bandwidth" and those connect to five data centers, said Williams.
"We interconnect those POPs via a high-speed core network and structure them in a way to have them act as universal integration points," Williams said. Meanwhile the core is still connected to an MPLS network.
The move to broadband for the branch means that each remote office "can get the bandwidth they need versus what they paid for," said Williams. It also lets MWH "dim the lights" on remote offices during downtime.
"Previously the average connectivity [on a remote site] was a T1 or an E1," said Williams. "Now the lowest [connectivity] is 10 megs and larger offices get 100 megs," said Williams. There is gigabit connectivity into the data centers and across the core.
To manage the Internet traffic locally, Williams" team implemented Cisco VRF to "break out Internet traffic" and also as a way to determine which traffic was contained in the network and which was backhauled for QoS reasons, for example. VRF lets routers maintain separate route tables for differing groups of interfaces that all share the same connection to the cloud.
The big concern about using best path routing over Internet is that there is no way for network engineers to guarantee the performance of the underlying infrastructure -- or to provide seamless connectivity between providers. That"s still the case for MWH.
"The underlying [infrastructure] is best effort," said Williams. The company set about selecting providers at the edge that would best peer, and there are about two in each POP. MWH also architected failover links out of the POPs.
So far, "the results are compelling," said Williams. Users are reporting increased performance for internal and cloud-based applications and the team can make a new site live as quickly as the team can obtain Internet connectivity, Williams said. All of the offices have redundant connections into the office and there is a fully redundant core.
Cisco iWAN is not yet full SD WAN
Williams refers to his iWAN infrastructure as SDN-capable WAN, but the ability to centrally control and provision the entire network through a Cisco controller won"t be available until the APIC-EM ships in the second quarter of 2015.
"If you look at where we are going with SDN WAN, it"s to bring in the policy component," said Jeff Reed, vice president of enterprise networking at Cisco. In an APIC-driven WAN, MWS or others would be able to apply application-aware policy to the entire system and with respect to best path selection.
The launch of the ISR 4000 routers was a first step to that end, Reed said. The platforms allow engineers to identify applications on the network in order to apply policy.
The controller will let engineers build the abstracted policy layer that can already be implemented in the ACI data center.
Lucera"s SDN WAN for high-frequency trading
SDN WAN: provisioning and high availability
SDN WAN startups: CloudGenix and vIPtela