Deploying a wireless network can present many physical, technical and managerial challenges. Just ask Nicholas Doggett, Director of Network Architecture at the National Railroad Passenger Corporation (Amtrak). SearchNetworking.com asked Doggett how he was able to equip stations and moving trains with Wi-Fi after his case study presentation, "Improving the Quality of Rail Service and Mechanical Operations: Rolling Out Wireless Networks at Amtrak," at Mobile & Wireless World.
You mentioned that before you deployed your wireless network, your organization added new applications that created more bandwidth demand. How did you manage these applications or solve your bandwidth problem?
Nicholas Doggett: Most of the bandwidth issues we faced were a result of how the new applications were packaged and delivered, rather than the bandwidth demands of the applications themselves. By this I mean how the end workstations were configured. The use of Windows XP on a thin client, the methods used to lock down the workstations with server-based policies and permissions, user authentication and security, the configuration of remote desktop software (Citrix), as well as the ancillary system management functions such as print server operations, distributed systems backups and patch management -- all these layers of system software and architecture contributed to inefficient use of network resources that needed to be corrected by configuration changes rather than increased bandwidth. The legitimate growth in bandwidth demand arose from the expanded number of devices at each site that would lead to a higher concurrent utilization rate.
You said in your session that you devoted time to narrowing down an application to make it real-world applicable. Can you give some examples of this? What parts did you throw out or adjust to make it real-world applicable?
Doggett: It is important to look at the complete platform that is being delivered to meet a specific business need. Adapting an existing application -- and carrying it forward with its complete legacy platform -- may result in end-user application delivery issues. The platform design for a walk-up kiosk intended for use by many different users, each performing a discrete set of generally simple tasks, will be markedly different from a general-purpose computer used by a single user for a variety of complex functions. Taking a general-purpose platform and altering it for use as a kiosk is not easy. In addition to tailoring it for a discrete set of business functions, the remote management practices need to be re-engineered to meet a new set of support requirements and limitations. "Excess baggage" that stays with the kiosk platform is likely to cause performance, reliability and support problems that the end user may perceive as "network slowness."
Can you describe what your network looked like before the wireless rollout? Why did you need to refresh your network, and how did you prepare it for this wireless deployment?
Doggett: At each of the sites, network connections were added in piecemeal fashion over many years, without the benefit of an overall network design for the facility. As part of the refresh project, we took a look at what the network had grown into, and we developed a redesigned network that followed some basic design principles. This meant adding fiber where needed to meet the basic design principles of a fully redundant, three-tier Layer 3 network. We also added network and power management capabilities, as well as WAN diversity, where feasible, and WAN bandwidth increases as needed. I've heard that transportation companies have few problems with deploying Wi-Fi on their trains but, as you mentioned, they have difficulty connecting Wi-Fi from the train to the core network.
What problems did you run into that prevented you from doing this?
Doggett: The main problem is the lack of consistent availability of broadband-class backhaul. The most prevalent method of communicating off the trains is via wireless cellular, such as AT&T, Verizon or Sprint. National coverage is generally geared toward metropolitan areas and highways, whereas passenger trains often travel outside of these denser corridors. Also, we run through tunnels and other areas that are difficult to reach wirelessly without specially added infrastructure. And there is the added problem in the northeast corridor of ensuring reliable roaming or hand-off from one radio tower to the next when operating at speeds above 135 mph.
What do you think would have to happen in order to get Wi-Fi off the train?
Doggett: There are several approaches, each with poor economics from the point of view of the network service provider. Cellular coverage can be expanded, augmented with satellite, or replaced with a dedicated mesh infrastructure such as WiMax. For passenger trains, there are some challenges in using satellite owing to form factor and physical clearance limitations, and the cost of bandwidth remains quite high. The use of a mesh network will require substantial up-front investment in wayside radios (at least one every three miles of track) and a backhaul infrastructure to support them. Since Amtrak operates many of its trains on freight lines, we would need to look to the host railroad to enable the required infrastructure. Expanding cellular coverage to sparsely populated areas that passenger trains traverse is not a favorable investment for the wireless carrier, since there are few trains per day in these areas.
To roll out your wireless deployment, you certified places that were wireless-ready. How did you go about this process, and why would (or wouldn't) you recommend doing it that way?
Doggett: Ideally, we would perform spectrum analysis at each site post-implementation to reconfirm that the as-built system met the expected coverage projected from the original wireless survey. I chose not to do this level of certification, since the coverage can vary in the maintenance facility because of the frequent movement of train equipment and because the actual usage of the wireless system by the mechanical workforce will change over time. The approach we took was to have the application delivery team conduct a walk-about at the facility to gauge the end-to-end application performance in the various work areas and to troubleshoot performance problems where they were detected. Resolution of these site-specific problems may be as simple as improved workstation antenna placement, or the problems may be unrelated to the wireless network. However, I expect that, over time, weak coverage areas will be plugged with the addition of access points to the new infrastructure. This will be driven by the actual users of the system as they become more mobile and flexible in their work habits.
You mentioned that you used lawyers to negotiate contracts with wireless vendors, but that using them lengthened the project a great deal. If you had to do your wireless deployment over again, would you use them? What did they bring to the table, and what could you have done without?
Doggett: My advice is to try not to underestimate the amount of time it takes to negotiate a contract, and to build into the project strategies to mitigate the risk of unexpected delays, among other things. The negotiation of terms and conditions is extremely important and definitely protects the buyer of services from substantial potential losses. When contracting with multiple service providers at multiple locations, it is very possible that contractual issues may arise at any given time and place. Having solid contracts in place greatly facilitates termination and redirection, should that become necessary.
One of the major elements in your overall project plan was the work breakdown structure (WBS). Did you draft this before your deployment? What information did you use to draft this plan, and how might other enterprises gather practices for their own WBS?
Doggett: The WBS follows generally accepted network engineering practices. This particular WBS was developed during the planning stages of this project and was drawn from several sources, including my personal experience, the network planning methodology espoused by Cisco Systems Inc. as part of the CCDA certification process, as well as several vendor-specific (proprietary) network engineering methodologies I have been exposed to. Care was taken to ensure that no proprietary methodological information was reflected in the material presented.
When you ended your presentation you said that if you were to do this project over again you would have sent the people who survey the errors "earlier and more often." Would the cost of having these surveyors offset the cost of any mistakes you might have made?
Doggett: The network planning, design and implementation process is similar to other engineering projects in that the sooner in the process an error is detected and corrected, the less expensive it is to fix. In this particular project, some of the site survey and conceptual design work at a few locations was completed before all the requirements were fully documented. More on-site follow-up of the requirements would have been worthwhile because the errors in coverage were fully realized only after the installation work was completed. Bringing a contractor (and its equipment) back on-site after the job is complete is significantly more expensive and time-consuming than having the additional work included in the original statement of work.
Is there anything else you wish you had done differently, or do you have any other recommendations on wireless deployment?
Doggett: The technical solution must be responsive to the business drivers, and the execution of the project to deliver the technology must be as timely and cost-effective as possible. In addition, it is imperative that the project team communicate clearly and constantly with the project stakeholders. I strongly recommend that the project manager and team frequently review their communication plan and strive continually to improve it. This is the best way to gain support in the field, garner feedback, and detect and resolve errors early in the deployment. As mentioned previously, there is no substitute for having boots on the ground -- early and often.