Following Sept. 11, when it seemed that the media was looking for people to canonize for sainthood, I recall an off-hand comment made to me by a Port Authority employee: "Be sure to look at who was awarded the contract for the dump trucks used to haul away the WTC rubble. The guy who got that job -- and believe me, it was a big dollar contract -- was a certain politician's relative. This is New York, after all."
I didn't bother to follow up the lead at the time. Reflecting on it now, I have come to suspect that a signature trait of democracy and capitalism is that even the worst disaster creates opportunities and markets.
In South Florida, the devastation of Hurricane Andrew was followed by a sudden increase in the area in the prices for milk, bottled water and other foodstuffs. This price gouging and profit-taking was not nearly as sinister, however, as the activities of a gang of unlicensed contractors who descended on the area, took money from victims of the storm to fix roofs and do other repairs to damaged homes and businesses, then blew town before pounding a single nail with a single hammer.
While there is certainly much to be lauded in the post-disaster efforts of emergency management personnel and others involved in the heroic efforts to restore businesses and human lives to a manageable state, there are unfortunately also as many events that underscore the baser characteristics of humanity. Disasters bring out the best and the worst in people –- often well after the cessation of the event itself.
In the wake of the Sept. 11 disaster, we have seen a flood of disaster recovery planning or business continuity planning-oriented products and services in the market. I have personally received offers to endorse several "canned plans" -- fill-in-the-blank planning templates intended to aid novice planners in building their plan document without the expense or effort of figuring it out for themselves. Needless to say, I passed on all of them.
Not only were the dollar amounts of the endorsement offers well below what, say, Nike offered to Michael Jordan, I also took issue with the underlying notion of a one-size-fits-most DR planning template. The hassle of developing a plan is the purpose of planning itself: It helps you define what you've got so you can consider options for sustaining or replacing it in the wake of an event. It's a mental exercise that, in the final analysis, teaches you to respond rationally in the face of the great irrationality of a disaster.
The time and expense of planning is a big issue for many companies, especially now when staff and budget resources are in short supply. As a veteran of some 68 planning efforts and author of four books on the subject, I am frequently approached by folks looking for a recipe for a disaster recovery planning process that won't break the bank -- a kind of DRP lite.
Of course, there is no such thing as DRP lite. But, I understand and sympathize with the motives of those who are asking the question. So, here is the best guidance I can offer.
Generically, there are about nine parts to a full-fledged DR planning effort. The first step is project initiation, which involves obtaining management buy-in, setting up a planning team, etc. Next is risk analysis, which involves undertaking a process to identify what you have, how critical it is, what its exposures are, what an interruption will cost you, how long you could sustain an outage, and gathering info on what others are doing to protect their environments.
The third component of DR planning involves the deployment of disaster prevention capabilities: Everything from smoke alarms and sprinkler systems to facility, network and application security systems to systems, network and storage management systems. Then, you need to develop a data protection strategy. Since data cannot be replaced, it must be protected through some sort of redundancy.
The balance of planning involves, first, the development of strategies and logistics for application, network and user work area recovery or replacement. Then, the ideal DR plan includes the development of training and testing regimes, and a change management plan.
If you have limited funds, should you spread your dollars thinly across all of these activities or just focus on a few? That isn't an easy question, but common sense provides some clarity. In any organization, the two most irreplaceable assets are people and data. Skilled workers are protected through the judicious application of disaster prevention capabilities, while data protection entails the use of a backup/restore or mirroring strategy. So, I would encourage you to focus on these, but only after you have done a risk assessment. Without risk analysis, you don't know which applications or data are critical so you can target these assets for protection first.
This is not to minimize the importance of logistics planning for recovery of other infrastructure components or the need for training, testing and change management. Fact is, however, that most of the well-defined and well-rehearsed system/network/work area recovery strategies tend to be abandoned as a matter of expediency in an actual disaster. Just as all the work you invest in mapping out every second of your next Mexican vacation may prove irrelevant when you inadvertently drink some tap water at Cozumel, the best laid recovery plans often fall prey to unforeseen events.
As you move ahead, consider making disaster recovery part of all other activities from application design to hosting platform architecture and even middleware selection in n-tier client server systems. The more that disaster recovery considerations are built into your applications and infrastructure upfront, the easier and less expensive your job will be to protect these assets after the fact.
About the author: Jon William Toigo has authored hundreds of articles on storage and technology along with his monthly SearchStorage.com "Toigo's Take on Storage" expert column and backup/recovery feature. He is also a frequent site contributor on the subjects of storage management, disaster recovery and enterprise storage. Toigo has authored a number of storage books, including Disaster recovery planning: Preparing for the unthinkable, 3/e. For detailed information on the nine parts of a full-fledged DR plan, see Jon's web site at www.drplanning.org/phases.html.
