In 1982, San Francisco 49er quarterback Joe Montana hit Dwight Clark in the end zone with seconds to play to propel the 49ers into Super Bowl XVI. That play has gone down in National Football League history as "the catch."
Who was the major winner as a result of "the catch?" Well, certainly you can make a case for the 49ers, but others will make the case for media advertisers. As a result of the last second heroics, Super Bowl XVI was the most watched Super Bowl of all-time.
Behind the numbers for Super Bowls and virtually every other major media rating is New York-based Nielsen Media Group. Sales teams within media outlets use the numbers supplied by Nielsen to set advertising rates.
"Ratings are a very serious business," says Nielsen's Vice President of Technology Strategy, Marty LeFebvre. "Television ratings drive a $60 billion a year industry between local and national sales. The data has to be right -- no excuses."
To get the data, Nielsen has a number of collection methods including phone interviews, written weekly diaries and the most prevalent -- metered televisions. Each night from 3 a.m. to 7 a.m. 40,000 meters attached to TVs act as modems and call a central processing data center (this uses a 1-800 number, and can be migrated in a DR scenario). The roughly 50-second "call" downloads all of the data captured by the meter each day (about 75MB daily) -- including when the TV was on, what channel it was tuned to and some demographic information.
Then the real work begins.
Before the proliferation of cable, VCR/DVDs, satellite TV, TiVO and the Internet -- capturing rating information was relatively simple. There were three major networks and not a great deal of entertainment options. For storing this data, LeFebvre relied on a direct-attached storage (DAS) environment. But as options grew, there was no way DAS was going to cut it.
"We had no single view into our storage. Not only did no single person know how much storage we had, but no single person knew what kind of storage we had," said LeFebvre.
So the decision was to turn to a storage area network (SAN). LeFebvre says there were six driving factors for his SAN implementation: one, centralize a team of storage employees; two, a single point of control (policy and procedure); three, large volume purchases to save money; four, single operational focus that moved the storage to the center of the network; five, high availability and performance; and six, more informed planning procedures.
Now, the data is sent to a data center in Dunedin, FL, and is initially met by a Vax/Alpha cluster. From there the data is uploaded to a mainframe and then parsed out to a SAN powered by two Brocade 12000 core switches and redundant 3800s on the edge. Arrays from a variety of vendors such as Sun and HDS help collect the data. For backup, Nielsen uses two StorageTek Powder Horn automated tape libraries. Within those two ATL's are 30 fiber-attached tape drives, also from STK. The SAN is where the real data analysis occurs.
The major data set that needs to be processed is the "overnight" rating numbers. The "overnights" let media companies get an early read on how a program did the day before. For the East Coast preliminary "overnights" need to be made available by 7 a.m., for the West coast by 11 a.m. and by 3 p.m., all of the data captured that morning needs to be finalized.
The SAN now makes it possible for customers of Nielsen to get the numbers by phone, through a private network or by a Web-based interface. In order to get the demographics the media company wants, Nielsen allows for almost any conceivable permutation of the data (for example, men between 18 and 34-years-old, watching TV at 6 p.m. on the West Coast).
Aside from the customer-facing SAN, LeFebvre and his dedicated storage team is running three additional SAN fabrics. One is used for desktops, one for running applications and a development SAN. The development SAN also doubles as an interoperability lab where the IT staff tests different vendor configurations.
Now with the SANs, it's very clear how much data is coming in and flowing out of Nielsen, and the numbers are staggering. Each month, Nielsen backs up 110 terabytes (TB) of a data to tape -- up from 26TB a month in 2000. LeFebvre expects to see roughly 60% data growth rates compounded yearly.
He says this type of storage management cannot be done without a dedicated storage staff. "What you need is people that do storage all day long. They go to bed thinking about storage and wake up thinking about storage. You also have to use the storage at the right place and at the right time."
Sort of like where Dwight Clark was when Joe Montana threw that touchdown pass to him -- the right place -- at the right time.
For more information on Nielsen Media Group visit its Web site.