Networking data visualization not just for pointy-headed bosses

By Michael Morisy, News Writer 20 Aug 2008 | SearchNetworking.com

News on networking, mobility and voice Digg This!     StumbleUpon     Del.icio.us

"If you have a huge amount of data, you need to get a feel for what's really in there," Marty said. "If you have to go through 100,000 log files, by the time you get to line 100, you've forgotten what's on line 1."

Visualizations, by contrast, let networking professionals skim large amounts of data at once and quickly home in on outliers or other hard-to-detect trends.

Once people start thinking about the problem differently, they quickly reap the benefits, Marty said. The most challenging learning curve is at the beginning as IT professionals become comfortable moving from numerical data to graphical representations in the form of curves and color.

Part of the problem has also been a lack of tools to dive into data visualization for the interested professional not quite ready to plunk down between $2,000 and $20,000 on specialized reporting and visualization software with vendors like netForensics or CrossTec. Marty said that cost or complexity should not be a barrier in getting at least some of the benefits of data visualization, and he's created an open source tool called AfterGlow and the website SecViz.org to help users get started.

Visual Security Analysis: Free Chapter Download Download a free chapter of Marty's book, courtesy of Addison-Wesley Professional.

So what is needed to begin getting useful work done with data visualization? As when approaching almost all networking problems, good logs are critical, Marty said.

"I think a good starting point is collecting the logs [in one place]," he said. "A lot of people don't keep the correct logs or keep them around long enough, depending on what their use cases are."

Getting good logging data to start with should not be a challenge: Firewalls, applications, and intrusion detection platforms all can or are generating voluminous data, and it is just a matter of organizing it in an easily accessible manner so that one set of logs can be properly correlated to another.

The next step is to develop a clear purpose.

"A lot of people say, 'I have these NetFlow logs, and I want to analyze them,' " Marty said. "Do you want to verify traffic against usage policy, though? Or look for attacks?" The better understanding networking pros have of what they are looking for, the more likely they are to find it, rather than just having some potentially interesting wallpaper, with little to show.

For inspiration, several SecViz users have posted their own graphs, used to detect everything from Worm attack patterns to their current IP table configuration.

Marty also suggests that networkers read his book or search online for tutorials to fit their needs.

"It's great grounds for exploring what's there, or even asking: 'I have this dataset; how do I go about analyzing it?' " he said.

To truly tap into visualization's power, however, some professionals will want to consider tapping into a variety of scripting languages that can help them pull data in a more automated, particular way than Excel's user-friendly but finite controls allow. Tapping into a framework like ChartDirector means that a bit more technical learning is required, but more precise graphs can be scripted to update themselves as new data comes in.

And once a networking pro has his charts cooked up, how best to use them?

Marty outlined three major use case areas:

• Discover and explore: Internally, sifting through thousands of log files is inefficient and, worse, it's easy for critical elements to be overlooked. A good graph can help spot trends and correlate them to other parts of the network, helping to spot and diagnose problems in one fell swoop.
• Communicate: Whether it's with the storage group on how their backups are tanking your network or your CIO on why the wireless controller upgrade is critical this budget cycle, charts can help quickly explain trends and demands to people who don't have a networking pro's specialized knowledge. Marty said one IT group had regularly sent another group log data under the assumption that they were self-explanatory. But the second IT group had no clue what they were looking at until they saw a chart.
• Strategize: Logs are great for tackling problem spots, but graphs take networkers up a level and let them see the trends, giving them a chance to look at the longer-range picture and decide how to survive not only the next 12 hours but the next 12 months.

Marty did have one warning for those getting ready to dive into visualization's benefits: garbage in, garbage out.

"One of the dangerous things is if you don't understand the log file itself, don't assume you'll understand the visualization of it or even generate a visualization that makes sense," he said. "If I have a firewall log file, and I have no idea about the IP addresses that are used or the role of internal machines, it gets very hard to analyze that."

Tags: Network Security Monitoring and Analysis,  Network Monitoring,  Network Administration,  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Network Security Monitoring and Analysis Network penetration testing guide Performing a penetration test Penetration testing strategies Penetration testing methodology and standards Types of penetration tests Network security forecast 2010: Startups cash out, uber-devices step up Mobile computing security concerns lead to more IPS, SSL VPN spending Application-specific network intrusion detection systems emerge Anomaly-based intrusion protection configuration and installation How can I calculate perimeter firewall throughput? Network Monitoring Matrix switches can eliminate network tap and SPAN port shortages University tackles large-scale 802.11n wireless network management Meru reinvents wireless LAN troubleshooting and management Green enterprise: Three networking investments that make a difference Network device management overload: Engineers managing too many boxes What preventative maintenance procedures for network devices exist? WLAN QoS and SLA monitoring with 7/24 Wireless Quality Assurance costs How important are network infrastructure maps for engineers or admins? Understand Windows tracert output to troubleshoot network connectivity Network management and monitoring market remains crowded, fragmented Network Monitoring Research Network Administration 2010 predictions: What IT trends will affect network administrators? How do I set the Web page default on a network of computers without setting them all individually? What IP enables file sharing among multiple computers on different Internet connections? Why is access denied to my Active Directory (AD) users and computers? What network loss testing tools/methods calculate dropped packets from a PC? Network user management Do I have to disable DHCP on my router to create a DHCP server? What preventative maintenance procedures for network devices exist? Top 10 reasons why computers do not have network access to each other Troubleshooting -- 'Network Know-How' Chapter 17 Network Administration Research

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary deep packet inspection (DPI)  (SearchNetworking.com) FCAPS  (SearchNetworking.com) Nessus  (SearchNetworking.com) netstat  (SearchNetworking.com) port mirroring  (SearchNetworking.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary