Networking data visualization not just for pointy-headed bosses

By Michael Morisy, News Writer 20 Aug 2008 | SearchNetworking.com

News on networking, mobility and voice Digg This!     StumbleUpon     Del.icio.us

"If you have a huge amount of data, you need to get a feel for what's really in there," Marty said. "If you have to go through 100,000 log files, by the time you get to line 100, you've forgotten what's on line 1."

Visualizations, by contrast, let networking professionals skim large amounts of data at once and quickly home in on outliers or other hard-to-detect trends.

Once people start thinking about the problem differently, they quickly reap the benefits, Marty said. The most challenging learning curve is at the beginning as IT professionals become comfortable moving from numerical data to graphical representations in the form of curves and color.

Part of the problem has also been a lack of tools to dive into data visualization for the interested professional not quite ready to plunk down between $2,000 and $20,000 on specialized reporting and visualization software with vendors like netForensics or CrossTec. Marty said that cost or complexity should not be a barrier in getting at least some of the benefits of data visualization, and he's created an open source tool called AfterGlow and the website SecViz.org to help users get started.

Visual Security Analysis: Free Chapter Download Download a free chapter of Marty's book, courtesy of Addison-Wesley Professional.

So what is needed to begin getting useful work done with data visualization? As when approaching almost all networking problems, good logs are critical, Marty said.

"I think a good starting point is collecting the logs [in one place]," he said. "A lot of people don't keep the correct logs or keep them around long enough, depending on what their use cases are."

Getting good logging data to start with should not be a challenge: Firewalls, applications, and intrusion detection platforms all can or are generating voluminous data, and it is just a matter of organizing it in an easily accessible manner so that one set of logs can be properly correlated to another.

The next step is to develop a clear purpose.

"A lot of people say, 'I have these NetFlow logs, and I want to analyze them,' " Marty said. "Do you want to verify traffic against usage policy, though? Or look for attacks?" The better understanding networking pros have of what they are looking for, the more likely they are to find it, rather than just having some potentially interesting wallpaper, with little to show.

For inspiration, several SecViz users have posted their own graphs, used to detect everything from Worm attack patterns to their current IP table configuration.

Marty also suggests that networkers read his book or search online for tutorials to fit their needs.

"It's great grounds for exploring what's there, or even asking: 'I have this dataset; how do I go about analyzing it?' " he said.

To truly tap into visualization's power, however, some professionals will want to consider tapping into a variety of scripting languages that can help them pull data in a more automated, particular way than Excel's user-friendly but finite controls allow. Tapping into a framework like ChartDirector means that a bit more technical learning is required, but more precise graphs can be scripted to update themselves as new data comes in.

And once a networking pro has his charts cooked up, how best to use them?

Marty outlined three major use case areas:

• Discover and explore: Internally, sifting through thousands of log files is inefficient and, worse, it's easy for critical elements to be overlooked. A good graph can help spot trends and correlate them to other parts of the network, helping to spot and diagnose problems in one fell swoop.
• Communicate: Whether it's with the storage group on how their backups are tanking your network or your CIO on why the wireless controller upgrade is critical this budget cycle, charts can help quickly explain trends and demands to people who don't have a networking pro's specialized knowledge. Marty said one IT group had regularly sent another group log data under the assumption that they were self-explanatory. But the second IT group had no clue what they were looking at until they saw a chart.
• Strategize: Logs are great for tackling problem spots, but graphs take networkers up a level and let them see the trends, giving them a chance to look at the longer-range picture and decide how to survive not only the next 12 hours but the next 12 months.

Marty did have one warning for those getting ready to dive into visualization's benefits: garbage in, garbage out.

"One of the dangerous things is if you don't understand the log file itself, don't assume you'll understand the visualization of it or even generate a visualization that makes sense," he said. "If I have a firewall log file, and I have no idea about the IP addresses that are used or the role of internal machines, it gets very hard to analyze that."

Tags: Network Security Monitoring and Analysis,  Network Monitoring,  Network Administration,  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Network Security Monitoring and Analysis Where can I find a sample security audit report? How can I run my own? The firewall remains the network traffic cop, but its role is changing Troubleshooting VLANs: How to monitor 802.1q tagged traffic Poor data-loss prevention practices almost cost Intel a billion How can I block my competitor's IP address range from my website? Hospital gains network visibility by convincing vendors to collaborate What software monitors and locks users from accessing my router? Data leak prevention starts with trusting your users NagVis -- 'Nagios: System and Network Monitoring, Second Edition,' Chapter 18 What is a genetic algorithm and where can I learn more about them online? Network Monitoring Understand Windows tracert output to troubleshoot network connectivity Network management and monitoring market remains crowded, fragmented When do applications suffer from poor network performance? Xangati help desk 'DVR' feature speeds up trouble ticketing resolution Network change and configuration management vendors see big changes YouTube, Facebook make bandwidth monitoring best practices challenging How a new casino manages a giant network with 500 switches, IP voice How network performance management can save money, boost applications Return-all-values script: Managing Windows networks using scripts, Part 13 HTTP error code troubleshooting, Part 2: How to use IIS tool WFetch Network Monitoring Research Network Administration How server virtualization improves efficiency in a client-server model Understand Windows tracert output to troubleshoot network connectivity Why would a computer show drive letters for discs that don't exist? Using tracert and TTL to troubleshoot network connectivity problems Open source software for enterprise network management and monitoring When do applications suffer from poor network performance? Tight times? Organize your networking group to stay above the stress Managing Network Problem Users: The make-it-so CEO Checking IP configuration to troubleshoot Windows network connectivity Bandwidth allocation: How can I give a download limit for each user? Network Administration Research

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary deep packet inspection (DPI)  (SearchNetworking.com) FCAPS  (SearchNetworking.com) Nessus  (SearchNetworking.com) netstat  (SearchNetworking.com) port mirroring  (SearchNetworking.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary