Networking data visualization not just for pointy-headed bosses

By Michael Morisy, News Writer 20 Aug 2008 | SearchNetworking.com

News on networking, mobility and voice Digg This!     StumbleUpon     Del.icio.us

"If you have a huge amount of data, you need to get a feel for what's really in there," Marty said. "If you have to go through 100,000 log files, by the time you get to line 100, you've forgotten what's on line 1."

Visualizations, by contrast, let networking professionals skim large amounts of data at once and quickly home in on outliers or other hard-to-detect trends.

Once people start thinking about the problem differently, they quickly reap the benefits, Marty said. The most challenging learning curve is at the beginning as IT professionals become comfortable moving from numerical data to graphical representations in the form of curves and color.

Part of the problem has also been a lack of tools to dive into data visualization for the interested professional not quite ready to plunk down between $2,000 and $20,000 on specialized reporting and visualization software with vendors like netForensics or CrossTec. Marty said that cost or complexity should not be a barrier in getting at least some of the benefits of data visualization, and he's created an open source tool called AfterGlow and the website SecViz.org to help users get started.

Visual Security Analysis: Free Chapter Download Download a free chapter of Marty's book, courtesy of Addison-Wesley Professional.

So what is needed to begin getting useful work done with data visualization? As when approaching almost all networking problems, good logs are critical, Marty said.

"I think a good starting point is collecting the logs [in one place]," he said. "A lot of people don't keep the correct logs or keep them around long enough, depending on what their use cases are."

Getting good logging data to start with should not be a challenge: Firewalls, applications, and intrusion detection platforms all can or are generating voluminous data, and it is just a matter of organizing it in an easily accessible manner so that one set of logs can be properly correlated to another.

The next step is to develop a clear purpose.

"A lot of people say, 'I have these NetFlow logs, and I want to analyze them,' " Marty said. "Do you want to verify traffic against usage policy, though? Or look for attacks?" The better understanding networking pros have of what they are looking for, the more likely they are to find it, rather than just having some potentially interesting wallpaper, with little to show.

For inspiration, several SecViz users have posted their own graphs, used to detect everything from Worm attack patterns to their current IP table configuration.

Marty also suggests that networkers read his book or search online for tutorials to fit their needs.

"It's great grounds for exploring what's there, or even asking: 'I have this dataset; how do I go about analyzing it?' " he said.

To truly tap into visualization's power, however, some professionals will want to consider tapping into a variety of scripting languages that can help them pull data in a more automated, particular way than Excel's user-friendly but finite controls allow. Tapping into a framework like ChartDirector means that a bit more technical learning is required, but more precise graphs can be scripted to update themselves as new data comes in.

And once a networking pro has his charts cooked up, how best to use them?

Marty outlined three major use case areas:

• Discover and explore: Internally, sifting through thousands of log files is inefficient and, worse, it's easy for critical elements to be overlooked. A good graph can help spot trends and correlate them to other parts of the network, helping to spot and diagnose problems in one fell swoop.
• Communicate: Whether it's with the storage group on how their backups are tanking your network or your CIO on why the wireless controller upgrade is critical this budget cycle, charts can help quickly explain trends and demands to people who don't have a networking pro's specialized knowledge. Marty said one IT group had regularly sent another group log data under the assumption that they were self-explanatory. But the second IT group had no clue what they were looking at until they saw a chart.
• Strategize: Logs are great for tackling problem spots, but graphs take networkers up a level and let them see the trends, giving them a chance to look at the longer-range picture and decide how to survive not only the next 12 hours but the next 12 months.

Marty did have one warning for those getting ready to dive into visualization's benefits: garbage in, garbage out.

"One of the dangerous things is if you don't understand the log file itself, don't assume you'll understand the visualization of it or even generate a visualization that makes sense," he said. "If I have a firewall log file, and I have no idea about the IP addresses that are used or the role of internal machines, it gets very hard to analyze that."

Tags: Network Security Monitoring and Analysis,  Network Monitoring,  Network Administration,  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Network Security Monitoring and Analysis Application-specific network intrusion detection systems emerge Anomaly-based intrusion protection configuration and installation How can I calculate perimeter firewall throughput? How do I find the application on my network that's dropping packets? Integrating NAC with network security tools Where can I find a sample security audit report? How can I run my own? The firewall remains the network traffic cop, but its role is changing Troubleshooting VLANs: How to monitor 802.1q tagged traffic Poor data-loss prevention practices almost cost Intel a billion How can I block my competitor's IP address range from my website? Network Monitoring University tackles large-scale 802.11n wireless network management Meru reinvents wireless LAN troubleshooting and management Green enterprise: Three networking investments that make a difference Network device management overload: Engineers managing too many boxes What preventative maintenance procedures for network devices exist? WLAN QoS and SLA monitoring with 7/24 Wireless Quality Assurance costs How important are network infrastructure maps for engineers or admins? Understand Windows tracert output to troubleshoot network connectivity Network management and monitoring market remains crowded, fragmented When do applications suffer from poor network performance? Network Monitoring Research Network Administration Why is access denied to my Active Directory (AD) users and computers? What network loss testing tools/methods calculate dropped packets from a PC? Network user management Do I have to disable DHCP on my router to create a DHCP server? What preventative maintenance procedures for network devices exist? Top 10 reasons why computers do not have network access to each other Troubleshooting -- 'Network Know-How' Chapter 17 How server virtualization improves efficiency in a client-server model Understand Windows tracert output to troubleshoot network connectivity Why would a computer show drive letters for discs that don't exist? Network Administration Research

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary deep packet inspection (DPI)  (SearchNetworking.com) FCAPS  (SearchNetworking.com) Nessus  (SearchNetworking.com) netstat  (SearchNetworking.com) port mirroring  (SearchNetworking.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary