Now that Blue Cross Blue Shield of Minnesota relies on Software as a Service (SaaS) for core business processes like administering claims, network performance engineer Barry Pieper relies on deep packet analysis to tap inbound and outbound Internet traffic in order to ensure his providers are delivering on their service-level agreements (SLAs).
But it wouldn’t be worth using costly deep packet inspection for all of his network monitoring needs, so Pieper still turns to good old fashioned NetFlow analysis for a broader view of what’s happening on the network.
Combined, Pieper uses a Network Instruments Gigastor appliance for packet capture, Compuware’s Vantage network monitoring product— recently rebranded as Gomez Network Performance Monitoring—for analysis of that packet information, and then Fluke Networks’ Optiview NetFlow Tracker for NetFlow.
“I use NetFlow a lot on our wide area network mainly because it works so well there,” he said. “Our branch offices are T1 and T3 links, so we would do software distributions with Altiris or Tivoli and that would cause problems for people using our in-house applications or web apps. NetFlow could quickly tell us this was Altiris traffic and we could find out if people were streaming radio and things like that.”
Pieper is not alone in including NetFlow in a next-generation combination of monitoring tools. NetFlow monitoring may not always get the respect it deserves from the network management community, but it can alert engineers to bandwidth hogs or anomalous behavior, and NetFlow v9 allows users to pull even more data from flow records.
NetFlow monitoring may not always get the respect it deserves from the network management community.
What’s more, while packet stream monitoring tools go deeper, allowing network engineers to dig into exactly what is happening across the wire, continuous packet monitoring and analysis is not cheap. Probes and taps are expensive, and storing the data collected can be pricey, particularly for larger companies that are producing many terabytes a day. Therefore, most enterprises can typically only monitor packet streams in select, critical locations on the network, offering only a narrow view of the network at a time when many enterprises are clamouring for more and more visibility.
“The amount of visibility organizations need to totally quantify how their applications and infrastructure is running continues to increase,” said Brad Reinboldt, senior product manager for network monitoring and analysis vendor Network Instruments. “There can never be too much information.”
For that reason, Reinboldt has seen increased use of NetFlow monitoring by his customers.
“Based on what we talk to our customers about, 25% to 50% of them do at least some level of flow monitoring as part of their overall monitoring solution,” he said. “What flow technology can offer you is a broader perspective.”
NetFlow monitoring for broader network visibility
For many network teams, NetFlow offers enough information to handle about 90% of their problems, and then they turn to deeper tools for the other 10%, said Jim Frey, research director for Enterprise Management Associates.
“I have talked to a lot of folks who use packet instrumentation in important parts of their network. Then they use NetFlow to get a sense of what’s going on in remote sites,” said Frey.
Everett McArthur, a tier-three enterprise network support engineer at Texas Tech University Health Sciences Center, monitors his network with a combination of NetFlow and packet monitoring. While his packet capture technology is instrumented to collect traffic in specific areas of the network, he can turn on NetFlow in any location at any time when he needs to troubleshoot something.
Recently staff at a remote clinic 400 miles away from McArthur’s Lubbock, Tex.-area location complained of bandwidth saturation. McArthur turned on NetFlow on the remote clinic’s router and pointed it at his nearest NetFlow collector.
Read more about network visibility tools
NetFlow traffic analysis tool solves network congestion woes for nonprofit
Deep packet inspection: Controversial but valuable traffic management tool
Network traffic capture systems offer broader security visibility
“We found out very quickly that the inbound link to this clinic was receiving update traffic from Microsoft, but outbound it was saturated because the clinicians were all hitting a particular electronic medical records server,” he said.
“So we had two different problems. It was saturated one way from the updates being run and the other way by people dealing with medical records. We were able to make some decisions on what to do immediately about the issue, and then they increased their bandwidth for the long-term. Without NetFlow, I would have had to go out with a portable analyzer and put a tap on the line.”
Find problems with NetFlow analysis, dig deeper with packet analysis
Most of the engineers who use NetFlow get a lot of value out of it for higher level monitoring, Frey said. “Then they use packet analysis for the difficult problems.”
At Integra Telecom, a network communications and cloud services provider based in Vancouver, Wash., network support manager Jeff Willard uses CA Technologies NetQoS NetFlow for visibility across his broader network, particularly at the peering transit edge so that he can detect network threats coming from customer locations. To increase visibility, Willard is in the process of adding NetFlow in the aggregation points of his network too.
“That will allow us to have a better understanding of our customers’ networks and their usage and improve our ability to assist them with any problems or issues they have.”
NetFlow is useful for adding context to better understand the information obtained through packet capture.
“Having a raw pcap file to sort through with no idea of what you’re looking for can be daunting,” Willard said. “Leveraging NetFlow data to give you a better understanding of what is traversing the link…gives you a frame of reference for where to look within a packet capture."
“Having NetFlow for the visibility and graphical representation of the network and using that for trending and alerting can shed light on hotspots or conditions that we need to investigate further. Then we can sniff the wire for traffic at this particular link or aggregation point.”
Integration of NetFlow monitoring and packet monitoring tools needed
As network operations teams increasingly use both NetFlow and packet monitoring technologies together for broader visibility, they will need tools that can offer a common view of both sets of data—but there is no easy solution available.
“If you’re trying to use a combined set of [packet capture and NetFlow] for monitoring, you need some method for bringing this data together in a common console. There is some work still to be done to bring these together.”
McArthur of Texas Tech relies on Network Instruments for both his NetFlow and packet monitoring.
“Since it’s the same interface, it makes it a lot easier to do our analysis,” McArthur said. “You're not having to relearn a different way of doing things.”
As more network engineers combine these methods, it’s likely that a set of integration tools will emerge.