"In the past, a lot of this was trial and error," said Bob Branski, systems administrator at Goodwill Industries of Southeastern Wisconsin and Metropolitan Chicago, the largest Goodwill worldwide in terms of revenue and employees. "Now I can just say, 'Oh, 80% of my data is coming from this one server,' and I can focus my energy on [resolving] that instead of going on a hunt."
For Branski, the trouble with resolving network strain was a matter of scale. End users throughout the nonprofit's 56 locations -- interlinked through an MPLS connection, plus another 13 remote sites via VPN or ISDN -- draw on more than 120 physical and virtual servers at corporate headquarters in Milwaukee.
Using a NetFlow traffic analysis tool, Branski saw that automatic updates for virus scanners and Windows -- contrary to what he had thought -- were not configured to act as background transactions.
"When you're talking about 60-plus sites, I don't care how much bandwidth you throw at the problem -- you're never going to fix it," he said. "[I learned that] I didn't have a network problem. I had a systems configuration problem."
Goodwill's network, built with Cisco Systems routers and switches, connects headquarters to regional retail stores, as well as sites handling commercial services, logistics, product packaging, laundry services, food services and uniform services. Among the 30,000 people Goodwill serves annually are sailors and their families at the U.S. Naval Station Great Lakes in nearby North Chicago.
"They order the food they're going to prepare and they have to submit these orders daily [through the network]," Branski said. "We can't go to the Navy and say, 'We didn't get your food order today. We can't serve you.' That's not possible."
NetFlow traffic analysis tool cuts troubleshooting time
For the past two years, Branski had used an open source network monitoring tool, Zenoss, to alert him when bandwidth maxed out. But he pointed out that getting an alert about a clogged pipe doesn't tell you how to identify the source of the clog.
"I could see it queuing and dropping packets … but it doesn't tell me what it's dropping," he said. "I could see we were over-utilizing things; but what was causing it?"
Through his contacts at Zenoss, Branski learned about Scrutinizer from Plixer International , a small vendor in Maine specializing in sFlow and NetFlow traffic analysis. The software integrated directly into Zenoss, alleviating Branski's concerns about having to manage multiple monitoring tools.
Plixer offers a free demo version of its sFlow and NetFlow traffic analysis tools in addition to more advanced features in its commercial version -- reporting and archiving the hosts, applications and protocols that are eating up the most bandwidth across all routers and switches throughout an enterprise.
For Branski, Scrutinizer has turned resolving network congestion problems from a task that took hours into one that takes minutes.
"The biggest improvement we've seen is [shortening] the time to troubleshoot and isolate a problem, especially in utilization on a network or if we see a spike in network traffic," he said. "I can easily go into Scrutinizer to see, 'Yeah, it was these workstations … using up most of the bandwidth.'"
Without any granular sFlow and NetFlow traffic analysis, networking staff spend a lot of unnecessary time "running around with a packet analyzer" to find out why their pipe is maxed out, said Plixer president Mike Patterson.
"We pinpoint problems and can drill down into those problems," Patterson said. "But sometimes there is no problem. Sometimes the network pipe just isn't big enough, and we can tell companies that this is all valid traffic and maybe the pipe is just too small."
Let us know what you think about the story; email: Jessica Scarpati, News Writer