As networking professionals know all too well, their responsibilities often extend beyond the network operations center (NOC), and the network itself is frequently made scapegoat when applications don't perform as expected. At Interop Las Vegas last week, Dr. Jim Metzler, vice president of Ashton, Metzler & Associates, called on network pros and IT pros to overcome technological and organizational silos in order to increase the prestige of IT and to improve overall technology performance.
Metzler moderated several sessions at Interop last week that had a single unifying theme: Application delivery is now an issue of critical importance for IT (and consequently network) staff, and successful application delivery depends upon the ability of that staff to transcend technology and organizational silos or "stovepipes."
"My concern here is not the existence of stovepipes, but how they're multiplying," Metzler told attendees of a session called Breaking Down the Technology and Organizational Silos. "We keep bolting on new technologies [such as Wi-Fi, WiMax and virtualization in its many forms], bolting more and more stovepipes onto this -- what is potentially a Tower of Babel."
He found that most NOC staff spend the majority of their time on application delivery and the WAN. "The network operations center is a lot more than networks," Metzler said in a phone interview. "In fact, I think it's a misnomer to use the phrase NOC or 'network' manager or 'network' engineer because the majority of their time is spent on other things."
At Interop, Craig Hulbert, senior network engineer at Toledo, Ohio-based HCR ManorCare, agreed that the network staff in his company is often called upon to fix or identify problems outside the network. "More people come to us first; we have to tell them what's broken -- their application -- before they will believe it's not a network issue," Hulbert said. "We're trying to get out of that business, to be able to allow instrumentation to tell the server guys when their applications are performing poorly, to be able to tell us when the network is performing poorly, and kind of be that event marshalling."
Hulbert said his company, which provides long-term healthcare, is looking to incorporate network monitoring software into its next WAN to give him flow-based reporting and alerting capabilities. "We own all the events in the enterprise. We're telling everybody when their stuff's broken. Now, I can go a step further and be able to defend it by having the flow data."
"Mean time to innocence"
Because of its traditionally siloed structure, IT is often guilty of finger-pointing rather than actual problem-solving -- a phenomenon Metzler jokingly calls "mean time to innocence."
"Right now we so often troubleshoot in a very defensive way," he said. "It's assumed it's the network in most cases. It gets into me vs. you vs. somebody else."
This holds true in Hulbert's company. He said that one co-worker spent 80% of his time proving that the wireless network was not causing an application to run slow. "We can show packet captures, we can show all this stuff, and it still comes right back around," Hulbert said. "The business people still say it's the network. Well, no, your app is hosted over a T1 line in California, and you've got 300 users doing it, and you're smashing your circuit because you're all watching YouTube. Sorry, I can't help you!"
Metzler said it's just not "cool" any more for the network staff to merely pin the blame on somebody else. "It increases uptime and saves time solving problems if you can break down those technological and organizational silos. People who used not to work together now do. My somewhat OK process has to interact with your somewhat OK process."
Hulbert's staff is doing just that. "We're actually doing things to fix developers' code because they don't want to go through the change-management process to change their code," he said. "So we're actually putting things in to protect what the user sees and to hide that error that's in the code."
And this isn't just an IT problem, according to Metzler. "It's the exceptional large company that has good processes that go across sales and engineering and customer service. It's not just an IT problem; U.S. industry does not do a good job of cross-organizational processes," he said. "It's a simple problem to state, but tough to solve."
More prestige = more resources
Network pros find that they can't get the resources they need when the rest of the IT organization fails to understand the responsibilities of the network.
"When you're asking for resources and don't have the credibility -- and IT is seen as some necessary evil -- [the management is likely to react to requests by saying] 'Give 'em their damned 3% increase and make them go away!' and that's not a good position to be in," Metzler said.
The NOC also often fails to win the same amount of credibility as the rest of IT, which can give it limited access to company resources, Metzler said. "It's more than a curiosity to me that most of IT doesn't 'get' what the NOC does. There's lots of change going on in the NOC. If the NOC is trying to make changes inside the NOC, they need resources. Everybody else needs those resources. If the rest of IT doesn't get the NOC, they're less likely to get those resources."
"They don't market what it is they do," Metzler said. "People don't get it. I think we have to change some of that."
Metzler urged IT pros at Interop to add value to what they do in order to increase their prestige. "Do the projects. Add value," he said, "Otherwise you're regarded as a utility like electricity and plumbing -- I don't know what you get from it, but I'm shy to pull the plug."
How to change the value of the NOC professional
Metzler said that IT and NOC staff should speak in terms of business needs and cost savings in order to prove their value.
"My take has always been, [business owners] don't care about MPLS, VPNs, NetFlow, or SNMP," he said. "We have to attach what we do to what they do care about. You don't want to go to the vice president of sales and say, 'I have MPLS.' It sounds like a disease. You would not get a second meeting with that person; you have just screamed, 'I am a techie nerd.' "
Networking pros can win prestige by tying their technology implementations directly to business initiatives, Metzler said. Or they can deliver a successful technology, such as videoconferencing, which holds value for business users.
It's an issue of being able to delight your customer, according to Metzler -- and that customer has high expectations.
"If you're in IT, you're always being asked, 'What have you done for me lately?' " he said. "So if you can roll out something, let's just say using video as an example, people are going to get value from that. I don't have to travel as much. I have better communication. Ooh, I value that. So I get that."
At Interop, Robert Whiteley, senior analyst with Forrester Research, told Cisco luncheon attendees that "there is a huge correlation between IT organizations that have video and those that say, "We're seen in a positive light.""
Hulbert proved his network group's worth with a small network monitoring application called Plixer, which provided a valuable "stepping stone" to enterprise network management.
When he first broached the topic of needing a $350,000 tool to monitor traffic network-wide, his request was denied, Hulbert said. "I found Plixer -- it was like a $40,000 investment. It's not going to scale enterprise-wide, but we still have more visibility and more data now than we ever did."
Hulbert said that by integrating Plixer with some in-house scripts, and other tools including SolarWinds Orion, he was able to demonstrate the value of having a network management system in place for the company's new WAN. "It's a jumping-off point, but now everybody's starting to see some of that value. We start throwing out numbers like $350,000, and they're not batting an eye now," he said. "Before, they were telling me, 'Go fly a kite! You're never going to get that kind of money.' "
Frameworks, such as ITIL, help break down technological and organizational silos
The constant demand to delight the customer brings with it the demand of delivering new and increasingly complex applications, making it even more imperative to work across organizational and technological stovepipes.
"Video presents tremendous challenges -- let's just start with the simple ones. When's the last time you've been on a videoconference that went well, was set up easily and didn't crash?" Metzler said. Video involves challenges of providing enough bandwidth and quality, he said, as well as the cost being delivered over a WAN, sometimes internationally. "So you've got some economic challenges, QoS challenges. It's a real opportunity for IT to show value to delight their customers, but like everything, it comes with a price."
Another challenge, according to Whiteley, is that delivering complex, end-to-end applications such as videoconferencing or Sharepoint requires coordination across organizational and technological silos.
Whiteley said companies can become more "process-centric" to help defeat silos. "That's not to say that silos are going to go away; they won't." But companies can "put folks in place that can handle different aspects -- typically based on ITIL."
While server and storage teams have already implemented ITIL, the network has "by and large been left out from this trend," Whiteley said. ITIL, a framework of best practices for IT management, can give IT organizations "the capability to become more process-centric, freeing up cycles from rebooting switches and things like that, and that's sort of the people side of the equation that allows people to get ready for things like video."
Metzler said that some people are optimistic about ITIL and others are pessimistic. "We know we have to get there with processes -- that's not a question. The question is: how? Technology continues to need processes, and they need to interrelate. The question that I pose rhetorically is: 'What could cause an application to degrade?' Anything! All the pieces have to fit together, from a planning perspective and from an operational perspective. That's a tough challenge."