peshkova - Fotolia
Big data analytics enjoys its share of hype today. Many enterprises like the idea of uncovering unexpected, actionable business insights via the application of advanced analytics to vast amounts of structured, unstructured and partially structured data. But all this data crunching comes with a price. In particular, a network can get slammed by the technology.
Big data and the questions of big data impact on network operations are not for the faint of heart. Set aside, for the moment, the fact that big data tools are immature and people who know how to use them are in short supply. Instead, let's talk about the new burdens big data imposes on IT infrastructure and operations. Suddenly, the enterprise is moving around and crunching vast amounts of data that might otherwise have been discarded or left to collect dust.
Enterprise Management Associates (EMA) Inc. polled more than 150 North American enterprise adopters of big data to get a picture of how big data affects IT infrastructure management. The report, "Big Data Impacts on IT Infrastructure and Management," examines which aspects of IT are most impacted by big data. The study, the first of its kind by EMA, also explores how IT organizations are flipping the script and using those same big data technologies to improve IT planning and operations. Perhaps not surprisingly, the network and the network management team feel the effects of big data more deeply than almost anyone else, the research found.
Big data adds network traffic, forces engineers to adjust
Our research found that 45% of enterprises with active big data projects have experienced increased network traffic due to the collection of that data. We also found that 46% of these companies had experienced network traffic growth due to the backup of these big data repositories. The only part of IT infrastructure that was more often affected by big data was storage -- no surprise, given that big data is all about the collection and storage of larger and larger amounts of data.
The networking and big data impact problem becomes more pronounced as organizations expand their use of the technology. For instance, EMA classified enterprises with six or more big data projects in production as "advanced" big data users. Among these companies, 55% saw more network traffic from big data collection and 61% reported more traffic from big data backups.
Given this surge in network traffic caused by big data, EMA asked these enterprises to explain how their networking teams are reacting to these conditions. Our research found that of all infrastructure managers within an IT organization, it's network managers who are adjusting their infrastructure planning and design practices in response to big data. More than half of network managers say they are making plans to meet the demands of big data -- from refining capacity management practices to expanding and upgrading network infrastructure to support traffic growth. In particular, these managers need to baseline big data traffic; this traffic will only grow over time, so engineers need to understand those trends and plan capacity accordingly.
Meanwhile, almost half of network teams said big data forced them to tweak their daily operational practices. In all likelihood, these network managers have to adjust both their performance monitoring and troubleshooting tools and processes as a result of the increased volume -- and bursty nature -- of big data network traffic.
Network managers make big data work for them
After exploring how big data impacts infrastructure, EMA examined how IT organizations make big data work for them. We identified a large subset of research participants who were exporting IT monitoring data to big data environments for collection and analysis. We asked these organizations to identify which management practices were helped by the advanced analysis of this IT monitoring data.
EMA found that 57% were using big data analytics to support network capacity planning, and 53% were using it to support network availability and performance monitoring. Finally, among troubleshooting tasks, 35% were applying big data to isolating infrastructure issues in the network. As I mentioned above, this same research found that network managers were adjusting their planning and operational practices to deal with the network impacts of big data. Now, we find that many of these network managers are turning around and using big data analytics to bolster planning and operations.
Big data analytics of IT monitoring data has a lot of potential. Not only did EMA observe changes to IT management practices, but it also measured the benefits that IT organizations are experiencing with it. Forty-five percent of these IT organizations told us that big data analytics had enabled them to proactively prevent infrastructure problems; 45% reported that they had become more efficient with IT operational expenses. Furthermore, 46% said they were more efficient with infrastructure capacity with big data analytics. And finally, 41% said big data analytics of IT monitoring data had helped them better align the IT organization with the business organization.
This EMA research looked at the impacts and benefits of big data on all aspects of IT infrastructure and IT management, but it's clear that the network management team should pay special attention to the big data impact issue. The big data impacts on infrastructure are real and will probably increase over time. At the same time, the opportunities for network engineers and managers to apply big data analytics to their own work are very promising.
How business analytics can help your organization
Reining in big data for good