Get started Bring yourself up to speed with our introductory content.

When SDN meets Hadoop big data analysis, things get dynamic

SDN can provide the network orchestration necessary to bring dynamic Hadoop big data analysis to life today.

IT organizations have the wherewithal to bring new technologies to the lab and test them, but moving from legacy designs and products to something as revolutionary as SDN is a huge challenge. The question becomes: How can we integrate SDN in manageable steps?

Conrad MenezesConrad Menezes

One answer would be to start with applying SDN to specific network management-related applications -- and one such application is Hadoop big data analysis.

In the retail sector, IT organizations are working with static or shrinking capital budgets and a dwindling headcount. Meanwhile, they face growing demand for applications, data storage and analysis. Against this reality, SDN is interesting but only insomuch as it delivers immediate business value.

At this stage of development, so-called northbound applications for SDN are more promise than reality -- but southbound, network-management-related applications are becoming deployable.

Retailers commonly see data stores increase 20% to 40% annually, and of course, nothing ever gets deleted. Hadoop big data analysis can manage as well as analyze these stores. But without robust and effective orchestration tools for both servers and networking, growing or setting up new Hadoop clusters can take from days to weeks. Networking, server and application teams must work together to deploy and test new installations, and without good orchestration tools, it's a labor-intensive job.

On the server side, systems like OpenStack allow for such orchestration. On the networking side, SDN and its associated management tools also can get the job done. The reduction in deployment time and improvement in deployment accuracy are big wins for IT and for the business.

More on the SDN issues tackled by ONUG

It'll take more than a single vendor to tackle IoT security

Why we need SDN and orchestration for IoT

Other southbound applications with big potential include storage networking, telephony and data convergence, and aggregated network management. On the storage side, we're seeing a big push toward IP storage -- both FCoE and iSCSI. With that comes the need to allocate resources based on both the capability of the storage arrays involved and the applications in use. The dynamic nature of virtualized servers requires that the network be just as responsive to application needs. For convergence, it’s critically important that parameters such as quality of service and bandwidth allocation be easily configured and reconfigured to match the needs of users. These naturally belong in software based management systems like SDN. And finally, one major miss for today’s network management systems is the ability to aggregate usage data so that network managers can show the overall network usage by application.

The Open Networking User Group (ONUG) is working to prioritize and bring to market SDN applications that are realistic today. Hadoop is one of those applications. ONUG members can use their aggregated buying power to drive these solutions to market.

ONUG Spring 2014 will be held in New York, May 5-6.

About the author:
Conrad Menezes is one of the newest members of the ONUG advisory board. For the last four years he has been the VP of IT at Sears Holdings, a Fortune 100, where he focuses on modernizing the retailer's IT infrastructure. Conrad has been involved with IT architecture and engineering for more than 20 years. See his full bio here.

This was last published in April 2014

Dig Deeper on Software-defined networking

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.