Published: 27 Feb 2012
Troubleshooting slow network performance is difficult enough in a physical environment, but virtualization requires engineers to run even more complex network diagnostics.
After all, in a bare metal data center, network administrators have lots of tools at their disposal to follow packets from end to end, but in a virtualized data center, east-west network traffic between virtual machines on the same physical server is invisible to physical network infrastructure.
Fortunately, there are ways for the network team to peer into the black box of the virtualized environment using a patchwork of tools and strategies.
Using virtual switches to manage virtualization traffic
Hypervisors on virtualized servers use built-in virtual switches (vSwitches) to manage traffic among virtual machines. These software vSwitches are designed to replicate the functions of a physical network switch, including support for port monitoring. However, they typically lack the deep functionality of a traditional hardware-based network switch.
Cisco Systems' virtual Nexus 1000v switch and the open source Open vSwitch offer more features and functionality than hypervisor-embedded virtual switches, and they more closely resemble their physical counterparts. As a result, they offer network administrators centralized management and visibility of both physical and virtual network infrastructure.
But virtual switches also have some downsides. Every function they offer adds additional workloads and saps CPU cycles from the virtual server environment. Network administrators need to balance the need for port monitoring and network troubleshooting with the overhead that such monitoring will pose to compute resources.
Integrating network management tools with hypervisors for network diagnostics
More on network diagnostics and management for virtualization
BUYERS GUIDE: Virtualization monitoring tools
Virtual switch architecture best practices
Virtual network security: Vendor comparison
VIDEO: Network analysis in virtualized environments
Some vendors offer data center network management platforms that can simply gather information about virtual machine activity from the hypervisor. VMware offers an application programming interface (API) on its management platform that allows third-party management tools to track virtual machines. However, network administrators generally depend on their management tool vendors to do this integration. What's more, this approach does not necessarily provide the depth of information that pure packet captures provide. Integrating with the virtualized environment, however, does contextualize the impact that virtual machines are having in the network and helps define the path that an application is taking through the environment.
Deploying network probes inside the virtual environment
Network diagnostic and forensic vendors are developing virtual probe software that can provide instrumentation within a virtualized environment. WildPackets’ OmniPeek analysis solution, for instance, now features OmniVirtual, a virtual network probe that installs on each virtual machine as it communicates with both virtual and physical elements of the network. OmniVirtual transmits data to a centralized network analysis appliance.
Combining virtual and physical network probes delivers complete network visibility. These virtual services, however, are specific to each network analysis product and will only provide visibility to the vendor's network management products.
The network team will also have to work closely with the server team to ensure that virtual network probes are deployed on every virtual machine in the environment. Any workloads running without a probe will remain invisible to the network management tool.
Forcing virtual machine traffic back into network hardware
A new set of standards pending with IEEE offers administrators virtual network visibility without the CPU overhead associated with virtual switches or virtual probes. Edge Virtual Bridging (EVB), also known as 802.1Qbh, brings together hardware, software and protocol standards to simplify and automate the links between physical and virtual Layer 2 networks in the data center. The standard will allow physical and virtual switches to talk to one another and share configuration information.
The EVB standard will also include a technology called Virtual Ethernet Port Aggregation (VEPA), which instructs virtual switches to send all traffic from virtual machines upstream to the nearest physical network switch. This exposure of the virtual machines to the physical network allows network administrators to apply traditional network analysis and management.
VEPA also includes provisions for communications between virtual machines on the same physical server and network hardware, known as a reflective relay or a hairpin turn. In addition, VEPA allows a physical switch to send data back across the same network port it came from. Administrators can deploy VEPA either through upgraded virtual switch software or within hardware on supported network interface cards. Depending on the workloads and utilization on a given server, most enterprises will deploy VEPA as a mix of software and hardware.
Edge Virtual Bridging can also help automate network configuration and policy management through the Virtual Station Interface Discovery Protocol (VDP). VDP allows the network to know of the movement of a virtual machine in advance of the move and automate network configuration for the destination hypervisor host.
Pulling it all together for network diagnostics
Each of these solutions gives the networking and server teams the ability to focus on their own domains, but they all require collaboration between the two teams for implementation. Enterprise infrastructure can no longer be siloed. It must function as an ecosystem. Efforts to automate the virtual environment will have to be matched with automation on the data center and edge networks. Likewise, silos of IT management will have to be broken down and collaboration between network, server and storage teams will have to be just as agile as the infrastructure they hope to build.