Problem solve Get help with specific problems with your technologies, process and projects.

Virtualization security challenges are plentiful: What's the answer?

The network plays a role in addressing virtualization security challenges, but typical firewalls are not necessarily part of the solution.

The takeaway: It's difficult to firewall and defend constantly migrating virtual servers. To address virtualization security challenges, network managers must explore new strategies that can include a combination of virtual firewalls and virtual switches for traffic inspection.

In an ideal world, servers would be hardened enough to withstand any attack from the Internet, and the network’s primary task would be to provide optimal end-to-end transport. Realistically, though, the data center network plays a central role in security with firewalls embedded throughout, providing at very minimum inbound TCP-port-based traffic filtering to minimize the impact of brute-force denial-of-service attacks.

Using these firewalls to defend physical servers in a data center network is difficult enough, but securing virtualized servers—or a private cloud environment—is even tougher. After all, virtual servers migrate regularly, blowing the basic premise that firewalls should sit in physical proximity to servers. However, there are a few strategies to provide firewall protection in a virtual environment.

Virtualization network security challenges

Network security in traditional data center architectures is a well-understood design task: Firewalls are usually inserted somewhere in the aggregation layer, assuming physical proximity of servers belonging to the same security zone. When you start virtualizing servers and deploying virtual machine (VM) mobility and automatic load distribution with VMware’s vMotion and Distributed Resource Scheduler, for example, the physical proximity paradigm no longer works. In that scenario, the traffic between the servers and the rest of the network still has to pass through a firewall, resulting in infamous traffic trombones and increased load on the data center core.

The situation only gets worse as you start deploying virtualized network appliances that allow you to quickly deploy a firewall, router or load balancer anywhere in the network in virtual machine format. These virtualized appliances can move freely between the physical servers, resulting in even more circuitous traffic flows. VMware’s vCloud Director is experiencing the same design problems.

Using DVFilters and virtual firewalls

Several years ago, VMware added a hypervisor DVFilter API that allows third-party software to inspect network and storage traffic of collocated virtual machines. Some firewall/IDS vendors were quick to realize its potential and started offering virtual firewalls that did not exhibit the tromboning behavior. VMware joined these vendors last year, launching the vShield Zones/vShield App products.

A DVFilter-based network security appliance does not act like a typical firewall. Instead of forcing the traffic to flow through the appliance based on IP routing rules, the firewall is transparently inserted between the virtual machine’s network interface card (vNIC) and the virtual switch (vSwitch) within the hypervisor. The firewall inspects all traffic entering and leaving the vNIC without any additional configuration on the virtual machine, the virtual switch or the physical network. The vShield products augment this idea with a remarkable configuration hierarchy: You can configure firewalling rules on data center, cluster and port group (security zone) level, and the firewall combines the appropriate groups when building per-vNIC policy.

The idea of a collocated firewall automatically protecting the virtual machines sounds perfect, but it has several potential drawbacks resulting from the architecture of the DVFilter API, which works only within the hypervisor.

Some virtual firewall drawbacks include the following:

A firewall VM must exist on every physical server. A firewall appliance can protect only those virtual machines that run on the same physical server. If you want to protect your virtual machines regardless of their physical location, you have to deploy the firewall VM on every physical server.

All traffic is inspected. It might be possible that you can apply the DVFilter API only on select vNICs—and protect only some virtual machines and not the others—but that’s not how vShield products work. The moment they're deployed, all traffic passing through the hypervisor is inspected, resulting in increased CPU utilization and decreased network performance.

Firewall crash blocks VMs. Another failure of the firewall VM is that it blocks the DVFilter API. All the virtual machines running on the affected physical server are cut off from the network. However, the physical server is still running and connected to the network; high-availability features will thus not move the affected VMs to other physical servers.

Multiple inspections of the same traffic flow are performed. DVFilter API inspects traffic at the vNIC level. Traffic exchanged between virtual machines is thus inspected twice, even when it belongs to the same security zone—in which case a traditional firewall would not get involved at all.

Where virtual switches play in virtualization security

Makers of virtualized security appliances also have the option of the vPath API, which can be used to implement custom virtual switches. Cisco Systems has recently launched its Virtual Security Gateway (VSG) product, which is supposed to combine the traditional (non-DVFilter) virtual firewall approach with traffic flow optimizations. Cisco claims that VSG performs only the initial traffic inspection and offloads the traffic forwarding to the Virtual Ethernet Modules (VEM, modified virtual switches in the hypervisor), avoiding traffic trombones and performance problems. If that’s the case, VSG may be ideal for engineers looking to deploy secure cloud services.

About the author: Ivan Pepelnjak, CCIE No. 1354, is a 25-year veteran of the networking industry. He has more than 10 years of experience in designing, installing, troubleshooting and operating large service provider and enterprise WAN and LAN networks and is currently chief technology advisor at NIL Data Communications, focusing on advanced IP-based networks and Web technologies. His books include MPLS and VPN Architectures and EIGRP Network Design. Check out his IOS Hints blog.

This was last published in July 2011

Dig Deeper on Network virtualization technology