Problem solve Get help with specific problems with your technologies, process and projects.

Content filtering -- Taming the wild Web

Delivering safe, fast, transparent Web access that meets workforce needs has never been more challenging. Nearly four out of five FBI-surveyed companies report that employees abuse Internet privileges by downloading pornography or pirated software. Expert Lisa Phifer explains how Web content filters dramatically reduces the risk of valuable information ending up in the wrong hands. Learn how to implement and deploy content filters to your network in this tip.

The World Wide Web has been a boon to business and a bane to those tasked with managing Internet use. According to IDC, 40% of workplace Internet activity is unrelated to business. Nearly four out of five FBI-surveyed companies report that employees abuse Internet privileges by downloading pornography or pirated software. Worse, even business-related Web surfing has grown dangerous. For example, losses from phishing attacks alone exceeded $2.8 billion last year. Delivering safe, fast, transparent Web access that meets workforce needs has never been more challenging.

Business case for content filtering

Gartner recently identified five steps to "dramatically reduce the risk of valuable information ending up in the wrong hands or forcing an embarrassing public disclosure." Topping that list: content monitoring and filtering for common Internet vectors, including email, IM, FTP and HTTP. Email security covers part of this territory; content filtering can tackle the rest.

Web content filters permit or deny outbound HTTP and related requests in accordance with your Internet Acceptable Use Policy (AUP). This can reduce the bandwidth and productivity drain of non-business activities such as personal Web mail, music downloads, online gambling and porn surfing. Documenting countermeasures can limit liability for employee misdeeds or help you comply with industry-specific regulations. For example, the Children's Internet Protection Act requires that schools and libraries prevent online access to sites that are obscene, contain child pornography, or are harmful to minors. Phishing, pharming, drive-by spyware and other Web exploits also offer ample motivation for inspecting responses, preventing HTTP-borne threats from entering your network.

Adding content filtering to your network

Content filters can be deployed on firewalls, Web caches, or dedicated servers/appliances. Although each has its benefits, appliances are designed to offload the burden of URL filtering, detailed HTTP inspection, and Web usage logging -- resource-intensive tasks that could turn a heavily used cache or firewall into a bottleneck. In short, content-filtering appliances complement those systems, adding the muscle and features necessary to efficiently enforce your AUP.

Content-filtering appliances may operate in line and/or out of band. For example, the 8e6 R3000 supports three modes: invisible, router or firewall. In invisible mode, switch port replication copies Web requests to the appliance, which returns a "blocked page" response for denied requests. In router mode, the appliance sits in line, filtering outbound Web requests but not inbound responses. Firewall mode filters both outbound and inbound Web packets. In all cases, outbound Web traffic must be sent through the appliance, by the network or browser. But relying on browser settings (even automated configuration via PAC files) won't cover visitors or unsupported devices/browsers.

Content-filtering appliances should be placed inside your perimeter firewall. The firewall provides TCP/IP screening, while the appliance enforces Web content-specific policies. In larger distributed networks, appliances can be deployed for each site/subnet or in load-balanced clusters. Content filtering adds latency to a transactional application with high user expectations, so performance and transparency are important.

Finding a content-filtering appliance

Content-filtering features are widely available for firewalls (e.g., CheckPoint, Cisco, eSoft, Fortinet, Juniper, SonicWALL) and Web caches (e.g., Blue Coat, Network Appliance, Network Engines, Stratacache). Such packages leverage a platform that already inspects Web traffic but spreads existing resources across multiple tasks.

Those who prefer to dedicate a system to content filtering can install similar software on an off-the-shelf server, using such products as WebSense Web Security, Secure Computing SmartFilter, SurfControl Web Filter and Symantec Web Security. This focuses resources on content filtering and requires expertise and elbow grease to harden the platform and optimize performance.

Content-filtering appliances combine the TCO advantages of turnkey security hardware with the laser-like focus of a dedicated filtering server. A few examples are:

Some are general-purpose appliances that can be deployed as dedicated content-filtering servers. For example, Crossbeam blade servers can run Secure Computing or Websense content-filtering software. Others are purpose-built appliances developed exclusively to provide "Internet filtering." Which is the better fit for your company? That depends on your filtering feature needs, performance requirements and security architecture.

Choosing the right appliance

Like other security systems, content-filtering appliances must be hardened against attack and unauthorized admin access. Effective content filtering requires speed and storage for a large number of transactions, so consider workforce size and average/peak request rate when selecting appliance models and deciding how many to deploy.

Beyond these fundamentals, look for an appliance that can implement your defined AUP and auditing requirements. Every content-filtering appliance can block outbound HTTP, but not all filter responses that might carry banned content. Similarly, most can deny HTTPS to forbidden domains, but some do not inspect SSL-encrypted payload. In fact, "Internet filtering" appliances often examine other traffic, from conventional protocols like FTP and NNTP to newer channels like IM and P2P. This diversity complicates comparison, so start by deciding how you want to distribute enforcement between your firewall and content-filtering appliance, then find products that can implement that split.

Next, consider how Web requests are filtered. Blacklists may be composed of configured IP addresses, domain names and URL patterns -- or they may contain dynamic quarantine entries that reflect recent experience. Many appliances also offer categorized URL databases. Evaluate coverage in categories that interest you, database update frequency, and the granularity of whitelist exceptions.

If you want the appliance to filter responses, what do you expect the appliance to look for? Possibilities may include blocking or cleansing responses that contain banned words, image files, risky MIME types, unsigned active code, or malware. Some appliances can force Google or Yahoo Safe Search mode "on" to eliminate explicit sexual content from search results, but this is more of a complement than replacement for policy-based filtering at the edge of your own network.

If your AUP establishes different rules for individuals or workgroups, appliance policies must reflect that granularity. User/group profiles may specify categories, whitelists, time of day, bandwidth, or supported user agents. To avoid extra authentication when users access the Web, look for an appliance that supports single sign-on and your existing authentication system (e.g., NTLM, AD, LDAP, eDirectory). Also consider whether organizations must have the ability to specify their own policies.

Finally, look carefully at reporting tools provided by the appliance to analyze and track Internet use throughout your network. Some companies start using content filters simply to spot and document inappropriate Web activity. This is an excellent way to learn what your network is really being used for when defining an Internet AUP. Unless your workforce is small, however, automated analysis and summary reports with drill-down capability will be key to isolating Web abuse and risk exposure.

About the author:
Lisa Phifer is vice president of Core Competence Inc., a consulting firm specializing in network security and management technology. Phifer has been involved in the design, implementation and evaluation of data communications, internetworking, security and network management products for nearly 20 years. She teaches about wireless LANs and virtual private networking at industry conferences and has written extensively about network infrastructure and security technologies for numerous publications. She is also a site expert to and

This was last published in January 2007

Dig Deeper on Network Security Monitoring and Analysis

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.