Problem solve Get help with specific problems with your technologies, process and projects.

Preventing hacker attacks with network behavior analysis IPS

Deter enterprise network attacks by learning how network behavior analysis can be used to protect data. Understand the advantages and disadvantages of anomaly-based and signature-based protection and how to use them together to secure your network against complex hacks.

Two approaches to preventing network security attacks are signature-based and anomaly-based network behavior analysis...

(NBA). To learn how these intrusion prevention system (IPS) techniques deter hackers and protect your network, continue reading this tip by technical writer David Jacobs.

News of successful network attacks has become so commonplace that they are almost no longer news. Hackers have broken into commercial sites to steal credit card information and into defense industry sites in search of top-secret military plans. Recent denial-of-service (DoS) attacks have made sites unavailable to legitimate users. Firewall and intrusion prevention systems across various enterprise networks routinely log hundreds of hacker attempts a day.

To prevent successful attacks, two key detection approaches have evolved: signature-based and anomaly-based network behavior analysis (NBA).

Signature-based intrusion protection and detection
Signature-based systems are extremely effective against attack types that have been detected in the past. They can be installed quickly and become effective immediately. These systems examine each incoming packet and compare its contents against a list of known attack mechanisms. False positives, legitimate activity that appears to be an attack, are rare. Generated reports are easy to understand because each incident indicates the type of attack that was detected.

While signature-based systems are effective against known attack types, they cannot detect zero-day attacks. Hackers understand that any new attack type will be quickly detected and countermeasures will be adopted by intrusion prevention vendors. They therefore launch attacks on a large number of sites as soon as a new attack method is developed.

Because of this, signature-based systems must be continually updated. Vendors collect and monitor attack reports from across the world. They also collect data from products installed at customer sites. When one customer experiences an attack, vendor staffs analyze it, develop a defense and distribute the update to all other customers' sites. While vendors can often detect new attack methods and devise a defense quickly, the first sites to be attacked have already been compromised.

Anomaly-based intrusion detection systems
Anomaly-based detection systems detect network activity that does not fit the pattern of expected behavior. The system must be configured, according to the product, with information on normal patterns of activity. For example, applications may legitimately access a single database record at a time. If the intrusion protection system detects access to a large number of records, the cause is likely to be an attack. Similarly, if a user with permission to access a restricted set of records begins to attempt access to other types of information, the user's workstation is likely to have been infected.

Unlike signature-based systems, zero-day attacks can be detected because the attacks do not have a pattern that is recognizable as legitimate to the anomaly-based intrusion system. All that is necessary is that something outside the ordinary is occurring. The downside is that anomaly-based systems must be carefully configured to recognize expected patterns of activity. Configurations must be updated when new applications are added or existing applications modified. False positives can occur when legitimate activity departs from its normal pattern.

Configuring IPS to defend against complex attacks
Attacks in which elements of the attack are spread across multiple commands such as HTTP messages for Web-based attacks present a difficulty for both signature-based and anomaly-based systems. For signature-based systems, the signature may be spread across a series of commands with no one packet matching an attack profile. Anomaly-based systems may fail to detect an attack that simultaneously targets several hosts. The sequence sent to each host may appear legitimate but may cause applications on the hosts to interact in such a way as to cause a breach.

More on network behavior analysis IPS
U.S., Korea Internet attacks indicate sophisticated criminals on the rise

Reasons to deploy NBA

Using IDS to detect hacks

Securing dynamic network architecture

Compounding the difficulty, not all of the packets may enter the network at the same point or gateway. Although enterprise networks often maintain more than one gateway to the Internet with intrusion prevention systems at each gateway, guarding all the gateways is not sufficient.

Viruses can penetrate a network through places other than gateways. Employees take home laptops for use on their minimally protected home networks. When they reconnect the infected laptop on the internal network, viruses enter the network without passing through an Internet gateway. Wireless networks are another vulnerable point and cannot be overlooked when implementing an intrusion prevention system. An outsider breaking in via the wireless LAN (WLAN) has also bypassed the network gateways.

Intrusion protection systems must also be installed at key points throughout the network (like a switch connecting network gateways to servers where applications run or connect to database servers) to detect these attacks. Systems must exchange information with each other and evaluate reports from sources such as router and host logs to correlate the sequence of packets to detect the attack.

While signature-based systems can be quickly installed and immediately become operational, designing, configuring and installing an anomaly-based system is more complex. The next article in this series explores the steps involved in configuring and installing an anomaly-based system.

About the author:
David B. Jacobs of The Jacobs Group has more than 20 years of networking industry experience. He has managed leading-edge software development projects and consulted to Fortune 500 companies as well as software startups.


This was last published in October 2009

Dig Deeper on Network Security Best Practices and Products

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.