Troubleshooting IPSec

Troubleshooting IPSec
Tom Lancasater

Configuring a VPN is a fairly complex task, and configuring ISAKMP (Internet Security Association and Key Management Protocol)

    Requires Free Membership to View

    Login

and IPSec (IP Security Protocol) can be especially challenging. Odds are good that the configuration may not be quite right on your first attempt. This is a brief summary of the best practices for problem determination.

The first thing you should do is eliminate the easy stuff. Remove your security policies and verify that you have basic IP connectivity between your tunnel endpoints using PING.

Next verify that no device in the path is blocking your traffic. In this case, that would be UDP 500 and IP protocols 50 and 51, for ISAKMP, ESP and AH respectively. Also make sure NAT isn't in the path, either.

Next, break your security policies into three major sections:

ISAKMP negotiation
ISAKMP, when you use it, will negotiate a connection with the peer before IPSec begins its negotiation, so check to make sure the ISAKMP negotiation is successful. The tools used to verify this process will be different for each vendor, but generally, ISAKMP should recognize its peer and exchange several packets (including transforms and keys).

IPSec negotiation
Once ISAKMP is finished, IPSec will attempt to negotiate its SA (Security Association). This process is similar to ISAKMP negotiation. If your vendor's equipment is capable of showing the details of this process, check to make sure its SA is using the correct source and destination IP addresses, the proper encryption protocols, and be sure that the SA lifetimes match. The SA lifetime can be expressed in either time or the amount of traffic passed.

Traffic classification
Finally, in your configuration, you typically specify what traffic to encrypt. Make sure this is correct and that any routing you might be doing through the encrypted tunnel is correct.

Also, be careful to include your testing tools in this configuration. For instance, if you want to test connectivity through your encrypted tunnel using PING, you need to let the system know to encrypt the ICMP traffic.

Thomas Alexander Lancaster IV is a consultant and author with over ten years experience in the networking industry, focused on Internet infrastructure.


This was first published in May 2002

Join the conversationComment

Share
Comments

    Results

    Contribute to the conversation

    All fields are required. Comments will appear at the bottom of the article.

    Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.

    Back to top