Understanding conflicts between Network/Port Address Translation and IPsec VPNs is definitely a challenge. IPsec is designed to detect and discard any change to encapsulated, encrypted packets. NAT does just that - it changes outgoing packets by mapping private source IP address and port to the firewall's public WAN IP address and a unique port.
When responses are received, NAT must map each one back to the original private IP/port to reach the client. Problems here include:
1) In some IPsec flavors, any change to the source IP invalidates the IPsec integrity check value carried within the packet, so forget it.
2) In IPsec ESP tunnel mode, the source IP in the "outer packet" can be changed without invalidating the integrity check value. However, the content of the original packet - including the TCP/UDP source port - is obscured by encryption. So NAT can't do its job unless the firewall implements what is commonly referred to as a "pass-through." Firewalls with VPN pass-through can often forward IPsec-encrypted packets without breaking them.
3) But IPsec tunnels don't just happen magically - they get set up by a companion protocol called the Internet Key Exchange (IKE). VPN clients send IKE packets on UDP port 500 to authenticate, negotiate security parameters, and establish IPsec tunnels (security associations). There are several issues here that I won't get into, but you can learn more about these problems and proposed "NAT Traversal" solutions by reading these Internet Drafts:
The upshot is that different VPN products support different variations of this NAT Traversal solution to allow IPsec and IKE to pass safely through NAT devices. Both of the D-Link products that you mention appear to have some trouble with Nortel's implementation of NAT Traversal. A stateful packet inspection (SPI) firewall uses more sophisticated algorithms to determine when to get rid of UDP pseudo-sessions and associated NAT mappings. Just guessing now, but this may be why you are having a harder time with the 714P+ than the 611, which provides only simple NAT, no stateful inspection. In general, NAT traversal is NOT SUPPOSED TO require changes to NAT devices in between (like these two wireless routers). In practice, as you have found, things don't always work like they should.
This was first published in March 2004