Problem solve Get help with specific problems with your technologies, process and projects.

VPN operating system interoperability -- Configure VPNs with Linux

Learn how to configure VPNs on Linux in the second tip in our series on VPN operating system interoperability.

There are several ways to implement a VPN on your Linux servers. This is the second tip in our series on VPN operating system interoperability. In the previous tip, Configure VPNs with Unix, we looked at how to configure two Unix servers.

In this part, we'll discuss using VPNs on Linux. Some of these solutions will depend on the Linux distribution that you are using.

IPsec for GNU/Linux -- provided by FreeS/WAN

This popular implementation of IPsec for GNU/Linux implementation is available for Linux kernels 2.2 and later (they provide a version for the 2.6 kernel). This is a free software-based VPN solution that uses a standardized Internet Protocol security (IPsec) implementation.

More on VPNs
Virtual Private Network defined

IPsec defined

Browse tips on VPNs and Wide Area Networks


You can start with OpenVPN, which is a full-featured SSL VPN solution. Unlike other SSL VPN solutions, OpenVPN needs an installed OpenVPN client on remote machines. It is not standardized (which means you have to stick to one solution both on client and server). It also needs a client application, though it is supported on many platforms, including Windows. It is very easy to configure; this is done using a tun/tap device, so it automatically builds a tunnel between the client and the server.

RHEL IPsec using IKE

Each Linux distribution has its own way of doing things. Red Hat Enterprise Linux supports IPsec for connecting remote hosts and networks to each other using a secure tunnel on a common carrier network such as the Internet. IPsec can be implemented using either a host-to-host (PC-to-PC) or network-to-network (one LAN/WAN to another LAN/WAN). Red Hat's IPsec implementation uses Internet Key Exchange (IKE), which is a protocol implemented by the Internet Engineering Task Force (IETF). It is used for mutual authentication and secure associations between connecting systems.

Here's how it works: Essentially, an IPsec connection is split into two logical phases.

  • Phase 1
    An IPsec node initializes the connection with either the remote node or network. In the case of the remote node, it would check the requesting node's credentials. Both parties would then negotiate the authentication method for the connection. The IPsec connection uses the pre-shared key method of IPsec node authentication. In a pre-shared key IPsec connection, each of the hosts must use the same key in order to move to the second phase of the IPsec connection.
  • Phase 2
    This is where the security association (SA) is created between IPsec nodes. This phase actually establishes an SA database with configuration type information. This includes the encryption method and the secret session key exchange parameters. This phase manages the actual IPsec connection between the remote nodes and networks.

Red Hat's implementation of IPsec uses IKE for sharing keys between hosts across the Internet.


In order to implement IPsec in Red Hat, the ipsec-tools RPM package must be installed on all IPsec hosts or routers, depending on whether this is a host-based or network-based design. There are two ways to configure IPsec on Red Hat. One is to use the GUI (Network Administration Tool), and the other is to edit networking and IPsec configuration files manually. We detail the GUI steps in this article.

The first step in creating a connection is the information-gathering stage. You'll need the IP addresses, a unique name to establish the connection, a fixed encryption key and a pre-shared authentication key, which is used to initiate and encrypt keys during the session.

The steps are as follows:

  1. Start the GUI (Network Administration Tool).
  2. Look for the IPsec tab and select New.
  3. Click the Forward button to start configuring the actual IPsec connection.
  4. Provide the name (I.E. "ipsec0") for the connection, and select whether the connection should be automatically activated when the computer starts. Then click Forward.
  5. Since we're doing host-to-host select, select Host to Host encryption as the connection type. Click Forward.
  6. Select the type of encryption to use. Your choices are manual or automatic.

    If you select "manual," then an encryption key must be provided later in the process. If "automatic" is selected, a system daemon (raccoon) is used to manage the actual encryption key. Here is where you would need the ipsec-tools package to be installable.

    Click Forward to continue.

  7. Specify the IP address of the other host.

    Click Forward to continue.

  8. If manual encryption was selected in step 6, you must now specify the encryption key or click Generate to create one. When completed, click Forward to continue.
  9. Verify that the information on the Ipsec is correct on the summary page and click Apply.
  10. Save your configuration.
  11. Select the IPsec connection from the list and click Activate.

At this point, you must follow the same steps for your other host. Don't forget to use the same keys from step 8 on the other hosts; otherwise, IPsec won't work.

About the author:
Ken Milberg is the founder of Unix-Linux Solutions. He is also a board member of Unigroup of NY, the oldest Unix user group in N.Y.C. Ken regularly answers user questions on Unix and Linux interoperability issues as a site expert on

This was last published in January 2007

Dig Deeper on Network Security

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.