Problem solve Get help with specific problems with your technologies, process and projects.

VPN operating system interoperability -- configure VPNs with Unix

Learn how to configure IPsec VPNs on various platforms and ensure interoperability. In our first example, we configure two IBM pSeries Unix (AIX) servers.

In this series, we'll discuss how to configure VPNs on various platforms -- including Unix, Linux and Windows -- while providing interoperability tips and tricks. For the purposes of the series, we'll focus on IPsec VPNs. In our first example, we'll configure two IBM pSeries Unix servers. IBM's Unix is AIX.

Here are the steps we'll follow to implement a VPN in our AIX environment:

  1. Identify and install the IPsec VPN filesets on AIX
  2. Create a VPN IPsec tunnel between two pSeries servers
  3. Activate and monitor the IPsec tunnels

More on VPNs
Virtual Private Network defined

IPsec defined

Browse tips on VPNs and Wide Area Networks
Let's start with the filesets (the equivalent of RPMs in Linux or packages in Solaris). In AIX, one needs to install specific filesets that are not part of the base distribution of AIX. This is necessary to create VPN IPsec tunnels. You can do this using either the System Management Interface Tool (SMIT) or installp. You can verify whether these filesets are present by running the command:

# lslpp –f fileset

The AIX filesets required include:

  • bos.crypto-priv (from the AIX expansion pack)
  • bos.rte.ipsec.rte (AIX CD)
  • bos.rte.ipsec.keymgt (AIX CD)
  • bos.rte.ipsec.websm (AIX CD)

You can configure Internet Key Exchange (IKE) tunnels using the Web-based System Manager tool (using the wsm command), SMIT or the command line.

Usually, the easiest way to configure IKE tunnels is via the IKE tunnel wizards in WSM (though most AIX admins despise WSM with a passion). I usually prefer SMIT to WSM as an interface because it is not a GUI, but you still have to edit many xml files, so this solution is not ideal either. Using SMIT, its fastpath would be smit ike4 (IPv4). The database functions are found in the Advanced IP Security Configuration menu.

Using WSM, one can use the GUI basic configuration wizard, but only for the initial configuration. To configure using WSM, one need only:

  1. Select "network plug-in"
  2. Select "VPN networks"
  3. Select "overview and tasks folder"
  4. Select "configure a basic tunnel config wizard"

At this point, you follow the steps in the wizard to configure the IKE tunnel. After it is defined, it displays on the tunnels list and can be either activated or modified.

Using the command line, one must employ the following procedure, assuming there is a tunnel identifier:

# Ike cmd=activate numlist=1

One can also use the IP address:

# ike cmd=activate ipaddr=

To display status:

# ike cmd=list

One should see an output showing that the tunnels are active:

Phase 1 Tunnel ID [1]
Phase 2 Tunnel ID [1]

To view tunnel definitions, one would use the ikedb command:

# ikedb –g

I'm often asked how AIX interoperates with Linux-based VPNs. A popular implementation of IPsec for GNU/Linux implementation for Linux is provided by FreeS/WAN and is available for Linux kernels 2.2 and later (they provide a version for the 2.6 kernel). It provides a subset of the IPsec functionality that AIX has. There is a conversion utility available through the ikedb command that helps interoperability with Linux. Using the -c option, one can define tunnels between Linux and AIX on the Linux systems. Instead of using the ike activate command, one would use this command to activate the tunnel from Linux:

# ipsec auto –up aix

Regardless of which method you use, you will always need to start your daemons. You can start the AIX daemons using the System Resource Controller (SRC) facility in AIX.

# startsrc –k ike

This will start it, and stopsrc (using the same options) will stop it.

We'll go more deeply into Linux in the next tip.

About the author:
Ken Milberg is the founder of Unix-Linux Solutions and a board member of Unigroup of NY, the oldest Unix user group in NYC. Ken regularly answers user questions on Unix and Linux interoperability issues as a site expert on

This was last published in December 2006

Dig Deeper on Network Security