VPNs and VPN Technologies

Chapter Description

This sample chapter defines virtual private networks (VPNs) and explores fundamental Internet Protocol Security (IPSec) technologies. This chapter also covers IPSec crypto components, an overview of IKE, IPSec security, and a certificate authority (CA) support overview.

IPSec Crypto Components

This section covers in detail the component technologies used in IPSec. This section covers the following:

  • DES encryption
  • Diffie-Hellman (D-H) key agreement
  • HMAC

DES Encryption

The components of DES encryption are as follows:

  • Encryption and decryption algorithms
  • Matching shared secret keys on each peer
  • Input clear text data to be encrypted

At the core of DES is the encryption algorithm. A shared secret key is input to the algorithm. Clear text data is fed into the algorithm in fixed-length blocks and is converted to ciphertext. The ciphertext is transmitted to the IPSec peer using ESP. The peer receives the ESP packet, extracts the ciphertext, runs it through the decryption algorithm, and outputs clear text identical to that input on the encrypting peer. The DES encryption algorithm can be seen in action in Figure 1-10, where a preshared key is in use.

Figure 1-10 DES Encryption

Diffie-Hellman Key Agreement

The Diffie-Hellman (D-H) key agreement is a public key encryption method that provides a way for two IPSec peers to establish a shared secret key that only they know, although they are communicating over an insecure channel.

With D-H, each peer generates a public and private key pair. The private key generated by each peer is kept secret and never shared. The public key is calculated from the private key by each peer and is exchanged over the insecure channel. Each peer combines the other's public key with its own private key and computes the same shared secret number. The shared secret number is then converted into a shared secret key. The shared secret key is never exchanged over the insecure channel.

As you can see in Figure 1-11, Diffie-Hellman key exchange is a complicated process. This adds to the effectiveness of the encryption algorithm.

Figure 1-11 Diffie-Hellman Key Agreement


Diffie-Hellman is very important because the shared secret key is used to encrypt data using the secret key encryption algorithms specified in the IPSec security associations, such as DES or MD5.

The Diffie-Hellman Process

The Diffie-Hellman process is as follows:

  Step 1 The D-H process starts with each peer generating a large prime integer, p and q. Each peer sends the other its prime integer over the insecure channel. For example, Peer A sends p to Peer B. Each peer then uses the p and q values to generate g, a primitive root of p.
  Step 2 Each peer generates a private D-H key (Peer A: Xa, Peer B: Xb).
  Step 3 Each peer generates a public D-H key. The local private key is combined with the prime number p and the primitive root g in each peer to generate a public key: Ya for Peer A and Yb for Peer B. The formula for Peer A is Ya =g^Xa mod p. The formula for Peer B is Yb =g^Xb mod p. The exponentiation is computationally expensive. The ^ character denotes exponentiation (g^Xa is g to the Xa power); mod denotes modulus.
  Step 4 The public keys Ya and Yb are exchanged in public.
  Step 5 Each peer generates a shared secret number (ZZ) by combining the public key received from the opposite peer with its own private key. The formula for Peer A is ZZ=(YbXa) mod p. The formula for Peer B is ZZ=(YaXb) mod p. The ZZ values are identical in each peer. Anyone who knows p or g, or the D-H public keys, cannot guess or easily calculate the shared secret value largely because of the difficulty in factoring large prime numbers.
  Step 6 Shared secret keys are derived from the shared secret number ZZ for use by DES or HMACs.


Each IPSec peer has three keys:

  • A private key that is kept secret and is never shared—It is used to sign messages.

  • A public key that is shared—It is used by others to verify a signature.

  • A shared secret key that is used to encrypt data using an encryption algorithm (DES, MD5, and so on)—The shared secret key is derived from Diffie-Hellman key generation.



The fundamental hash algorithms used by IPSec are the cryptographically secure MD5 and SHA-1 hash functions. Hashing algorithms have evolved into HMACs, which combine the proven security of hashing algorithms with additional cryptographic functions. The hash produced is encrypted with the sender's private key, resulting in a keyed checksum as output.

In Figure 1-12, the hash function takes as input the variable-length clear text data that needs to be authenticated and a private key. The private key length is the same as that of the output of the hash. The HMAC algorithm is run, with a resultant fixed-length checksum as output. This checksum value is sent with the message as a signature. The receiving peer runs an HMAC on the same message data that was input at the sender, using the same private key, and the resultant hash is compared with the received hash, which should exactly match.

Figure 1-12 Hashed Message Authentication Codes (HMAC)


The HMAC-MD5-96 (also known as HMAC-MD5) encryption technique is used by IPSec to ensure that a message has not been altered. HMAC-MD5 uses the MD5 hash developed by Ronald Rivest of the Massachusetts Institute of Technology and RSA Data Security Incorporated and is described in RFC 1321.

HMAC-MD5 uses a 128-bit secret key. It produces a 128-bit authenticator value. This 128-bit value is truncated to the first 96 bits. Upon sending, the truncated value is stored within the authenticator field of AH or ESP-HMAC. Upon receipt, the entire 128-bit value is computed, and the first 96 bits are compared to the value stored in the authenticator field.

MD5 alone has recently been shown to be vulnerable to collision search attacks. This attack and other currently known weaknesses of MD5 do not compromise the use of MD5 within HMAC, as no known attacks against HMAC-MD5 have been proven. HMAC-MD5 is recommended where the superior performance of MD5 over SHA-1 is important.


The HMAC-SHA-1-96 (also known as HMAC-SHA-1) encryption technique is used by IPSec to ensure that a message has not been altered. HMAC-SHA-1 uses the SHA-1 specified in FIPS-190-1, combined with HMAC (as per RFC 2104), and is described in RFC 2404.

HMAC-SHA-1 uses a 160-bit secret key. It produces a 160-bit authenticator value. This 160-bit value is truncated to the first 96 bits. Upon sending, the truncated value is stored within the authenticator field of AH or ESP-HMAC. Upon receipt, the entire 160-bit value is computed and the first 96 bits are compared to the value stored in the authenticator field.

SHA-1 is considered cryptographically stronger that MD5, yet it takes more CPU cycles to compute. HMAC-SHA-1 is recommended where the slightly superior security of SHA-1 over MD5 is important.

4. IKE Overview | Next Section Previous Section