Linux and network encryption

By Kurt Seifried, [email protected]


September 8, 1999 – And now for the last in my three part mini-series on Linux encryption; network encryption. We've covered the basics, and filesystem encryption, however these systems are absolutely no good if you log into your server via telnet, and then provide the password to mount your encrypted home directory. There are also several file encryption systems that do not lend themselves well to networking, and many file sharing methods that provide no encryption at all. Encrypting the data that moves across your network is a simple and effective answer (ok, it's probably not simple, but you get the idea).

The basics

There are several levels at which you can encrypt data in a network setting, so far we have only dealt with methods at the application and presentation layer. That is to say the encryption is provided by software and not really integrated with the network (TCFS being a notable exception). Encryption can be done at almost any layer of the OSI stack, with various benefits and drawbacks to each method.

OSI layer Description Example
Application Software program, creates the data. Email Program
Presentation Interaction with the OS, character set, etc. Message data
Session Session setup, logging, etc. Sockets, built into the OS (SSL).
Transport Actual protocol, insures data integrity, etc. TCP, built into the OS
Network Routing of data IP, a router
Datalink Node to node communications Network card, modem
Physical Physical wire/fiber Cat 5, single mode fiber


For this article we are concerned with network based encryption, which typically happens at the session, and / or transport layer (green). You typically don't encrypt the network (IP) layer as the routers/etc along the path must be able to view some data in the packet (like destination). Encryption can also be done at the application layer (PGP), presentation (X.509 integration with Netscape mailer) which was discussed in my previous article, or at the datalink layer (modems with pre-shared secrets and hardware encryption chips) which are in (blue).

Session layer encryption (SSL)

Session layer encryption can be achieved with a variety of add-on protocols. The most popular is SSL (Secure Sockets Layer), which was originally developed for web based commerce (but like any good solution can be used elsewhere). SSL provides authentication and encryption, with the use of public key encryption and certificates. Like any public key encryption technology it is susceptible to man in the middle attacks, which are essentially caused by the fact you have to create a secured communications channel over an insecure network. To solve this SSL supports the use of certificates, which are blocks of information about a party (for example a www server certificate includes the domain name, issuer, expiry date, and so on) that is signed by a trusted third party. This still requires the distribution of public keys for the trusted third party, which is typically achieved by shipping the keys with the application. During the course of negotiating a session on party (say the client, a www browser) can ask the other party (say a www server) to prove it's identity. In which case the second party would send the first party it's digital certificate. This can go both ways, you can buy personal certificate from Verisign, and after proving who you are, you will be issued a form of digital identification that you can use to prove who you are to other people (assuming they trust Verisign). 

SSL is relatively easy to implement in applications, there is a good set of SSL libraries (OpenSSL) freely available to use. For example you can use Slush which is Telnet with SSL support built in, and instead of using a username and password to login, you use a digital certificate as proof of identification. This can also be done with POP, and IMAP servers (Microsoft and Netscape both support SSL'ified POP/IMAP out of the box) to prevent user's passwords from going across the network in plain text. 

SSL also lends itself to PKI (Public Key Infrastructure), and is scaleable. For example your company might buy a certificate from Verisign so that it in turn can sign certificates for departments. These departments could then sign your personal certificate and server certificates, so when you communicate with another employee or departmental server you can be sure it is who it claims to be. Unfortunately the costs of PKI (setup and maintenance) are still very high, and certificates are not very portable, which has lead to slow acceptance.

SSL is appropriate when you have many short lived connections (www sessions, pop sessions, etc) between parties that are unknown to each other. Since SSL supports certificates, assuming you trust the issuer of the certificate you can for instance browse on (which uses a Thawte site certificate) and be relatively sure you are in fact browsing on the real site and not a fake one.

Network layer encryption (IPSec)

Network layer encryption is best done with IPSec (in my opinion) but can be done with a variety of other tools. IPSec is set to become an industry standard, and is already supported on most major platforms (Linux, OpenBSD, Sun, Windows and AIX to name a few). Like SSL IPSec supports a variety of different encryption algorithms, however the one used in FreeS/WAN (the Linux IPSec implementation) is typically 3DES. Currently IPSec for Linux (and most other implementations) do not support the use of certificates to prove identity, this is a planned feature. Despite these drawbacks IPSec is still a viable solution for encrypting network traffic.

To prevent man in the middle attacks you have a pre-shared secret between the two (or more) parties participating in that IPSec configuration. This secret is usually 256 bits, meaning there are 2^256 possible secrets, making a man in the middle attack difficult at best. In addition to this the connection can be automatically re-keyed (the default being 8 hours). If someone manages to break the session key (168 bits for 3DES), they can only snoop in on the traffic for that session. 

One major benefit of IPSec is that it is implemented at the network layer, which the OS handles, so applications need not be aware at all that any encryption is going on. Additionally most operating systems support it, there is even a free client for Windows 95/98/NT in the form of PGPi. Windows 2000 is listed as having IPSec support, but I have not yet tried it so I cannot comment much on it. 

IPSec is an ideal solution if you have heavy data traffic between several points (currently the lack of certificate support makes managing large installations painful). Another widespread use of IPSec is for connecting networks to each other over insecure network (i.e. the Internet). Simply put a Linux box at each location and you can now talk securely.


Reference links:


SSL Documentation: 

SSL Libraries:

SSL Apps for Linux:


Other network encryption programs:



Last updated 7/7/2002

Copyright Kurt Seifried 2002