Linux Mail Server Setup

Foreword

As system administrator I have had the privilege of working with all sorts of linux servers and services in a variety of distributions. Here I present two configurations I have used in most of my installations.

The first one is based on qmail SMTP with virtual domain support with VPopmail/MySQL on RedHat ES. This configuration I used mostly as a system admin on a cable modem company that handled thousands of residential users and hundreds of commercial users (i.e. with their own email domains). I can say this setup is scalable in terms of users but may present some problems with large queues caused by large traffic bursts. Also due to the maildir format I had some problems with inode exhausting but this is a file system problem (i.e. using XFS instead of ext3 could have avoided this problem).

The second configuration is more updated with current standards. It uses the more easy to configure and secure Postfix for SMTP and Courier-POP for POP. It could have used other POP servers like Dovecot. I have tested this configuration on a small laboratory with 30 users so no idea on how scalable it may be.

I do not support virtual domains in this configuration because I consider current solutions to email virtual domains confusing. With the virtualization hype these days I believe SMTP servers should not handle virtual mail domains. We should install several virtual machines in a single hardware and manage each mail domain with a fully functional SMTP server in each of the virtual machines.

Redhat Eterprise Server (ES) Mail setup

Pros:

• Fully functional qmail SMTP and POP services.


• Supports virtual domains with VPopmail/MySQL.


• Maildir storage format.

Cons:

• Installation from source code and requires some source modifications.


• Rather complicated manual configurations.


• No support for encrypted communications. This limits the server to local network or locahost connections (i.e. Webmail client).


• Roaming support uses the outdated POP-Before-SMTP.

Instructions:

• Setup qmail server
  First we install the qmail SMTP server on RedHat ES 4 from source code. Due to some incompatibilities with the version of glibc that comes with RedHat we are forced to modify the source code a little to get it to compile.


• Setting TCPSERVE
  We use tcpserve to handle all incoming connections to the qmail SMTP port (25). With tcpserve we can control access to the port and allow/deny relaying of emails to users depending on IP address. This access control is needed to avoid spammers from using our server for free.


• VPopmail
  We end our setup by installing Vpopmail to support virtual domains managed with MySQL backend. We configure Vpopmail to support maildirs and to manage roaming. Vpopmail modifies the tcpserve tcp rules files to allow POP authenticated users to relay mail (POP-Before-SMTP).
  

Ubuntu Mail Server Setup (Kubuntu Feisty Fawn/Ubuntu Server)

Pros:

• Fully functional Postfix SMTP and Courier-POP services.


• Installation and configuration is a breeze (soooo easy).


• Maildir storage format.


• Roaming using SMTP-AUTH.


• Encrypted transmission for SMTP (TLS) and POP (SSL).

Cons:

• No support for virtual domains.

Instructions:

• Installing the SMTP server
  First we install and configure a fully functional Postfix server and enable SMTP-AUTH for roaming with SASL authentication.
  
• Installing the POP sever
  Next we install the Courier-POP server with Maildir storage support.


• Enable Encrypted Communications
  Now we enable secure transmission by encryption of SMTP (TLS) and POP (SSL) traffic. We create self signed certificates and enable Postfix and Courier-POP to use them.


• Testing
  Finally some pointers on how to test SMTP/POP services using a console window via telnet/openssl commands.