Virtual Domains with Vpopmail on a qmail server.

In previous post I explained how to install a qmail server on a Red Hat ES machine and how to manage the server connections via tcpserver. Now I will add support to virtual domains using vpopmail.

This guide is based on this one with some modifications to fit my needs. The usual disclaimer is that everything written here that is good is from that guide while the bad tidbits are mine.

Preparing Red Hat ES for vpopmail

To ease the work I chose to use the rpm packages that come with Red Hat by default. First we install the MySQL client and server packages that are included on the Red Hat installation CD's.

# rpm -ivh mysql-4.1.7-4.RHEL4.1 mysqlclient10-3.23.58-4.RHEL4.1 mysql-server-4.1.7-4.RHEL4.1

We also install the devel package needed to compile Vpopmail with MySQL support:

# rpm -ivh mysql-devel-4.1.7-4.RHEL4.1

The default installation of MySQL server has a black password for the root account. This of course is a security risk and we must assign a password to the root account as soon as possible.

First start the MySQL server

# /etc/init.d/mysql start

Now we setup a root password

# mysqladmin -u root password 'mysql-root-pwd'

Before we continue to install Vpopmail we need to install the zlib-devel package but I could not find it in any of the Red Hat CD's. To install it I downloaded a zlib-devel rpm from here and installed it. You can use wget to download the file.

# rpm -ivh zlib-devel-1.2.1.2-1.i386.rpm

Installing Vpopmail

We need some preparations before we can compile and install Vpopmail on the machine. First we create a vchkpw group and vpopmail user with home directory set to /home/vpopmail.

# groupadd -g 89 vchkpw
# useradd -g vchkpw -u 89 -d /home/vpopmail vpopmail

Now we create a vpopmail database on the MySQL server and assign a user and password in the database to access it.

# mysql -u root --password="mysql-root-pwd"

# mysql> CREATE DATABASE vpopmail;
# mysql> GRANT select,insert,update,delete,create,drop ON vpopmail.* TO vpopmailuser@localhost IDENTIFIED BY 'vpoppasswd';
# mysql> quit

In the above command we created a database called vpopmail that will store all the vpopmail domains/users/log/config etc. Then we create a user "vpopmailuser" with password "vpoppasswd" with all priviledges on the vpopmail database. Please feel free to change the user and password to your favorite ones.

Next we create a configuration file that vpopmail processes will use to connect to the MySQL server:

# mkdir ~vpopmail/etc
# chown vpopmail.vchkpw ~vpopmail/etc
# echo "localhost|0|vpopmailuser|vpoppasswd|vpopmail" > ~vpopmail/etc/vpopmail.mysql
# chown vpopmail.vchkpw ~vpopmail/etc/vpopmail.mysql
# chmod 640 ~vpopmail/etc/vpopmail.mysql

Don't copy and paste the echo command above. You must change the vpopmailuser and vpoppasswd to reflect the username and password you configured on the database.

Now we are ready to install Vpopmail on the machine. Download the source package and decompress it in anywhere you feel like:

# wget http://kent.dl.sourceforge.net/sourceforge/vpopmail/vpopmail-5.4.13.tar.gz
# tar xvfz vpopmail-5.4.13.tar.gz
# cd vpopmail-5.4.13

Now we configure the source for our needs. Before running the configuration script make sure you know what features you want enabled/disabled. Once we compile changing a feature will require you to reconfigure and recompile the source code. In my case I am interested mostly on roaming and MySQL support:

# ./configure --enable-roaming-users \ # Enable POP-before-SMTP functionality.
--enable-logging=p \ # Log to syslog errors with passwords
--disable-passwd \ # Disable /etc/passwd (or shadow) accounts
--enable-auth-module=mysql \ # Enable MySQL backend support
--enable-auth-logging \ # Record time and ip of last auth attempt
--enable-sql-logging \ # Enable authentication logging to MySQL/Postgres.
--enable-valias \ # Store email aliases in MySQL
--enable-mysql-limits \ # Use MySQL to store limits.
--enable-many-domains # Store all virtual domain users in a single table.

To learn about other options you can always use the command:

# ./configure --help | less

Finally we simply compile by calling the make command:

make
make install-strip

That's it for installing Vpopmail. Now we can create/delete virtual domains and users within those domains using all Vpopmail commands found in the /home/vpopmail/bin directory:

To add a domain :

/home/vpopmail/bin/vadddomain yourdomain.com yourpassword

To add a mailbox:

/home/vpopmail/bin/vadduser someone@yourdomain.com apassword

To remove a mailbox

/home/vpopmail/bin/vdeluser someone@yourdomain.com

To remove a domain :

/home/vpopmail/bin/vdeldomain yourdomain.com

To change a user's password

/home/vpopmail/bin/vpasswd someone@yourdomain.com newpassword

(Or you can do it via qmailadmin)

To lookup info about a user

/home/vpopmail/bin/vuserinfo someone@yourdomain.com

Configureing tcpserver to manage POP3 connections

As with qmail SMTP we would like our clients to connect to the pop server from anywhere in the world. To do so we setup tcpserver in the qmail init script to accept tcp connections to port 110 and excecute the qmail-pop3d command. To do this simply add the next line after the tcpserver invocation for qmail-smtpd inside the start() method

/usr/local/bin/tcpserver -H -R -l test.canmail.jp -v 0 pop3 var/qmail/bin/qmail-popup test.canmail.jp /home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir 2>&1 | /var/qmail/bin/splogger pop3d 3 &

Save the qmail init script and restart the service:

#/etc/init.d/qmail restart

Testing SMTP and POP3 via Telnet

Since SMTP and POP are text based protocols we can easily test them via telnet.

First let's create a test domain and a user:

First create the virtual domain. This command will also create a postmaster account and ask you for a password.

# /home/vpopmail/bin/vadddomain testdomain.jp

Next create a test account in the newly create domain

# /home/vpopmail/bin/vadduser testuser@testdomain.jp

Let's try first sending emails to the testuser using our newly installed qmail SMTP. In the next examples the blue lines are our commands and the red ones are the server responses.

From any machine different from the server using a telnet client:
# telnet server_ip_address 25
Trying 218.45.218.139...
Connected to 218.45.218.139.
Escape character is '^]'.
220 example.jp ESMTP
helo example.jp
250 example.jp
mail from: anyaccount@anydomain.jp
250 ok
rcpt to: testuser@testdomain.jp
250 ok
data
354 go ahead
hello world
.
250 ok 1159529928 qp 17735
quit
221 example.jp
Connection closed by foreign host.

With the above commands we send an email to the testuser at the testdomain in the server. You can see the message was delivered as the server responded with a 250 ok message.

Now let's do the same but send a message to a remote account. For example a gmail account.

From any machine different from the server using a telnet client:
# telnet server_ip_address 25
Trying 218.45.218.139...
Connected to 218.45.218.139.
Escape character is '^]'.
220 example.jp ESMTP
helo example.jp
250 example.jp
mail from: testuser@testdomain.jp
250 ok
rcpt to: anyaccount@gmail.com
553 sorry, that domain isn't in my list of allowed rcpthosts (#5.7.1)
quit
Connection closed by foreign host.

The server did not allow us to send the message because the gmail.com domain is not configured on the list of rcpthosts of qmail. This is very good since we don't want anyone using our server so relay email (specially spam). In the previous example the message was delivered because the testdomain.jp was added to the rcpthosts list of qmail by the vadddomain command.

But then how can the legitimate users that exist in our server send emails to other domains that are not in the rcpthosts list?? Is not like our users are going to send email only between them is it??. Well that is why I configured Vpopmail with roaming support. With roaming support after a user connects to the POP server, Vpopmail records the IP address of the machine the user connected from and modifies the tcp.smtp file to allow the recorded IP address to relay emails to any other domain.

This is called POP-before-SMTP since to be able to send emails with the server the user first needs to connect at least once with the POP server. Let's try by connecting to the POP server via telnet:

telnet 218.45.218.139 110
Trying 218.45.218.139...
Connected to 218.45.218.139.
Escape character is '^]'.
+OK <17844.1159530536@example.jp>
user testuser@testdomain.jp
+OK
pass testuserpass
+OK
list
+OK
1 269
.
quit
+OK
Connection closed by foreign host.

The list command ask the server to list the testuser emails and as we can see the listing shows one new email in the inbox. This is the email we sent on the first test. After this the IP address of the machine we are running the telnet client must be recorded on the tcp.smtp database and if we retry the second example the email will be accepted for delivery this time. Go ahead.. you can try.

Warning: Check your IPTables

When I did remote tests the first time I could not connect remotely to the port 110 on the server. After some time I realized that Red Hat ES by default has some very strict iptable rules (Firewall) set that would not allow remote POP connections.

The easiest way to allow remote SMTP (port 25) and POP3 (port 110) connections is to modify the iptables configuration file in the /etc/sysconfig directory.

# nano /etc/sysconfig/iptables

# Firewall configuration written by system-config-securitylevel
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 25 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 110 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

Make sure the lines in red are in the file and then restart the iptables service:

# /etc/init.d/iptables restart

Final Notes

If you want a first class email server then DO NOT USE this guide. This configuration is the old school of email server configurations for a medium size application with not so much users. Current email servers use SMTP-Auth instead of POP-before-SMTP and some sort of encryption like TLS for SMTP and SSL for POP.

For a real email server that comply with current standards you should look at Cyrus-SASL for SMTP and POP encryption, SMTP-Auth for qmail and Cyrus-IMAP.

Why I still use this old school configuration?? Ask my boss... This email server is to replace a legacy server that has been working for years. Maybe changing the way it works can be annoying to the current users so I was asked to replicate exactly the old server.

Google Talk on Kopete

So I have been using Kopete as a Google Talk client for a looooonnngggg time but a little accident (Human error of course) I got my Kopete configuration files gone. Well it is not big deal as setting my accounts again is very easy. MSN and Yahoo are supported by default and is easy to remember how to set them again but Google Talk is just a little more difficult to remember as it is actually a Jabber account you must set up. There is not such thing as Google Talk on Kopete but since it uses the Jabber protocol (XMPP) we can use the Jabber plugin to connect to Google Talk server.

Setting up Kopete

This is very simple in fact simply create a new Jabber account in Kopete and set the Jabber ID to your be your Gmail account and the password to your Gmail accout password.

Jabber ID: my_nick@gmail.com
password: my_gmail_password

Next go to the Connection tab and check all options and change the server to talk.google.com

[x] Use protocol encryption (SSL)
[x] Allow plain-text password authentication
[x] Override default server information

Server: talk.google.com
Port: 5223

Finished!! now I can use Kopete to chat with all my Google Talk friends. Next the difficult part of this is to convince all my 100 MSN contacts to switch to Gmail.

Notes on the TLS thing

Google talk requires the connection to be encrypted to be able to connect to the server. This is good for privacy and security reasons. To support encrypted connections Kopete uses a library called QCA so we must make sure Kopete has access to this library before we can use it as a Google Talk client.

In Kubuntu the QCA library package is called qca-tls and it seems to be installed by default. If not make sure you install this library when installing kopete.

sudo aptitude install qca-tls kopete

In other distros it may be similar. More details of these can be found at the KDE wiki page.

Accessing a machine behind a NAT using SSH remote port fowarding

Example situation (Real one)

My sister is not what you call a computer power user but she can use Windows XP and do some basic stuff (i.e. play flash games and listen MP3's). Now after years of phycological brain washing I finally convinced her to try a linux distro. Her choice is Ubuntu and she already installed it and has played a little with it.

Unfortunatelly not everything is perfect and now she has some problems basically because she messed up the sources.list of apt-get and has installed some non-standard packages downloaded directly from the Internet.

I have been trying to diagnose the problem but since I live at the other side of the world, literally, I can only see what she tells me and only give guesses on how to fix it via email and chat. As this was taking too much time the next step would be to login via ssh to her machine and find what is going on by myself, BUT, she is behind a firewall with NAT so she has a private IP address. She can access my PC via ssh but I cannot access hers. The solution?? the poweful ssh remote port forwarding.

Little Background

With SSH it is possible to forward any kind of traffic from one machine to another using port forwarding (tunneling). There are two kinds of port forwarding: remote port forwarding and local port forwarding. Both acomplish the same thing and differ only on the direction of the tunnel.

Local Port Forwarding

For the situation I presented above this method of port forwarding is not usefull. Actually I haven't found yet a good application for local port forwarding apart from creating a secure tunnel to transfer unencrypted data from my machine to another one. The idea is to create a tunnel between a port in my machine to a port in a remote machine. This way any connection to the port in my machine will be fowarded the the port in the remote machine.

As I said this is not usefull to solve my problem so I will not discuss this here. For some reference read the ssh man page and this link

Remote Port Forwarding

Let's review the situation: I need to connect to my sister's machine via ssh but she is behind a NAT so her machine is not accessible from the Internet. In the other hand she can access my machine from hers via ssh. The problem here is how can I get access to my sister's machine from mine?? simply ask my sister to create a tunnel between her machine and mine using ssh tunneling. This way I can simply connect to the tunnel port in my machine and ssh will forward it to her's.

How is this done?? with a simple command;

ssh -R 2222:localhost:22 my_ip_address

This is like a normal ssh connect command but with the -R switch that creates a tunnel. My sister has to run this command in her machine replacing my_ip_address with the IP address of my machine. After my sister excecutes the above command the tunnel created is like the one in the next image:

Everytime I want to connect to my sister's machine via ssh I simply connect to port 2222 on my own machine and magically I will be inside my sister's machine.

ssh -l sister_user -p 2222 localhost

Note that I am connecting via ssh to the port 2222 on my own machine, since this is a tunnel that my sister created it will be fowarded to the port 22 on my sister's machine where her SSHD server should be listening. This way I effectively get connected to her machine via ssh even if she is behind a NAT/Firewall or whatever.

Some additional notes

- First the sshd daemon must be running in port 22 on BOTH machines. This way my sister can connect via ssh to my machine and I can connect to hers via the tunnel.

- The port used in the remote machine (i.e. 2222) must be free for use, that is my machine must not be using that port. You can use any port but to use priviledges ports (< 1024) you must connect as root on the remote machine.

- By default this tunnel will be bound to the local address (i.e. 127.0.0.1) so only connections to localhost are allowed. To bind to another address first we must make sure the GatewayPorts option is enabled on the ssh server (i.e. on my machine). Simply add this line to the /etc/sshd_config file: "GatewayPorts yes" and restart the sshd server with "/etc/init.d/ssh restart". This is in Ubuntu distro... other distros must be similar. Then my sister has to excecute the command by adding a bind address like:

ssh -R my_ip_address:2222:localhost:22 my_ip_address

This way anyone can connect to my sisters machine via ssh by connecting to port 2222 on mine:

From any PC on the world execute

# ssh -l sister_user -p 2222 my_ip_address

Note the difference here is that we are not limited to connect from within my machine only. Now anyone that knows my IP address and the tunnel port can connect to my sister's PC. What is this useful for? There are two situations I can think of in which this is useful:

- First if I am not expert enough to solve my sister's problem I can ask some Ubuntu guru guy to connect to my machine on port 2222 and he will be logged in my sister's machine. This way this guru guy can help me fix the problem even if he is also far away from me and my sis.

- Second situation would be that both, my sis and me, are behind a NAT. In this case nor can I access my sister's machine nor she can access mine via ssh. The solution is ask a third person with a ssh server accessible in the Internet to allow my sister to create a tunnel to his ssh server. Then I can connect to that server to access my sisters's. Something like the next image:



The obvious problem is that we need a willing person to allow us to use his/her ssh server as relay. Maybe using some hosting services with ssh login access can be used.

What about GUI applications??

With this solution I can diagnose and fix most of the problems via shell command but sometimes there are some configurations that are easier to fix by opening the graphical application per se. For example configuring Firefox or Konqueror to use proxy or configuring accounts and filters on Kmail/Evolution whatever... I know all these task can also be done in the shell command by editing the configuration files but the developers of all these applications took the time to create nice configuration GUI's and I want to honor them by using the configuration GUI's; apart that I really don't know all the configuration files as I use Kubuntu (KDE) while my sis installed Ubuntu (Gnome).

Fortunatelly SSH can help me here too!! by allowing X11forwarding if I connect to my sister's machine via ssh and excecute a GUI program (i.e. kate, kmail, konqueror) that application display will appear in my own PC!!, that is if I have X running and then I can play with those applications as if they were running on mine.

To do this we must ensure that the sshd server has X11Forward enabled. In Ubuntu/Kubuntu it is enabled by default so no modification is necessary. In case it is not enabled it is easy to enable it by adding this line "X11Forwarding yes
" to the /etc/sshd_config file and restarting the ssh server with "/etc/init.d/ssh restart". In my example this has to be done on my sister's machine.

After the server is enabled with X11Forwarding we must connect using the -X swicth like:

# ssh -X -l sister_user -p 2222 my_ip_address

And that's it!!. Now I can run any application on my sister's machine (GUI and non-GUI) and they all appear as being running on mine. This way I have full access to my sister machine and can do all kinds of diagnosis, tunning and fixing to give here the best Linux experience ever.

Setting up tcpserver for qmail

In the previous post I explained how to install a qmail SMTP server in a Red Hat machine. In that post I only got to the point where I could send messages to local users from the local machine using qmail-inject. Now we want to be able to send messages using this qmail server but from remote machines using port 25 on this server. We also want to configure tcpserver to manage start/stop all qmail services at boot time.

Installing tcpserver

As mentioned in the previous post there are several ways to manage tcp connections in a linux machine (i.e. inetd/xinetd, tcpserver). For scalability and security reasons I choose to use tcpserver.

The normal way to install is to download the source code, extract it and run make

# wget ftp://ftp.jp.qmail.org/qmail/ucspi-tcp-0.88.tar.gz
# tar xvfz ../ucspi-tcp-0.88.tar.gz
# cd ucspi-tcp-0.88/
# make setup check

This software also has a compilation problem due to the errno.h header file as with qmail. Before we run the make command we must edit the error.h file and replace the line "extern int errno;" whith the line "#include <errno.h>" as we did with qmail.

After this we must create the access control list that determines who can relay emails with our server. Without this access control our SMTP server could be used by anyone to relay any email to any domain (think SPAM).

To create the access control list we create a text file "tcp.smtp" that will contain our rules and then generate a cdb file from it using the tcprules utility.

Let's create the text file with our rules:

(as root)
# vi /etc/tcp.smtp

192.168.0.:allow,RELAYCLIENT=""
127.:allow,RELAYCLIENT=""

Make sure to replace 192.168.0. with the subnet of the server you are installing. Then we run the tcprule command to generate the cdb file:

# /usr/local/bin/tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp

With this now we have a cdb file that contains our access control rules. In this example the rules we created allow only machines in the same subnet of the server to and the server itself to relay emails to other domains (i.e. domains that are not locally configured in the rcpthost file of qmail). To learn more about how to create other rules and the tcprules utility refer to the tcprules page and to learn about the cdb file format refer to this page.

Now we can use the tcpserver command to start the qmail service on port 25. The complete command to do this should look like this:

# tcpserver -v -u［qmaild UID] -g［nofiles GID］-x /etc/tcp.smtp.cdb 0 smtp /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger smtpd 3 &

The details of the command can be found on the tcpserver page or by looking at the man page. The only thing that requires mention is the redirection of the standard/error output to the splogger command. This only redirects the messages of the tcpserver and qmail-smtpd commands to the system logger (i.e. syslog).

To start the qmail services when the machine boots we need a startup script. I have modified the sendmail startup script so it starts qmail instead:

#!/bin/bash
#
# qmail This shell script takes care of starting and stopping
# qmail MTA.
#
# chkconfig: 2345 80 30
# description: Qmail is a Mail Transport Agent, which is the program # that moves mail from one machine to another.
# processname: qmail
# config: /var/qmail/control
# pidfile: /var/run/qmail.pid

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
[ -f /etc/sysconfig/network ] && . /etc/sysconfig/network

# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0

RETVAL=0
prog="qmail"

start() {
# Start daemons.

echo -n $"Starting $prog: "

csh -cf '/var/qmail/rc &'
/usr/local/bin/tcpserver -H -R -l test.example.jp -v -x /etc/tcp.smtp.cdb -u 92 -g 91 0 smtp /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger smtpd 3 &

## HERE GOES THE POP3 COMMAND. More on this in latter posts...

touch /var/lock/qmail

return $?
}

stop() {
# Stop daemons.
echo -n $"Shutting down $prog: "
PID=`/bin/ps -aefw | grep qmail | awk '{print $2}'`
if [ ! -z "$PID" ] ; then
/bin/kill ${PID} 1> /dev/null 2>&1
fi
rm -f /var/lock/qmail
return $?
}

# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
RETVAL=$?
;;
status)
status qmail
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|restart|status}"
exit 1
esac

RETVAL=$?
exit $RETVAL

I am far from an expert on Red Hat init scripts so don't expect this one to be perferct. It works for me but I am not responsible if it burns your machine or something.

Copy this script in a file called qmail inside the /etc/init.d directory and use this command to install the script as a startup script:

# /sbin/chkconfig --add qmail

Then we configure the newly added qmail startup script to be excecuted at boot up:

# /sbin/chkconfig --level 235 qmail on

and we are done!! now we can start/stop the qmail services using the Red Hat service command like:

# service qmail start
# service qmail stop

or

# /etc/init.d/qmail start
# /etc/init.d/qmail stop

Comming Soon!!
Next post will be about POP3 service so users in this server can download the mails they receive. During this process I will also install vpopmail to manage virtual domains using a MySQL backend.

Installing qmail 1.03 on Red Hat ES

I mentioned in my previous post that I am installing a email server on Red Hat ES version 4. In this post I plan to talk about how I managed to install a qmail SMTP server to create a reference I can easily find if I ever need to do this again.

Red Hat Preparation

Install all rpm packages needed for building source code. This rpm packages should be on the installation CD's:

rpm -ivh gcc-3.4.3-9.EL4.i386.rpm
rpm -ivh glibc-devel-2.3.4-2.i386.rpm
rpm -ivh glibc-headers-2.3.4-2.i386.rpm
rpm -ivh glibc-kernheaders-2.4-9.1.87.i386.rpm
rpm -ivh autoconf-2.59-5.noarch.rpm
rpm -ivh automake-1.9.2-3.noarch.rpm

Installing qmail
First download the qmail-1.03 source code and the qmail-date-localtime.patch.This patch causes the various qmail programs to generate date stamps in the local timezone.
ftp://ftp.jp.qmail.org/qmail/qmail-1.03.tar.gz
ftp://ftp.nlc.net.au/pub/unix/mail/qmail/qmail-date-localtime.patch

Next uncompress the qmail-1.03 source code and apply the patch

# tar xvfz qmail-1.03.tar.gz
# cd qmail-1.03/
# patch -p1 < ../qmail-date-localtime.patch

Before we compile we must do some manual work here. This version of qmail is not the most recent version of qmail and it has some incompatibility problems with glibc 2.3.1 and later. The problem is with the errno.h header file that qmail does not have included. To fix this we must modify the source code of qmail (don't be afraid, is not big deal) and change all the lines that say extern int errno; with the line #include <errno.h>. There are only three files that have this line:

cdb_seek.c
dns.c
error.h

Simply open this files and edit the problematic line. And then we make the famous configure/make/make install dance:

# make setup
# make check
# ./config

Please note that the command ./config will complain if the machine does not have a FQDN (Fully Qualified Domain Name) properly configured in the DNS server. If you are installing a test server that will not have a FQDN or that will have one later we should change the command to ./config-fast host.example.jp where host.example.jp is the test domain name.

At this point qmail must be installed in the /var/qmail directory and most cofiguration files must be already setup. We can check that everything went ok by looking at this files inside tha /var/qmail/control directory:

・defaultdomain
example.jp

・locals
localhost
localhost.example.jp
host.example.jp
example.jp

・rcpthosts
localhost
example.jp
.example.jp

・me
host.example.jp

・plusdomain
example.jp

In all this files example.jp must be the domain name of your server. You can edit the files in case they are missing any parts. Actually you may trust better the qmail ./config script and leave everything just like that.

We shall not forget the important accounts for root, postmaster and mailer-daemon.

# cd /var/qmail/alias
# touch .qmail-postmaster .qmail-mailer-daemon .qmail-root
# chmod 644 .qmail*

If sendmail is installed on the server we must disable it to avoid conflicts with qmail. Unfortunatelly uninstalling sendmail in Red Hat ES 4 starts complaining about missing dependencies and stuff. The solution for this is cheat the server by removing the sendmail binary and replacing it with that of qmail:

First stop the service

service sendmail stop

Next remove it from the statup scripts so sendmail is not restarted when the server is rebooted:

/sbin/chkconfig --level 12345 sendmail off

Finally replace the sendmail binary with a link to the qmail binary:

# cd /usr/sbin
# mv sendmail sendmail.dist
# chmod 0 sendmail.dist
# ln -s /var/qmail/bin/sendmail /usr/sbin/sendmail

With sendmail out of way we can now start the qmail services just for testing:

# cp /var/qmail/boot/home /var/qmail/rc
# /var/qmail/rc &

Testing qmail

To check if the qmail services are running we can use the ps command:

# ps -aef | grep qmail
qmails 14540 13180 0 00:54 pts/4 00:00:00 qmail-send
qmaill 14541 14540 0 00:54 pts/4 00:00:00 splogger qmail
root 14542 14540 0 00:54 pts/4 00:00:00 qmail-lspawn ./Mailbox
qmailr 14543 14540 0 00:54 pts/4 00:00:00 qmail-rspawn
qmailq 14544 14540 0 00:54 pts/4 00:00:00 qmail-clean

All the services must be running under the respective users. If the output is not like the shown here then something went wrong. Try to go back and check all steps again.

Now lets try to send a single mail locally to see if it arrives. To send and empty email to the root user we run the next command:

# echo to: root@host.example.jp | /var/qmail/bin/qmail-inject

To check if the mail was delivered or not we look at the root user Mailbox:

# more /var/qmail/alias/Mailbox

There should be an empty email in there directed to root@example.jp.

But I want Maidir instead of Mailbox!!
Ok we all now that Maildir have advantages over mailbox and that in these days Maildir is the default. To change qmail to use Maildir instead of Mailbox we must edit the start script /var/qmail/rc file.

We must change the line:

qmail-start ./Mailbox splogger qmail

with the next line

qmail-start ./Maildir/ splogger qmail

Note the trailing slach after Maildir... it is important.

Now each user that is to use Maildir must have the Maildir directory created in it's home directory. For this we use the maildirmake utility that comes with qmail. We don't want to be creating Maildirs everytime a new user is added so we create a Maildir in the /etc/skel direcotry. This way the Maildir will be created automatically when a new user is added to the system.

# /var/qmail/bin/maildirmake /etc/skel/Maildir

Finally we kill all the qmail process using the kill command and the pid of the processes we see with the ps command and restart the qmail again but with Maildir support.

# /var/qmail/rc &

Starting qmail at boot

There exists several ways to start qmail at server boot. The old way was using inetd or the newer xinetd. Unfortunatelly it is known that inetd/xinetd have scalability limitations as the number of requests increase. The new solution is to use tcpserver that not only is more efficient and scalable but also has security features.

Since this post is already very long I will blog about setting qmail with tcpserver in a new post. Subsequent posts will contain information about POP3 access and Vpopmail/MySQL virtual domain support.

Hitting the limits (inode-max)

Recently one of my qmail servers (at work) stopped to deliver mails. A quick look at the log files revealed that qmail was unable to create the mail files in the Maildir directories:

mail qmail: 1158303878.354871 delivery 5050: failure:
can_not_open_new_email_file_errno=28_file=/dir/to/maildir/folder

Trying to manually create a file in the folder gave me a no free space on device error but df command said the hard disk had plenty of free space. After some more research the problem resulted to be that the partition had hit the inode-max limit. This value is the max number of files that can be stored in the file system.

To check out this you can use the command "df -i" or look in /proc/sys/fs/inode-nr.

This seems to be a common problem in mail servers that create a lot of small files (mail messages and mail folders) since each file uses a inode.

In linux kernels previous to the 2.4 series there was a inode-max parameter that could be changed (i.e. increased) to fix this problem. From kernels 2.4 onwards this parameter is managed automatically by the kernel but it still has a max value and to increase that maximum we must increase the file-max parameter of the kernel.

After some more research I understand that inode-max is usually three time the file-max value and that file-max value must be set according to the available ram in the server.

In this online book they recomend setting 256 files per 4MB of ram to file-max, for example if we have 128MB ram then

256 * (128 / 4) = 8192

and to set it in a running server simple run

# echo "8192" >/proc/sys/fs/file-max

where value is the number obtained from the calculation. Then inode-max will be roughly 3*value that is, three times file-max.

In RedHat we can set this parameter on the /etc/sysctl.conf file so this setting is kept at every reboot of the machine.

# Improve the number of open files
fs.file-max = 8192

So if you are setting up a server that requires to store lots of files (mail, http) then you may need to increase the value of file-max.

There are questions I still cannot answer myself:

- What happens if we keep increasing file-max ignoring the RAM amount??
- Maybe using a File System that handles small files more efficiently we can
reduce the number of inodes required?? for example ReiserFS instead of Ext3?

Any tips on these is appreciated....


Here are some links with more information about server tunning and file-max explanation:

System Tunning
file-max discussion at kernel level

RedHat Enterprise 4 Administration

The first linux distribution I used was RedHat and after that I have used several distribution up to Kubuntu now. It's been more than 5 years since I left RedHat and today I got the task to install RedHat Enterprise V4 for a mail server. Now more than ever I realise the importance of recording all I have learned... I cannot even setup the default gateway on this RedHat thing. Of course I now the route command to set it up, but this need to be set when the server is rebooted etc. and to keep things clean I want to configure it as RedHat does it.

After several minutes googling I got all the required information for setting up the network and for managing packages in redhat.

Network Setup

All network configuration files are in the directory /etc/sysconfig/network-scripts/. In this directory exist a file for each physical interface in the computer with names like ifcfg-ehX.

The file format for the first Ethernet device is:

    /etc/sysconfig/network-scripts/ifcfg-eth0

        DEVICE=eth0
        IPADDR=192.168.1.42
        NETMASK=255.255.255.0
        ONBOOT=yes
        GATEWAY=192.168.1.1

We can have multiple configurations in the same device (i.e. Multiple IPs) by means of aliases

    /etc/sysconfig/network-scripts/ifcfg-eth0:0

        DEVICE=eth0:0
        IPADDR=192.168.1.41
        NETMASK=255.255.255.0
        ONBOOT=yes

    /etc/sysconfig/network-scripts/ifcfg-eth0:1

        DEVICE=eth0:1
        IPADDR=192.168.1.44
        NETMASK=255.255.255.0
        ONBOOT=yes

    /etc/sysconfig/network-scripts/ifcfg-eth0:2

        DEVICE=eth0:2
        IPADDR=192.168.1.45
        NETMASK=255.255.255.0
        ONBOOT=yes

For the new configuration to take effect we can reload the network service.

    # service network restart

The Gateway setup as I can see can be configured in the network card like in the previous example or in the /etc/sysconfig/network file. The only difference (maybe) is that this gateway is for the whole system (i.e. all cards) while the gateway configured in the ifcfg-etX files are device specific. Again this is a guess as I have not tried this yet.

DNS Setup

This configuration is stored in the file /etc/resolv.conf. This example resolv.conf is using 1.2.3.4 and 5.6.7.8 as the nameservers.

    /etc/resolv.conf:

        nameserver 1.2.3.4
        nameserver 5.6.7.8

Managing Services

When managing a server it is necessary to know how to configure services to start/stop and boot time and how to add/delete new services. We can do this manually by creating scripts in the corresponding rc.d directories but RedHat provides an utility for this: "chkconfig".

We must remember that linux has several run-levels and that each one can start different services. RedHats default run level is 3 so lets see what services get started when the machine is turned on.

    # chkconfig –list |grep “3:on”

To see the list of all service simply delete the grep part like :

    # chkconfig –list 

To add a new service to the startup list, execute:

    # chkconfig –add service_name

To stop a service from starting on bootup, execute:

    # chkconfig –del service_name

To manually start or stop a service, execute the following as root:

    # service service_name [start|stop]

For more info use the man page:

    # man chkconfig

Managing Packages (RPM)

The RPM command is used in RedHat to install/query/remove software packages. Unfortunatelly this utility does not provide automatic updates or package search capabilities like the debian apt-get or aptitude utilities. Anyway if you want to keep your system as clean as possible we should consider using rpm istead of istalling stuff from source code. The most important operations are:

To get all of the information for a package (newpackage-1-50.rpm), execute

    # rpm -qip newpackage-1-50.rpm

To check to see if a package is installed,

    # rpm -q newpackage
    Package newpackage is not installed.

To find out what files this package will install

    # rpm -qpl newpackage-1-50.rpm

To install,

    # rpm -ihv newpackage-1-50.rpm

To remove a package,

    # rpm -e newpackage

To upgrade,

    # rpm -Uvh newpackage-1-51.rpm

To rebuild the RPM database,

    # rpm -rebuild

To return a list of all installed packages in the RPM database,

    # rpm -qa

Spidering With Ruby

If your boss suddenly asks you to convert an HTML only web page with real state listings with more than 10 years of accumulated information into a database powered web page what you do?? This means that more than 150,000 HTML pages, each with house/apartment and their sell/rent/buy information must be inserted in a database.

What I did?? I wrote a ruby spider to scrap the pages and insert them to a database. What I learned?? Ruby is an amazing language with the flexibility to handle the bad HTML pages (i.e. edited by hand and lots of missing ending tags, etc.) and the power to handle more than 150,000 of them in a reasonable span of time.

The first page to see how this is done is the Cafe-Fetcher
http://neurogami.com/cafe-fetcher/

Next some advance explanation of the different approaches we can take with Ruby:
http://www.rubyrailways.com/data-extraction-for-web-20-screen-scraping-in-rubyrails

After reading this two excellent examples I chose to use WWW::Mechanize because it has all that is needed. It used REXML for HTML parsing and XPath manipulation but as version 6.0 it uses the faster Hpricot library.

So how this works in short? Mechanize is like a browser that can be programmed, with support to cookies, history, etc. You use Mechanize to navigate the web page until you get the particular page you are interested in (i.e. submit forms, click links, etc.). The page that Mechanize returns in fact is a hpricot object that has the HTML parsed and allows you to access all the DOM structure. You can also use XPath to access specific parts of the HTML like get the first column of the third line in the second table of the HTML page.

With this tools, some patience and a time you should be able to automatically scrap all information from any page in the Internet. Make sure you learn XPath as it is a powerful XML query language to access the data in HTML pages. It is something like an SQL for XML.


Warning: There is a little problem with the Mechanize history feature. If you use a single Mechanize object to scrap lots of pages you will observe that your PC ram is going to start growing non-stop. This is because Mechanize stores each page it downloads in a history array and this pages never get GarbageCollected. To avoid this simply setup the history_max value of the Mechanize like this:

agent = WWW::Mechanize.new
agent.history_max = 10

Never put it to zero as it will cause some problems when submitting forms.

Links:

hpricot
WWW::Mechanize
XPath Tutorial

A problem of bad memory

This blog is to be my technical memory... How many times I have learned something usefull only to forget it a few months o even years latter when I really need it. As one of my professors once told me: Knowledge does not mean knowing everything, but to know where to find it.

So this blog is to record everything interesting I learn in hopes that I can find it in latter years when I need it.