I'm going to update this article SOON! I'll shortly be talking about my communications with, and trials using the documentation available at http://www.goodcleanemail.com and http://www.freebsdrocks.net

Before I begin, let me tip my hat to the work done at qmailrocks.org. They have tried their best to make qmail work for a varied multi-domain email systems. That said, there's still a funkiness about that's all too common to "Open Source" products.

It's a virtual guarantee with open source or commercial software that nothing ever works as advertised. Part of this is just the confines of reality: hardware is different, operating systems are constantly changing, and environments and demands are always different in ever situation. So, it's all incredibly difficult for people to track. Programmers, believe it or not, like all people have their limitations. The vast complexity that is the personal computer, even the dedicated web server, make up a major challenge that multi-billion dollar capitalists and open source hackers alike find daunting.

When contracted to build a web-based solution to email management and email services, I decided to revisit qmail based solutions. It's a nightmare, I promise you. But, it's the only generic open standard out there. Everything else is proprietary or even worse documented than are the qmail solutions. I came across qmailrocks, while searching for tutorials. I joined the mailing list and decided to try it out.

The goal is part of what makes their solution look most attractive:

• Open Source and Configurable Email System
• Open Standard Communications Systems that Work with Varied Email Clients
• Web Access for Email Accounts (Web-based Email)
• Web Management Tool to Manage Email Services

qmail, UCSPI-TCP, qmail's Daemontools, qmail's POP3d, qmail SMTP Authentication, qmail SMTP TLS Patch, VPopMail, VQAdmin, Maildrop, qmailAdmin, IMAP & IMAP SSL, Squirrelmail all tied together do the job rather nicely.

One member of the QMR email list has posted his own version of the QMR process (script) for OpenBSD. In typical fasion, he's complained about how the QMR administration doesn't respond to his email. I'm not sure why people participate in the open source community and then ignore people trying to help. Chastisement!

I had several problems during the installation process (well, more the testing to see if it actually worked process) and nothing annoys me more than reading through thousands of questions with no authoritative answers. Much of the forums were riddled with guesses and bad suggestions. I haven't read so much bad advice since the last time I tried to install qmail solutions on one of my servers. I've tried my best, whether I decide to stick with this as a solution or not, to document real problems and
their authoritative answers. No guesswork is allowed here!

QMailRocks Install and Performance Failures

Background: After installing VPopMail, I decided to try out the command line controls to see if it was interfacing with the SQL dB (MySQL) correctly. So, I added one domain, which accepted email for one email account, fine. The second domain I added couldn't receive email and then authentication problems crept in regardinging the original working domain.

I encountered a problem when deleting a domain I'd created from the system to do a fresh install of some tools and create the domains (email) from scratch. The error I received was:

root% ./vdeldomain mydomain.com
Warning: Failed to delete dir_control for mydomain.com

This was command line error (obviously), but most of the alleged answers I found in the forums were related to the web-based tools and error logs. I found suggestions like "check the permissions" and so forth. None of these were useful. I was root, and I was running the command, so it wasn't a permissions issue. You can't get much more 'permissioned' than root.

It turned out it was a VPopMail/MySQL thing:

root% find / | grep -w dir_control
/var/db/mysql/vpopmail/dir_control.frm
/var/db/mysql/vpopmail/dir_control.MYI
/var/db/mysql/vpopmail/dir_control.MYD

So, I dug through the MySQL related stuff to root out the issue.

And, it looked like this was a bunch of binary files... I guessed created by VPopMail. So, why couldn't the script (ran as root) delete them?

Here's what was in that folder:

--

So, it seems that the system creates binary management files to interract with MySQL (or your SQL of choice). As I look at the primary control files like "dir_control.MYD", I found that the domain names were inside these binary files. Obviously, there was either a rule against removing a domain or something was screwy with my VPopMail installation. But, still no docs were found to support any theory I'd developed and nobody offered any solution on any list I tried. Google and mailing lists were failing me.

This is the challenge with installing and supporting open source software. There's lots of people throwing around free advice, directions, software, etc. But, there are few people with authoritative answers when something doesn't work just as planned. It's not different in the world of commercial software. Only, in commercial software the solution of uninstalling and reinstalling is always first one to the table when the query gets beyond "how do I turn it on"?

Finding Real Answers

I had a friendly conversation with a nice lady in an IRC channel for linux users. She talked about running qmail and the quirks and her positive performance experience. She expressed that she'd never used VPopMail before, but said she'd try to be helpful about qmail itself, and she was. She immediately pointed out to me that I couldn't expect VPopMail to edit the RCP lists in qmail. That's the job of the web tool they include: VQAdmin. This lightened my load, and made me realize why it was that the first domain (the mail server's domain) worked but the next didn't. I hadn't edited the recipient allowed list for qmail. And, through her contribution I learned that I either had to add them via the web tool or edit that file manually.

Though our conversation made me feel better, much like many things it's not true just because it makes you comforted. As I later found, this was not really the case at all. VPopMail was simply screwed up.

By digging in the docs included with the source files, I found more important details. At the bottom of this article, the txt file I began working from is attached. Don't for a second assume that there weren't inconsistencies. Specifically, the source location is pointed in the INSTALL_freebsd.txt file as /usr/src/qmail/qmail-1.03, which may well contain source if you have been keeping up with the source tree. But, that's not going to be the source QMR supplied in the download. You'd have to go to /download/ (if you followed the directions explicitly). When you untar each package, you'll get the source in that directory somewhere.

As it turns out, inside the tarball I downloaded from qmailrocks.org, contained an install file. And, it's what I'd hoped: more detailed. As a matter of fact, the web version of the directions don't deal with something I think is vitally important: dependencies. It gives you a list of pre-build dependencies. Wouldn't that have been useful! There was a checklist.php file on their website, but it wasn't linked to from the instructions I used.

Here's the vital list I missed in the first round:

1. The Apache Web Server - You can use either version 1.3.x or version 2.x. It shouldn't make that big of a difference.
2. PHP - Version 4.0.6 or higher. You will probably want to make sure that it's either compiled with imap and mysql support, or if you are installing from RPMs, install the php-imap and php-mysql packages alongside the php package.
3. Perl - I use version 5.8.0, but any version of 5 should work.
4. GCC - The gcc compiler. You should already have it installed, but if you don't you'd better.
5. MySQL - MySQL is only REALLY needed if you intend to use it with vpopmail. Also, you may run into trouble installing some packages if you don't have it installed. All in all, it's a good idea to have mysql server installed. Version 4.x works just fine, but 3.x will work too.
6. OpenSSL- Version 0.9.5a or higher.
7. OpenSSL-devel - For Redhat products and Fedora users.
8. libssl-dev, for Debian users.
9. wget - Downloading packages and software is alot easier with wget.
10. patch & patchutils - Available via RPM for Redhat, the ports collection for FreeBSD or apt-get for Debian. You'll need these packages to apply the needed patches along the way during the install.

% cd /usr/ports/mail/php5-imap
% make all install clean

Php needs to be installed, Perl (default in 4.X, separate install in 5.X), GCC (which I already had), MySQL (installed), OpenSSL (installed), wget (installed), and patch & patchutils.

Patch is in the ports along with patchutils:

% cd /usr/ports/devel/patch
% make all install clean

% cd /usr/ports/misc/patchutils
% make all install clean

These directions also include a directive to kill off any pop daemons. Perhaps this is common sense, but it's nice to see they considered the potential oversight.

One more remedial step was to open the ports needed for the firewall to allow connections to the daemons we were going to install.

Outbound Ports (tcp)25 - SMTP
110 - POP services
143 - IMAP
783 - Spamassassin
993 - IMAPS

Inbound Ports (tcp)25 - SMTP
80 - HTTP
110 - POP services
143 - IMAP
443 - HTTPS
783 - Spamassassin
993 - IMAPS

Next I looked at dependencies required:

1. Time-HiRes
2. Net-DNS
3. Digest-SHA1
4. Digest-HMAC
5. HTML-Tagset
6. HTML-Parser
7. Parse-Syslog
8. PodParser
9. Statistics-Distributions

The author admited this is likely not a complete list. However, without more information, I had to act on good faith.

They also provide a script to check for dependencies. That script is in the tarball.

/downloads/qmailrocks/scripts/util/check_perlmods.script

If you want to add them from the ports collection, they are located:

• Time::HiRes- /usr/ports/devel/p5-Time-HiRes
• Net::DNS - /usr/ports/dns/p5-Net-DNS
• Digest::SHA1 - /usr/ports/security/p5-Digest-SHA1
• Digest::HMAC - /usr/ports/security/p5-Digest-HMAC
• HTML-Tagset - /usr/ports/www/p5-HTML-Tagset
• HTML::Parser - /usr/ports/www/p5-HTML-Parser
• Pod::Usage- /usr/ports/textproc/p5-PodParser
• Parse::Syslog - /usr/ports/textproc/p5-Parse-Syslog
• Statistics::Distributions- /usr/ports/math/p5-Statistics-Distributions

As I followed the instructions, everything pretty much went as planned, until I arrived at VPopMail installation. Here, I needed to read the INSTALL file to find the readme file that contained the switched I would use to configure it correctly. The default installation directions don't tell you to enable MySQL, which gave me reason to wonder if they had missed other necessary arguments.

As the readme said, there were better guides to installing VPopMail with MySQL. However, I found a good website to use for general MySQL, Apache MySQL and php. In order to perform my goals, I ran configure with the options below:

./configure --disable-roaming-users --enable-logging=p --disable-passwd --enable-clear-passwd --disable-domain-quotas --enable-auth-module=mysql --disable-many-domains --enable-auth-logging --enable-mysql-logging --enable-valias --disable-mysql-limits

So, in the order above, here's what those settings mean:

I'm not building roaming user support, log POP3 auth errors to syslog (/var/log/maillog), don't include /etc/passwd support (no system users), store passwords in cleartext in the dB, domain quotas allow you to limit a domain's storage capacity,store all the user and domain info in MySQL, tell VPopMail to create one MySQL table per domain,maintain a lastauth table in MySQL (shows when / how a user last accessed their email), maintain the vlog table in MySQL (shows failed authentication requests),enable MySQL valias processing,use disk-based ".qmailadmin-limits" files rather than storing this data in MySQL

Since I shamelessly stole this information from the reference page mentioned above, it may behoove an admin to read that page carefully before configuring and compiling VPopMail.

So, I configured the VPopMail installation with the exact command above, and the results were:

vpopmail directory = /home/vpopmail
uid = 1016
gid = 89
roaming users = OFF --disable-roaming-users (default)
password learning = OFF --disable-learn-passwords (default)
md5 passwords = ON --enable-md5-passwords (default)
file locking = ON --enable-file-locking (default)
vdelivermail fsync = OFF --disable-file-sync (default)
make seekable = ON --enable-make-seekable (default)
clear passwd = ON --enable-clear-passwd (default)
user dir hashing = ON --enable-users-big-dir (default)
address extensions = OFF --disable-qmail-ext (default)
ip alias = OFF --disable-ip-alias-domains (default)
domain quotas = OFF --disable-domainquotas (default)
auth module = mysql --enable-auth-module=mysql
mysql replication = OFF --disable-mysql-replication (default)
sql logging = ON --enable-sql-logging
mysql limits = OFF --disable-mysql-limits (default)
MySQL valias = ON --enable-valias
auth inc = -I/usr/local/include/mysql
auth lib = -L/usr/local/lib/mysql -lmysqlclient -lz -lm
system passwords = OFF --disable-passwd (default)
pop syslog = show failed attempts with clear text password
--enable-logging=p
auth logging = ON --enable-auth-logging (default)
one domain per SQL table = --disable-many-domains

Since it looked like it may do the trick, I ran a make && make install-strip.

It seemed to work just fine, so I moved to the VQAdmin installation. As before, I read the VQAdmin directory's readme for SQL. Here's where I went wrong the first time. I didn't read the readme file the first time and just followed the install directions. Reading the readme file I find:

0. Create the following table in a database in your MySQL:

CREATE TABLE domain_owner (
domain CHAR(64) NOT NULL,
owner CHAR(32) NOT NULL,
PRIMARY KEY(domain, owner)
);


1. Edit the file db_owner.c

Fill in the approriate names for host, username, password, and database.
The variables to set are clearly marked at the top of the file, and their
values should be set according to what you want vqadmin to connect to your MySQL database as. The value for variable db should be set to the name of the database that you created the table domain_owner in.

Hey, I missed that one the first time around. Logically, we need to set up VQAdmin to do it's thing through the SQL dB. The part you need to edit is:

static const char *host="localhost"; /* your auth machine, localhost typically */
static const char *user="db_user"; /* the dB user you created for VPopMail */
static const char *passwd="user_pass"; /* password */
static const char *db="db_name"; /* the name of the dB you created for VPopMail

Alert: if you don't do this, your VQAdmin installation will not work! The directions on the site were missing necessary information for an SQL supported installation.

So, I set my parameters in the file, saved it and then ran the command to configure for compiling.

% ./configure --enable-mysql=y --enable-cgibindir=/home/vpopmail/cgi-bin --enable-htmldir=/home/vpopmail/public_html

The arguments I passed with this configuration told it that I was using MySQL and that should have made it look to that file we edited to pass it the values for the user, password, and database to setup new domains. I chose to skip the standard placement for VQAdmin, and placed it specifically into a directory I'd set aside for the web-based tools. By default it assumes where your cgi-bin should be and throws it there. So, the options I used above will likely be prefered.

I suddenly started feeling much more positive about the whole process. It started making a lot more sense.

As soon as the configure completed, it told me immediately that I had set the configuration correctly:

vpopmail directory = /home/vpopmail
uid = 1016
gid = 89
cgi-bin dir = /home/vpopmail/cgi-bin
vqadmin dir = /home/vpopmail/cgi-bin/vqadmin
mysql features = enabled

Awsome! Now, I had solved three problems that were apparent in my first failed installation. These changes correctly configured VPopMail and VQAdmin, as well as setup the correct dependencies that were not present before.

Of course, I got an error when it comes time to compile :( Shock!

The details were:

In file included from domain.c:32:
/home/vpopmail/include/vpopmail_config.h:218:1: warning:

"PACKAGE_NAME" redefined
In file included from global.h:26,
from domain.c:30:
config.h:66:1: warning: this is the location of the previous definition
In file included from domain.c:32:
/home/vpopmail/include/vpopmail_config.h:221:1: warning:
"PACKAGE_STRING" redefined
In file included from global.h:26,
from domain.c:30:

This error isn't your script kiddy problem solver. It was going to take some more digging to figure this one out. So, it was back to errors, logs, and instructions. Surely there was an argument I could pass to the configure that could make this problem go away.

Frustrated, and very tired of the whole QMR experience by this point, I started playing around with code that was tempted an rm -r. I added a domain via vadddomain in vpopmail, wondering if I could at least get my client to a point of manually adding domains so I could get some sleep. As I went to add it manually to /var/qmail/control/rcpthosts, I found a pleasant suprise. VPopMail added the domain to everything just the way I needed it to. The domain was already there. So, remove, add... send email, receive... and it worked! Like the godfather in Godfather III said, "Just when you think you're out, they pull you right back in!"

So, I was stuck with a VQAdmin correctly configured as far as I could tell, but wouldn't compile. And, the VPopMail command line, and web interface were working just fine. I wondered why I shouldn't just add domains comand line (easy enough to teach the admin to be) and setup IMAPd and Squirrelmail. I figured I could be in bed in less than an hour.

Of course, IMAPd didn't pass the test, and I found all kinds of issues that required that SQL details be defined in configure files before configurations and compilations were made. This is when I decided I didn't have time for this anymore and decided to delete it all and move to a Postfix solution.