HOWTO_Email_System_for_the_Home_Network

Abstract

This guide will help you create a fully functional email service within a home network. You will run servers to allow you to both send and receive email from all over the world. We will use free services to facilitate this such as DynDNS.

This guide can be used as full blown mail server provided you have an MX record pointing to your mail server. No changes are necessary.

This guide was taken, lock, stock, and barrel, from here

Introduction

There's already quite a number of email systems available on the net, on Gentoo's web site and even in this very forum. Here's yet another email setup. Created since I could never find a middle ground for setting up email, either they were too simple or too robust for my needs. Perhaps if you find yourself in this position, this guide can help.

We will create a fully functional email service within a home network. We will become our very own SMTP, POP3/IMAP provider using free services, free software and a free operating system. We use SMTP Auth through Cyrus-SASL so that we can force users to authenticate before using our service.

Sending

Email Client->Cyrus-SASL->Postfix->Internet

This setup allows you to use this SMTP server from anywhere in the world, provided your ISP does not block port 25 incoming. This may not be what you want though, so we have another method of sending, taking a few extra steps to allow you to forward (or relay) your email to your ISP's SMTP server. You would use this method if you don't want port 25 open to the Internet and you only need to access it inside your lan. It is safer since you can block all port 25 connections from the Internet using IPTables. So just to recap, the above method is used if you want to become your own full-blown SMTP server, the method just below is used for an internal SMTP server with access to your ISP's SMTP server.

Email Client->Cyrus-SASL->Postfix->Cyrus-SASL->ISP SMTP Server->Internet

Receiving

ISP IMAP/POP3 Server->Fetchmail->Procmail->Courier-IMAP->Email Client

This is a standard setup covered extensively on the forum, gentoo's site and on the Internet. This is the setup you would choose if your ISP blocks port 25 (incoming) or you need to get email from external email services.

The second method makes your email service inside your lan function just like the big boys. You'll be able to use your own domain name (or free service) and create your own cool email address. For example, beowulf AT apparition DOT ath DOT cx is my custom email address.

Internet->Postfix->Procmail->Courier-IMAP->Email Client

Software Versions Used

Please note, this guide was written using these versions of the software. If by the time you read this, a newer version of the software has been released, I encourage you to check the developer's web site and read the changelog or find changes in behaviour.

• net-mail/fetchmail-6.2.5
• mail-filter/procmail-3.22-r6
• net-mail/courier-imap-4.0.1
• mail-mta/postfix-2.1.5-r2
• dev-libs/cyrus-sasl-2.1.20
• mail-client/squirrelmail-1.4.4
• mail-filter/bogofilter-0.92.8
• mail-filter/spamassassin-3.0.2-r1
• dev-libs/openssl-0.9.7e

What This Guide Doesn't Do Well

This email system does not scale well. I can't imagine managing more than 5 accounts with the current set up as it will just become cumbersome in my opinion. When sending email using your own SMTP server, or this setup; some POP server's may regard your mail as spam and it will either be blocked entirely or filtered into a spam folder. Yahoo! is one such email service Thanks to dteisser for the info. I haven't encountered this problem yet so it may be rare. All the same, it is something you should be wary about! Compuserve blocks all mail from mailservers on DHCP addresses.

Preparation

Since we're dealing with 2 computers, we must designate one of them as the server. We'll refer to the two computers as "workstation" and "server". We will assume that you already have your hostname setup (should have been done during your install process). So all that is needed is to find the information.

Local Servers

Below is an ASCII chart of what is needed, and the possible values that could be used. Of course your network setup may differ, and I do encourage you to find out all information needed before you continue.

Chart 2.1
.--------------------------------------------,
| Needed    ||     Server    |  Workstation  | Chart 2.1 - Server Info
|===========||===============================|
| Network   ||         192.168.2.0/24        |  - Copy paste this code
|-----------||-------------------------------|    block into a text editor
| IP        ||  192.168.2.2  |  192.168.2.3  |    editor for reference
|-----------||---------------|---------------|    later on.
| Hostname  ||    Chimera    |   Illusion    |
|-----------||-------------------------------|  - Substitute the values
| Domain    ||      apparition.ath.cx        |    here with your values
|-----------||-------------------------------|
| Username  ||   21s-beo     |      N/A      |
|-----------||---------------|---------------|
| Password  ||  21s-pass123  |      N/A      |
'--------------------------------------------'

Since we're a home user, we probably don't have a dns server running with an MX record pointing to our server. Therefore to facilitate the need to access this server from other places, I suggest a free IP service. I have used "apparition.ath.cx" as the domain name provided by DynDNS, however No-IP.com is another solution. Therefore my FQDN for my server is: Chimera.apparition.ath.cx.

Need more information about DynDNS? Check out this article: Dynamic DNS

In case you have skipped it, please enter your FQDN in /etc/hosts substituting your values that you've recorded in chart 2.1 with the ones I have used.

Remote Email Services

You should obtain this information from your ISP/Email service provider. We will use 3 different examples as designated in chart 2.2 (below). Copy the chart to the same text file and label it accordingly. We'll be referring to it later in the guide.

Chart 2.2
.----------------------------------------------------------------,
| Needed    ||      SMTP     |   IMAP / SSL      | POP3 / No SSL |
|===========||===============|===================|===============|
| Server    || smtp.isp.com  | imap.fastmail.com |  pop.huah.com |
|-----------||---------------|-------------------|---------------|
| User      ||    beo739     |    beo_agate      |  beowulf_999  |
|-----------||---------------|-------------------|---------------|
| Password  ||  rsmtp-pass   |   rimap-pass      |   rpop-pass   |
'----------------------------------------------------------------'

Installing The Software

It's about time we did something. Since we use portage and benefit from the Gentoo build system, this step is easy. Don't worry about editing make.conf as we'll set the flags using profuse instead. If you are installing this system on a system without portage, you should "./configure --help" to find out the configure flags needed to match our use flags. Please SSH into your server now, or physically walk over there.

NOTE: If you already have a MTA such as Sendmail or ssmtp, you may receive a block message from portage. Simply unmerge the package before continuing. Or, you can use the mailwrapper USE flag and emerge multiple MTAs. This is not covered in this HOWTO.

NOTE: profuse may require a lot of dependencies in x11-libs/* and others on a headless server. An alternative is ufed or edit the USE variable in /etc/make.conf directly.

root@server # emerge -av profuse
root@server # profuse

Once you are in profuse, select the following USE flags: authdaemond, berkdb, fam, ipv6, nls, pam, sasl, ssl. And make sure that the mbox USE flag is not selected.

root@server # emerge -vp courier-imap cyrus-sasl fetchmail postfix procmail courier-authlib

Sending Email

Let's set up Postfix to send email out. This can be the hardest section of the guide. Let's get it out of the way.

Postfix Main Config

We'll use a base configuration before we get into any configuring. Please make sure that your file matches mine so that we can all start with the same base. If you don't see an option in the following code block, it means it should be commented. Thanks to requiem for pointing out the mailbox_command variable to me.

queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix
myorigin = $myhostname
mydestination = $myhostname, localhost.$mydomain, $mydomain
unknown_local_recipient_reject_code = 450
mynetworks_style = subnet
mynetworks = 127.0.0.0/8 192.168.2.0/24
mailbox_command = /usr/bin/procmail
debug_peer_level = 2
debugger_command =
	PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:
	xxgdb = $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail
newaliases_path = /usr/bin/newaliases
mailq_path = /usr/bin/mailq
setgid_group = postdrop
manpage_directory = /usr/share/man
readme_directory = /usr/share/doc/postfix-2.1.5-r2/readme
default_destination_concurrency_limit = 2
alias_database = hash:/etc/mail/aliases
local_destination_concurrency_limit = 2
alias_maps = hash:/etc/mail/aliases
home_mailbox = .maildir/

The mynetworks variable needs to be changed to match your internal network. If you experience hostname problems, please fill out the variables myhostname and mydomain with the appropriate information. This however should not be needed since postfix tries to get the information automatically making any declaration on our part a bit redundant.

Adding SMTP-AUTH To Postfix

We'll add SMTP-AUTH to postfix by way of Cyrus-SASL. Since everything is already emerged, let's simply configure it. New in v2.0 of this guide, we use saslauthd and auth against shadow. This should cut down on some confusion generated from earlier versions of this guide.

The first thing to do is edit /etc/sasl2/smtpd.conf and tell SASL the method and mechanisms we intend to use for auth. Make sure your file matches this one exactly.

pwcheck_method:saslauthd
mech_list: plain login

NOTE: You may also be required to edit this file with the same information, I strongly urge you to do this. Thanks to Woolong for pointing this out.

pwcheck_method:saslauthd
mech_list: plain login

What we have specified is that we will use saslauthd (daemon) for authentication, thus no longer relying on sasldb and it's quirky ways. Next up, we will have to edit the conf file for the daemon start up. Let's go and do that now.

SASLAUTH_MECH=shadow

SASL_RIMAP_HOSTNAME=""
SASL_TIME_OF_DAY_LOGIN_RESTRICTIONS=yes

SASLAUTHD_OPTS="-a ${SASLAUTH_MECH}"

You'll notice I have removed the gentoo generated line. The reason for this is because the current version of SASL that I have would not start with the default line. I have commented out everything and suggest you do the same so that your file matches mine exactly. We've stated in this file that saslauthd should use shadow as the auth mechanism.

All we have to do now is to tell postfix that you want to use sasl. Let's do that now:

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes
smtpd_client_restrictions = permit_sasl_authenticated, reject_unauth_destination

This tells Postfix that we want to use SASL to provide SMTP-AUTH and that any user who can't authenticate against SASL should be rejected.

Postfix TLS Support

A section that has caused more than its fair share of trouble, this has now become fairly easy since postfix now provides some default keys for us. No more editing the CA.pl file, no more -nodes.

Simply copy this code block exactly down in your /etc/postfix/main.cf file.

smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/ssl/postfix/server.key
smtpd_tls_cert_file = /etc/ssl/postfix/server.crt
smtpd_tls_CAfile = /etc/ssl/postfix/server.pem
smtpd_tls_loglevel = 3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

We have told postfix to always use TLS for authentication or reject the mail. Since we've decided to use plain text to send our passwords, we must use encryption to ensure any network sniffer doesn't get our password.

Making Postfix Relay To Our ISP

As was mentioned in section 1.1, we can use Postfix to be an email relay and send any mail to our ISP's SMTP server before it hits the Internet. Please keep in mind, that this step is NOT needed if you intend to use Postfix as a full-blown MTA. This step is optional and should NOT be used if you have an MX record.

Note: It is common practise for big email providers to filter or reject incoming email messages originating from dynamic IP addresses. If you do not own a static IP, then it is a very good idea to relay all your mail through a mail server with a static IP, e.g. your provider's mail server. In fact, it is also a bad idea to run a receiving mailserver on a dynamic IP with DynDNS connection if you fear losing mails or have them ending up in someone elses mail server, which by chance just got your old IP and the sender used old, but still valid DynDNS data from its DNS cache.

Email Client->Cyrus-SASL->Postfix->Cyrus-SASL->ISP SMTP Server->Internet

This is easily accomplished as well. Although it's not really needed, it may be needed with some ISP's or some network setups so I've included for anyone who may need it.

First you must create a file that holds our ISP's SMTP Server authentication information. The format is rather simple "[server] [user]:[pass]". Here's what one could look like if we used the information in chart 2.2 under the SMTP heading:

smtp.isp.com		beo739:rsmtp-pass

After you've completed that, let's protect the file and hash it so postfix can work with it. We do this with the following commands:

root@server # /bin/chown root:root /etc/postfix/saslpass
root@server # /bin/chmod 600 /etc/postfix/saslpass
root@server # /usr/sbin/postmap hash:/etc/postfix/saslpass

Next, all we must do is tell Postfix that we want it to relay the email using SASL to our ISP's SMTP server. Let's do that now.

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpass
smtp_sasl_security_options = noanonymous
relayhost = smtp.isp.com

If your provider's SMTP-server doesn't accept your mails ("status=bounce" in /var/log/mail.log), you should create the file /etc/postfix/generic and enter the mapping of your local mail account to your ISP-account. This feature is available since postfix-2.2: http://www.postfix.org/ADDRESS_REWRITING_README.html#generic

21s-beo@local.domain   beo739@smtp.isp.com

After you've completed that, enter the following commands:

chown root:root /etc/postfix/generic
chmod 600 /etc/postfix/generic
/usr/sbin/postmap hash:/etc/postfix/generic

Add the following line to your /etc/postfix/main.cf and restart postfix afterwards.

smtp_generic_maps = hash:/etc/postfix/generic

Adding Anti-Virus Protection

I have not done this myself (yet) but there is a detailed guide by john5211 on how he got clam-av working on his server. If you are interested, I would recommend clicking here for more information. axxackall also adds a few more comments in his post which is on the same page.

Automating and Finalizing

The sending section is just about done. We should add/edit our aliases that we want to use on our system. Let's do that now:

# Well-known aliases -- these should be filled in
root:          2ls-beo

You'll notice I aliased root to my username on my server (information found in chart 2.1 under the "Server" column. Let's create the alias db and check our postfix configuration:

root@server # /usr/bin/newaliases
root@server # /usr/sbin/postfix check

If all went okay, and no errors arose we can continue. If an error occured during the "postfix check" procedure, double check your main.cf file for spelling errors and syntax errors. One note to keep in mind: any option that is prefixed with a blank space will produce an error. Thanks to paulfl for pointing this out.

Now let's just automate the server's startup and start the actual server. Your ouput should match mine exactly:

root@server # /etc/init.d/saslauthd start
 * Starting saslauthd...                                                  [ ok ]
root@server # /etc/init.d/postfix start
 * Starting postfix...                                                    [ ok ]
root@server # rc-update add saslauthd default
 * saslauthd added to runlevel default
 * Caching service dependencies...                                        [ ok ]
 * rc-update complete.
root@server # rc-update add postfix default
 * postfix added to runlevel default
 * Caching service dependencies...                                        [ ok ]
 * rc-update complete.

Remember, the username and password you use to authenticate to your SMTP server is the same pair that we listed in chart 2.1. It is the same information that is found in /etc/passwd.

Filtering Email

We made mention to Procmail in the Sending section so it's only fitting that we set that up next. Procmail is a powerful piece of software that is very stable. Procmail uses rules (or recipes) similar in idea to the rules used in email clients, the difference however is we sort everything on the server side and deliver the email to various mail directories. Let's create our procmail file now.

First thing to do is drop out of root and go to our regular user.

MAILDIR=$HOME/.maildir/
DEFAULT=$MAILDIR

#
## Begin recipes
#

# put cron job emails in my aptly named cron-jobs maildir
:0
* ^Subject:.Cron*
.cron-jobs/

# Deliver Gentoo Specific email to our special maildir's
:0
* ^List-Id:.*gentoo-announce\.gentoo\.org
.gentoo-announce/

:0
* ^List-Id:.*gentoo-gwn\.gentoo\.org
.gentoo-gwn/

# Catch email from Gentoo not related to the lists (IE: Forums,Bugs)
:0
* ^From:.*gentoo\.org
.gentoo/

# Catch all email directed to my business email address:
:0
* ^To:.*myrealname\@apparition\.ath\.cx
.business/

## All the rest of our email will be delivered to our default INBOX
## so no additional rule is needed

As you can see, I have a very simple procmailrc file. You could do real special things with procmail such as set up autoresponders, automatically forwarding email, parse the email and call external applications. It's really a powerful piece of software, but for our needs, this example file works nicely. Make sure any maildir you wish to filter to is preceded with a dot (.) and that a forward slash (/) follows. This will deliver email in maildir format.

We only need to make our base maildir, procmail will create any other directory structure you need. Let's make our default maildir.

user@server # maildirmake ~/.maildir/

Providing IMAP Email Access

We use Courier-IMAP as the server to provide access to our email from anywhere on the Internet or in our LAN. We chose this piece of software since it's designed to work with maildir's. We've already emerged the software, so let's configure it.

Setting Up Authentication

First thing to do is change to root and check that authdaemon is running with the appropriate method:

AUTHDAEMOND="authdaemond.plain"

If you didn't find the file in /etc/courier/authlib/ look at /etc/courier-imap/ .

We've told the authdaemond to use a plain method. It simply means we aren't going to use some of the more robust solutions such as mysql or ldap. After that variable has been set, we need to edit the conf file for authdaemond. Let's make sure that the authmodulelist is using pam. Again, as mentioned in the beginning of this guide, this setup is not for hundreds of users, so pam fits the bill nicely.

authmodulelist="authpam"

Since we're authing against pam, please make sure the imap pam file matches mine exactly. Now these values should be there by default, but just in case, they are provided here.

# PAM setup for
auth       required     pam_nologin.so
auth       required     pam_stack.so service=system-auth
account    required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

Since pam-0.99.8.1-r1 they removed pam_stack.so

# PAM setup for
auth       required     pam_nologin.so
auth       include      system-auth
account    required     pam_nologin.so
account    include      system-auth
session    required     pam_nologin.so
session    include      system-auth

Adding SSL Support

As mentioned, we want to only use SSL to connect to our IMAP server. Since we have chosen a safer method of authentication, it requires a bit more work. Let's do it now while we're still as root:

[ req_dn ]
C=CA
ST=ON
L=Toronto
O=Mail Server
OU=Automatically-generated IMAP SSL Key
CN=localhost
emailAddress=root@localhost

As you can see, I've changed the variables to match my network and location. I recommend you do the same. It doesn't really matter, but you should do it anyway. You can find all the variables to change in the "[ req_dn ]" section of the file. After you've done that, we can make our certificate file:

root@server # cd /etc/courier-imap && mkimapdcert

Automating and Finalizing

We've created the base maildir in section 4, all that's left is adding the servers to our default runlevel and starting the servers. Since we don't want IMAP to authenticate without SSL, we have only started the appropriate server. Make sure that courier-authlib: authdaemond started as a dependency.

root@server # /etc/init.d/courier-imapd-ssl start
 * Starting courier-authlib: authdaemond...                               [ ok ]
 * Starting courier-imapd over SSL...                                     [ ok ]
root@server # rc-update add courier-imapd-ssl default
 * courier-imapd-ssl added to runlevel default
 * Caching service dependencies...                                        [ ok ]

Please remember, the username and password combination that you use to authenticate here is the same pair found in chart 2.1. It is the same username / password you use to login to the server.

Fetching Email External Sources

Fetchmail is a program that allows a user to fetch email from various external servers. It's a great little program that can handle just about any protocol (IMAP/S - POP3). Fetchmail does not need to run as root, so let's not have any more programs running as super user than needed. First thing to do is drop out of root.

Setting Up The Configuration File

First we will go to our home directory, create the file and then add a configuration. We'll discuss what goes where and how to customize this file to your unique setup after. First let's look at the commands and template-like view of the fetchmail file.

set postmaster "[SERVER-USERNAME]"
poll [IMAP-SERVER] with proto IMAP user "[IMAP-USER]" there with password "[IMAP-PASSWORD]" is [SERVER-USERNAME] here options warnings 3600

As you can see, the options are surrounded with square brackets ([]). In chart 2.1 you have recorded your server's username. Substitute [SERVER-USERNAME] with your username. In chart 2.2 we gave two examples of servers which we could fetch email from. They are under the headings "IMAP/SSL" and "POP3/No SSL". Let's assume that this is my fetchmail file and the server I am fetching email from (polling) is under the "IMAP/SSL" heading. Here's what my .fetchmailrc file would look like:

set postmaster "21s-beo"
poll imap.fastmail.com with proto IMAP user "beo_agate" there with password "rimap-pass" is 21s-beo here options warnings 3600

As you can see, we added "auth password" to our poll line. This tells fetchmail not to use SSL when trying to fetch the email.

Chances are some of you have more than one email account that you'd like to fetch. Luckily, fetchmail can handles this with ease. Here's our two examples above combined into one file:

set postmaster "21s-beo"
poll imap.fastmail.com with proto IMAP user "beo_agate" there with password "rimap-pass" is 21s-beo here options warnings 3600
poll pop.huah.com with proto POP3 auth password user "beowulf_999" there with password "rpop-pass" is 21s-beo here options warnings 3600

Now that we've configured fetchmail, let's change its permissions. Fetchmail is picky about the permissions of this file, so to meet its requirements, we must chmod our file:

user@server $ chmod 600 ~/.fetchmailrc

If you are a HotMail user, you might also consider installing GotMail. A simple solution to this problem is detailed by marienZ can be found by clicking here. I have not tested this as I do not have a HotMail account, but from what I've heard here and elsewhere, gotmail works fine, even as a replacement.

Automating and Finalizing

Since we're using Fetchmail in non-daemon mode, we'll use cron to emulate it. Here's the correct cron line, however I have found vcron tends to choke on it.

*/10 * * * * /usr/bin/fetchmail -a -s -m "/usr/bin/procmail -d \%T"

The above code block adds a cron job that will run every 10 minutes and fetch email and then passes it directly to procmail. Only add the -a option if you haven't configured your .fetchmailrc-accounts with the keep statement. Thanks to requiem for the correct crontab line. If you're like me though, you'll want a script that you can run whenever you want. That way, if somebody tells you to check your email, you simply ssh into your server and run the script. This script should also be used in your cron in case cron chokes on the line above. As a normal user, enter this:

user@server $ mkdir ~/bin
user@server $ chmod 700 ~/bin
user@server $ echo -e "\043\041/bin/bash\n/usr/bin/fetchmail -a -s -m \"/usr/bin/procmail -d %T\"" > ~/bin/getmyemailnow
user@server $ chmod +x ~/bin/getmyemailnow

Now all you have to do is add a line to cron similar to this (edit cron.mail found in the user's home directory):

*/10 * * * * $HOME/bin/getmyemailnow

NOTE: If you're using an anti-virus system, you should instead have fetchmail redirect to port 25 (the default option). You will need to edit /etc/postfix/main.cf and edit this line:

smtpd_recipient_restrictions = permit_sasl_authenticated, reject

smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject

This will allow fetchmail to send email through postfix so that it can be scanned. Thanks to Advo for letting me know about this caveat.

Part II

To continue, see Part 2 of this guide.