Email Server
2015-10-29
I recently purchased a VPS from EcoVPS to replace an old box in North Dakota and hosting space with DreamHost. DreamHost used to handle my email, but now it won't, so I'm installing an email server. Here is some information about my installation process, along with some updates in subsequent years.
Of all the casual server administration stuff I've done over the past twenty years, setting up an email server with properly-functioning spam settings for both incoming and outgoing email has been far more difficult than anything else because there are many different components, some of the ideas are unintuitive, and a lot of the file syntax is old and slightly confusing. If you're considering setting up your own, take your time and make sure you really want to put in the effort. If you go for it, great, just do everything step by step and expect it to take a long time.
VPS (2015)
In 2015, I used Debian 8.2. The VPS had a gigabyte of RAM, 250 GB of storage, and one CPU.
$ uname -a Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) x86_64 GNU/Linux
$ lsb_release -a Description: Debian GNU/Linux 8.2 (jessie) ...
$ free --human total used free shared buffers cached Mem: 993M 885M 108M 21M 129M 602M -/+ buffers/cache: 153M 840M ...
$ df --human-readable Filesystem Size Used Avail Use% Mounted on /dev/sda1 246G 91G 143G 39% / ...
$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 1 On-line CPU(s) list: 0 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 44 Model name: Intel(R) Xeon(R) CPU X5660 @ 2.80GHz ...
VPS (2019)
As of 2019, I use Debian 9.7. The VPS has two gigabytes of RAM, 250 GB of storage, and one CPU.
$ uname -a Linux dperkins 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64 GNU/Linux
$ lsb_release -a Description: Debian GNU/Linux 9.7 (stretch) ...
The VPS has two gigabytes of RAM.
$ free --human total used free shared buff/cache available Mem: 2.0G 447M 102M 87M 1.4G 1.3G ...
$ df --human-readable Filesystem Size Used Avail Use% Mounted on /dev/sda1 246G 130G 104G 56% / ...
$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 1 On-line CPU(s) list: 0 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 26 Model name: Intel(R) Xeon(R) CPU X5667 @ 3.07GHz ...
MX Records
When you own a domain name, there are several ways that other computers use the domain name to connect to your server. For email, the MX record must be properly configured. If it isn't, other email servers won't find your email server.
My DNS is handled by DreamHost, so I go to the DreamHost Web Panel and configure the record there. My new record is...
10 dperkins.org.
This is the simplest case. I have one computer that handles the email for one domain. DNS changes can take hours to propagate around the world. This is a good time to take a break. Later, I check and see that the changes are widely available using iptools.biz and dig.
$ dig dperkins.org MX
Postfix
Postfix is a mail transfer agent. It is the main piece of server software for handling email. There is an out-of-date tutorial on the topic that makes for a handy reference.
# apt install postfix
When prompted, I choose the "Internet Site" option and enter my domain name, dperkins.org, when prompted. The rest of the installation is automatic. After apt finishes working, test the installation as follows.
$ telnet localhost 25 Trying ::1... Connected to localhost. Escape character is '^]'. 220 debian ESMTP Postfix (Debian/GNU)
I also test connecting via the domain name.
$ telnet dperkins.org 25 Trying 127.0.0.1... Connected to dperkins.org. Escape character is '^]'.
This is a good time for me to email myself. Still within telnet from just above...
mail from:<fakename@dperkins.org> rcpt to:<fakename@dperkins.org> data To: fakename@dperkins.org From: fakename@dperkins.org Subject: Test email This is a email on Debian using Postfix.
To end data, press enter, type a dot, and press enter again.
.
Then quit.
quit
Now, when logged in as fakename I can see if the email arrived by running mail.
This all works, and the basic Postfix installation is complete. Here is information on SMTP authentication which is used when sending email from client-side programs (e.g., Thunderbird). And here is /etc/postfix/main.cf.
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) biff = no append_dot_mydomain = no readme_directory = no smtpd_tls_cert_file=/etc/ssl/local/dperkins.crt smtpd_tls_key_file=/etc/ssl/local/dperkins.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_sender_restrictions = permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_non_fqdn_recipient, reject_unknown_recipient_domain smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination myhostname = dperkins.org alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname mydestination = dperkins.org, debian, localhost.localdomain, localhost relayhost = mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 mailbox_command = procmail -a "$EXTENSION" mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all
By default, Postfix listens for SMTP connections on port 25. Some ISPs block port 25 because historically it was linked to spam problems. If you have a situation where you can send email from one location (your house) but not another (your WiFi network), there's a good chance that the port is blocked. The solution is for Postfix to listen on port 587 as well. In /etc/postfix/master.cf there is a line like this.
#submission inet n - n - - smtpd
Remove the leading # so it looks like this.
submission inet n - n - - smtpd
Restart Postfix and it will now listen on both ports.
For me, the default maximum attachment size in Postfix is 10 MB.
# postconf | grep message_size_limit message_size_limit = 10240000
I'd prefer to make it larger, as detailed here. I edit /etc/postfix/main.cf and insert the following line.
message_size_limit = 20480000
I restart Postfix and can now send attachments of up to 20 MB.
Dovecot
To check and read email remotely, I need to install an IMAP server such as Dovecot.
# apt install dovecot-imapd
This should install the server. Test it with mutt. If you can log in, and if you can read the emails you sent to yourself when configuring Postfix above, it is to some degree functional.
$ mutt -f imap://fakename@localhost/Inbox
To make Postfix communicate with Dovecot for SMTP authentication, I uncomment and add the following to /etc/dovecot/conf.d/10-master.conf.
service auth { ... unix_listener /var/spool/postfix/private/auth { mode = 0660 user = postfix group = postfix } ... }
In /etc/dovecot/conf.d/10-auth.conf I add login to the end of the following line.
auth_mechanisms = plain login
To make my mailbox have separate folders for Inbox, Drafts, Trash, and Sent, I modify /etc/dovecot/conf.d/10-mail.conf as described here. The last line below fixes a Postfix/Dovecot bug described here
mail_location = mbox:~/mail:INBOX=/var/mail/%u namespace inbox { type = private inbox = yes mailbox Trash { auto = subscribe special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } } mail_privileged_group = mail
For SSL support, I modify /etc/dovecot/conf.d/10-ssl.conf as described here.
ssl = required ssl_cert = </etc/ssl/local/dperkins.crt ssl_key = </etc/ssl/local/dperkins.key
Roundcube
To exchange email in my web browser, I install RoundCube.
# apt install roundcube roundcube-sqlite3
Follow the installation instructions. The easiest database to use is sqlite. If you only have a few users, this is probably what you want.
At this point, we need to enable Roundcube in Nginx or Apache. If you already have PHP working in your web server, adding Roundcube will be quick enough. If you don't, look for online docs but expect to spend a long time figuring it out.
Some of the final steps in this setup describe how to sign outgoing email, and Roundcube needs a tweak to make that work. If you get this error when trying to send a message ...
SMTP Error: [550] 5.7.1 rejected by DMARC policy for dperkins.org
... then add this line to /etc/roundcube/config.inc.php.
$config['smtp_server'] = 'tls://%t';
Now Roundcube is sending email over TLS, just like your other email clients.
Reverse DNS
Certain email providers try to stop spam by requiring that a reverse DNS lookup functions properly for the email sender's domain. If you try to send an email and it bounces with an error talking about Connections not accepted from servers without a valid sender domain ... Fix reverse DNS ..., it's probably because you haven't configured this.
A simple explanation of reverse DNS is that it’s the exact opposite of DNS. Standard (aka forward) DNS maps a domain name to an IP address whereas reverse DNS maps an IP address to a domain name. The two are distinct and separate lookups however. Just because a forward lookup of example.com resolves to 1.2.3.4 doesn’t mean that a reverse lookup of 1.2.3.4 will resolve to example.com.
— Mathew Mombrea. IT World. 2013-06-26.
I own a domain, dperkins.org. My domain dperkins.org resolves to the IP address 77.235.46.155. That's what we normally want from a domain name. Some spam filters want to go the other direction. They want the IP address 77.235.46.155 to map to dperkins.org. Let's see what things look like before I start debugging.
$ dig -x 77.235.46.155 | grep -A 1 'ANSWER SECTION' ;; ANSWER SECTION: 155.46.235.77.in-addr.arpa. 0 IN PTR git.dperkins.org.
$ nslookup 77.235.46.155 Server: 127.0.0.1 Address: 127.0.0.1#53 155.46.235.77.in-addr.arpa name = git.dperkins.org.
The PTR record for my IP is not what I want. Where it reads git.dperkins.org, I want it to read dperkins.org. This information can also be confirmed by web interface.
PTR records require authoritative DNS nameservers before they can function properly. To find the authoritative DNS nameservers of your server's main IP address, trace the Start Of Authority (SOA). Changes to your server's DNS nameservers do not take effect if your server's DNS nameservers are not authoritative for your IP address. Many hosting providers do not delegate authority for PTR records to their customers. Contact your upstream provider to either delegate authority to your nameservers or set up PTR records for your nameservers.
— cPanel. Updated 2015-11-18.
To see who controls the PTR record, we do an advanced search as follows. Realistically, if you're renting a VPS from a company, that company probably controls the PTR record, but taking a few moments to confirm matters is sensible enough.
$ dig +nssearch 46.235.77.in-addr.arpa SOA ns1.hspc.sbp.attikh.net. root.hspc.corp.sbp.attikh.net. 2015101919 7200 3600 4294967295 3600 from server 77.235.35.160 in 391 ms. SOA ns1.hspc.sbp.attikh.net. root.hspc.corp.sbp.attikh.net. 2015101919 7200 3600 4294967295 3600 from server 77.235.33.160 in 400 ms.
The server controlling that IP address is attikh.net, and who.is shows us that it is owned by EuroVPS, the company leasing me the VPS. I contact EuroVPS via the support panel. They come through in matter of minutes. Their tech support person adds the PTR record as desired. Here's what it looks like after tech support does their stuff.
$ dig -x 77.235.46.155 | grep -A 1 'ANSWER SECTION' ;; ANSWER SECTION: 155.46.235.77.in-addr.arpa. 600 IN PTR dperkins.org.
$ nslookup 77.235.46.155 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: 155.46.235.77.in-addr.arpa name = dperkins.org. Authoritative answers can be found from:
DNS information takes hours to propagate; be patient after changes are made.
Blacklists
A few email addresses from a few domains send me spam on a regular basis. That's easy enough to handle — blacklist those email addresses and domains and be happy. This can be done in Postfix and only takes a few minutes. First, I make a file, /etc/postfix/sender_access.
thinkvantageclub@lenovo-news.com REJECT web-marketing-company.com REJECT
The first line shows an email address to reject. The second line shows an entire domain to reject. Once the file is prepared (and later, if it is changed), convert it to a format that Postfix can read.
# cd /etc/postfix # postmap sender_access
This produces a new file, sender_access.db. Next, I tell Postfix to filter incoming email using this file. In main.cf, I extend smtpd_recipient_restrictions by adding the check_sender_access line below. Order is important.
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, check_sender_access hash:/etc/postfix/sender_access, reject_unauth_destination
Restart Postfix for the changes to take effect.
Spamassassin
Blacklists help with spam, but I still get some, so I install Spamassassin.
First we add a Junk folder to our inbox. When email arrives, we give it to Spamassassin, and if Spamassassin decides it's spam, we will put it in the spam folder instead of the inbox. To create this folder, edit /etc/dovecot/10-mail.conf. The bottom section below is new.
namespace inbox { type = private inbox = yes mailbox Trash { auto = subscribe special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Junk { auto = subscribe special_use = \Junk } }
Restart Dovecot and the new Junk folder will be created. Next, I install Spamassassin.
# apt install spamassassin spamass-milter
Incoming mail is handled by Postfix, and we want Postfix to filter email through Spamassassin. In /etc/postfix/main.cf add these lines.
smtpd_milters = unix:/spamass/spamass.sock milter_default_action = accept
In /etc/default/spamass-milter.conf, modify the following line as follows. The -I argument bypasses spam filtering on my outgoing mail.
OPTIONS="-u spamass-milter -i 127.0.0.1 -I"
Note that Postfix runs in a chroot, and the above socket is the path inside the chroot. The non-chroot path is /var/spool/postfix/spamass/spamass.sock.
Restart Postfix and the milter.
# service postfix restart # service spamass-milter restart
To test that it's working, send an email to yourself (from an account on another server) to confirm. You should receive the email, and the header should contain something similar to this.
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on debian X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.0
To get daily updates for Spamassassin rules, edit /etc/default/spamassassin and either uncomment this line or change it from 0 to 1.
CRON=1
Now Spamassassin is working.
Razor & Pyzor
Razor and Pyzor are Spamassassin add-ons that check incoming emails against public spam lists. We tend to install them together, because it's easy and convenient, but you could use just one of the two perfectly happily.
# apt install razor pyzor
Set Razor up as follows.
# razor-admin -home=/etc/razor -register # razor-admin -home=/etc/razor -create # razor-admin -home=/etc/razor -discover
Enable both add-ons by adding the following lines to the end of /etc/spamassassin/local.cf.
razor_config /etc/razor/razor-agent.conf use_pyzor 1
Restart Spamassassin.
# /etc/init.d/spamassassin restart
Test that they're working using commands like these.
# echo "test" | spamassassin -D razor2 2>&1 | grep razor # echo "test" | spamassassin -D pyzor 2>&1 | less
Procmail
Using Spamassassin, junk mail is flagged as junk, but it still comes to your inbox. Thunderbird will automatically move junk mail to the Junk folder, but my cell phone won't. Instead, I use procmail to handle it. This is done on a per-user basis.
When Postfix receives mail, it runs each message through Spamassassin and then gives the message to procmail. Procmail puts it in each user's mail directory. Create the file /etc/procmailrc as follows.
:0: * ^X-Spam-Status: Yes $HOME/mail/Junk
It's done. New spam will now be sent to the Junk folder.
LetsEncrypt
In early 2016, I started getting SSL certificates from LetsEncrypt. Postfix and Dovecot settings needed minor adjustment.
For Postfix, two lines need changing in /etc/postfix/main.cf.
smtpd_tls_cert_file=/etc/letsencrypt/live/dperkins.org/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/dperkins.org/privkey.pem
For Dovecot, two lines need changing in /etc/dovecot/conf.d/10-ssl.conf.
ssl_cert = </etc/letsencrypt/live/dperkins.org/fullchain.pem ssl_key = </etc/letsencrypt/live/dperkins.org/privkey.pem
That's all.
SPF records
One way that servers can reduce spam sent from their domains is using SPF records. You don't have to configure SPF, but if you do, other servers are less likely to classify email from you as spam, so it's a good idea.
We need to create a new DNS record. Using spfwizard, I can quickly create the string I need. Here's what I enter in the wizard.
Your Domain: dperkins.org Allow servers listed as MX to send email for this domain: Yes (recommended) Allow current IP address of the domain to send email for this domain: Yes (recommended) Allow any hostname ending in dperkins.org to send email for this domain: No (recommended) IP addresses in CIDR format that deliver or relay mail for this domain: 77.235.46.155/32 How strict should be the servers treating the emails?: SoftFail (Not compliant will be accepted but marked)
From that, the wizard gives me the following string.
dperkins.org. IN TXT "v=spf1 mx a ip4:77.235.46.155/32 -all"
That string belongs in my DNS entry, which is over at DreamHost. I go to the DreamHost Panel, manage my domain, and add a custom DNS record. It looks like this.
Name: Type: TXT Value: "v=spf1 mx a ip4:77.235.46.155/32 -all" Comment: SPF
It can take time for DNS records to propagate. After they do, check that everything looks good. This can be done using an SPF checking website or by the command line.
$ dig dperkins.org txt | grep spf dperkins.org. 14400 IN TXT "v=spf1 mx a ip4:77.235.46.155/32 -all"
The above entry is good. Specifically, the end section in quotation marks is intact. SPF is configured for our server.
SPF filtering
We want to block spoofed incoming email by confirming the remote domain's SPF data. To do that, we need an extra Postfix package.
# apt install postfix-policyd-spf-python
Configure the SPF settings for incoming mail. Modify /etc/postfix-policyd-spf-python/policyd-spf.conf so that it contains the following lines, a fairly aggressive rejection policy.
HELO_reject = SPF_Not_Pass PermError_reject = True
Add the following line to the end of /etc/postfix/main.cf.
policy-spf_time_limit = 3600s
Then add the following to the end of /etc/postfix/master.cf.
policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf
Finally, modify /etc/postfix/main.cf and add the following lines. Make sure the bottom two lines are in the listed order, and that they appear below the top two lines.
smtpd_recipient_restrictions = ... permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf ...
Restart Postfix.
# service postfix restart
For a quick initial test, first run tail -f /var/log/mail.log in the terminal and then email yourself, first from the local machine and then from another account. If you don't receive these emails, something is definitely broken. Check out the log file a day or two later to see if domain-spoofed spam has been blocked.
DKIM
DKIM is an email authentication method. A DKIM filter adds a digital signature to mail I send, and other servers can check that the signature corresponds to information in my DNS entry. This is another anti-spam tool that you don't have to configure, but if you do, your mail looks better to spam filters elsewhere. Here are some instructions.
First I install the DKIM program.
# apt install opendkim opendkim-tools
Next I make a DKIM certificate and adjust the permissions.
# mkdir /etc/opendkim # opendkim-genkey -D /etc/opendkim/ -d dperkins.org # chgrp opendkim /etc/opendkim/* # chmod g+r /etc/opendkim/*
There are two files in /etc/opendkim/, a public key and a private key. I take a look at the public key file, default.txt.
default._domainkey IN TXT ( "v=DKIM1; k=rsa; "p=MIG...QAB" ) ; ----- DKIM key default for dperkins.org
The long string (truncated here) needs to go in my server's DNS entry. At the DreamHost Panel, I make a new custom DNS entry.
Name: default._domainkey Type: TXT Value: "v=DKIM1; k=rsa; p=MIG...QAB" Comment: DKIM
It takes some time for DNS to propagate. Everything is good when the following DNS query returns this result.
$ dig default._domainkey.dperkins.org txt | grep DKIM default._domainkey.dperkins.org. 14400 IN TXT "v=DKIM1; k=rsa; p=MIG...QAB"
I edit /etc/opendkim.conf and add the following lines to the bottom.
AutoRestart Yes AutoRestartRate 10/1h Canonicalization relaxed/simple Mode sv Domain dperkins.org KeyFile /etc/opendkim/mail.private PidFile /var/run/opendkim/opendkim.pid Selector default Socket inet:8891@localhost
OpenDKIM is configured. Restart it.
service opendkim restart
We want Postfix to filter incoming email and sign outgoing email through OpenDKIM. Modify the following lines in /etc/postfix/main.cf.
smtpd_milters = inet:localhost:8891, unix:/spamass/spamass.sock non_smtpd_milters = inet:localhost:8891
Restart Postfix and send a test email to see if it's all looking good.
service postfix restart
Configuring DKIM is quite delicate; even a small error could break things. To test your setup, if you have a GMail account, send an email to your GMail account. Take a look at /var/log/syslog. Then go over to GMail and see what the received email headers look like.
Also, if you send an email to check-auth@verifier.port25.com, it will send a reply email with information about your DKIM and SPF configuration. This is fast and convenient.
ADSP
There is a DKIM extension called ADSP that is historical and not widely used these days, but it only takes ten seconds to enable. After DKIM is working, I add a txt DNS record for my domain.
_adsp._domainkey.dperkins.org
This record has only one tag, which states that all of my outgoing email is supposed to have a DKIM signature.
dkim=all
DMARC for outgoing email
DMARC is a newer, better, and more popular extension similar to ADSP. It tells other servers what sorts of digital signatures to expect from email using your own domain. To enable it for my outgoing emails, I add a txt DNS entry.
_dmarc.dperkins.org
The value of the tag is long and confusing, but you can create your own using a DMARC record generator. Here is mine.
v=DMARC1; p=reject; rua=mailto:webmaster@dperkins.org
While setting things up, you could use p=none, and once it looks OK, change that to p=reject.
DMARC for incoming email
We want to use Postfix to check incoming email against DMARC and filter out the bad stuff.
# apt install opendmarc
In /etc/opendmarc.conf, add the following lines.
AuthservID HOSTNAME Socket inet:8893@localhost IgnoreAuthenticatedClients true
In /etc/default/opendmarc, make sure all lines starting with SOCKET are commented out. After that, restart OpenDMARC.
# service restart opendmarc
Configure Postfix to use OpenDMARC by modifying the following line in /etc/postfix/main.cf. The 8891 filter is DKIM (from above), the 8893 filter is DMARC, and spamass.sock is Spamassassin.
smtpd_milters = inet:localhost:8891, inet:localhost:8893, unix:/spamass/spamass.sock
Restart Postfix and test that it all works using some test emails to and from an account on this and another server.
After you've used the setup for a few weeks and feel that incoming email is properly classified, it's time to block messages that fail the DMARC check. In /etc/opendmarc.conf, change the following from false to true.
RejectFailures true
Restart OpenDMARC and get less spam.