my-vault/04_Archive/Inbox-Clippings/2023/04/Self hosted mail server – if then else.md

page-title, url, date

page-title	url	date
Self hosted mail server – if ? then : else	https://www.ifthenel.se/self-hosted-mail-server/	2023-04-27 17:18:34

I have had enough. It’s bad enough that social media has become the scourge of the Internet, but that doesn’t mean key services such as email should have to be stronghold(ed?) by Big Corp such as Google, Microsoft, and other massive vendors.

The Internet was built on a foundation of globally connected computers that chose to share information between eachother and all was well.

Now, pretty much _EVERYTHING_ on the web is behind some kind of walled garden and/or being harvested for user behaviour studies to increase revenue (through ads, mostly).

But I’d like to see if I can do something about this, in as far as my capabilities goes; it’s time to “take back” (I’ve actually never hosted my own email server before, so it’s more of a “move the responsibility of”) my emails.

This is a part guide, part resource, part tutorial on how I set my email up on a VPS service. You won’t be getting every single step as a command or how to edit your DNS records for your control panel on your registrar, but you should at least get a fairly detailed explanation of all the steps that are involved.

A word of thanks

I’d like to thank all the people involved for their tireless efforts in providing a solid out-of-the-box selfhostable mail server. I am a huge fan of all the individuals that put their time into creating a set of tools such as docker-mailserver and make it _next_ to a no-brainer to get it up and running.

Let’s get to the grind.

Tech stack

I have chosen to use the following techs

• External VPS hosting – It is fairly cheap, and email shouldn’t require exorbitant amounts of resources, and you _have to_ be able to edit reverse DNS records
• Debian 10 – No particular reason other than I like Debian
• docker-mailserver – It has proven to be a one stop shop for an easy (well) get up and running component stack of different software that can deal with email

Look at your inventory before you move on

IP blacklisting delisting

Make sure your chosen server host isn’t on any kind of IP or email blacklist. This activity is a tough one with a big risk of being costly (as in $$$) since you

• probably don’t know what IP you get (this post assumes 123.123.123.123)
• don’t know if that IP is on any list yet
• may not know of any tools can help you figure out what lists you are on

I don’t have an answer to all of this. You can use MX Toolbox’s blacklist lookup tool, but there are probably a lot more blacklisting going on that you will want to try and find so you have an ounce more chance of not getting you emails flagged as spam from the getgo.

I got to know my IP was on the BARRACUDA blacklist and I made an attempt at sending in a delisting request through their online form, and I seem to have been released from their clutches.

Microsoft/Outlook/Office are picky as all get out as well, so make additional efforts to get your IP delisted from there as well, even if you suspect you’re not on their list: https://sender.office.com/

rDNS (reverse DNS)

Get a host that supports rDNS (reverse DNS) and set a reverse value to something that you own, so that when a lookup happens (be it forward or reverse), they get the same hostnames and IP. E.g.;

$ host my.host // returns: "my.host has address 123.123.123.123"
$ dig +noall +answer -x 123.123.123.123 // returns: "123.123.123.123-in-addr.arpa [number] IN PTR my.host"

You could also use MX Toolbox reverse lookup tool to test this by supplying your mail host’s IP address and have it look this up for you.

Setting up docker-mailserver (and moar)

docker-mailserver

THIS GUIDE ONLY COVERS docker-mailserver v10.2 AND LATER!

1. Install docker and docker-compose on your host
2. Clone the docker-mailserver git repo (this way you will get all necessary files to start off including one of my personal favourites; docker-compose.yml): git clone https://github.com/docker-mailserver/docker-mailserver
3. Read the docker environment and topics following in the official README https://github.com/docker-mailserver/docker-mailserver#create-a-docker-compose-environment
4. Do as the README suggests and then start the mailserver
   $ docker-compose up
5. Wait for the startup to fail with a message along the lines of mailserver | [ ERROR ] Shutting down..
6. Shut down the server again
   $ docker-compose down

Here be dragons. Read this. It’s important. I have based a lot of this tutorial from https://docker-mailserver.github.io/docker-mailserver/edge/examples/tutorials/basic-installation/ but this specific step we are coming to, creating keys for DKIM, is missing a crucial part; you have to create a mail account on your mail server before generating the DKIM. Why? No idea yet. Why isn’t it mentioned in the doc? Beats me. But you should really really make it clear to the authors of this document that this is a serious flaw to not have included. I spent a lot of time on google to figure this out…..

1. Add an account to your mailserver:
   $ docker-compose run mailserver setup email add someone@example.com and input a password when requested
2. Set up DKIM with your docker-mailserver tag of choice:
   $ docker-compose run mailserver setup config dkim

When you have gotten this far without issues, you should be able to start the mailserver without the initial errors:
$ docker-compose up -d

You should now be able to interact with the server using the command pattern docker-compose exec mailserver from here on.

DNS

This part is tricky, because DNS records are finicky, they need to be thoroughly tested and take HEAPS of time to update. You will spend quite some time with this if you don’t get it right on the first try. Get yourself a couple of days vacation for this part.

DKIM

• TXT record
• Hostname: mail._domainkey.my.domain
• Value: Whatever is in the opendkim text file

Add the newly created DKIM details to your domain’s DNS records by first echoing them for easy copy and paste:
$ cat config/opendkim/keys/my.domain/mail.txt

This produces something you should add to your DNS records. Here’s the kicker; The strings are long and deliberately cut off because the maintainers don’t know if your DNS hoster supports strings that are longer than 255 characters in length. So, you will have either a good time entering these as you might know your DNS hoster’s limitations, or, more likely, a bad time and you have to figure out how to get these details stored properly. There’s some help though. MX Toolbox has a DKIM checker that you can utilise to verify you have your stuff set up right: https://mxtoolbox.com/dkim.aspx Good luck.

In the DNS editor, your hostname for this TXT record should be mail._domainkey.my.domain and the value should be the long ass string output from the cat command above.

After a long wait, check with the MX Toolbox DKIM Lookup

SPF

• TXT record
• Hostname: @
• Value: v=spf1 mx ~all

This is simpler, because there are very few parameters. It’s still required for your emails to have a decent chance of not being marked as spam. There can be some benefit to adding refernces to other known SPF records, such as tthe ones used by Outlook, etc. One of those is “spf.protection.outlook.com“

Add a TXT record to your @ (hostname/zone/whatit’scalled) with the value: v=spf1 mx ~all

Tip: If you want to add Microsoft Outlook’s SPF, you would change the value to v=spf1 include:spf.protection.outlook.com mx ~all

Subdomain

• A record
• Hostname: mail
• Value: IP address of your mail server host

Also a simpler DNS record change. You want to have a subdomain that keeps tabs on the IP address of you mail server host.

1. Add an A record to the subdomain mail with the value of your mail server host IP address

MX

• MX record
• Hostname: @ or mail.my.domain.
• Value: mail.my.domain

This is used for telling every server on the internets that you have a mail server somewhere with a certain subdomain name

1. Add an MX record to the @ (hostname/zone/whatit’scalled) with the value:
   mail.my.host.

DMARC

• TXT record
• Hostname: _dmarc.my.domain
• Value: v=DMARC1; p=quarantine; rua=mailto:postmaster@my.domain; ruf=mailto:postmaster@my.domain; fo=1; adkim=s; aspf=s; pct=100; rf=afrf; ri=86400; sp=quarantine

Complicateder piece of DNS record than the last two, because there are a bunch of parameters in the TXT records that you should read up on and understand. I’ll just give you what I use for now.

1. Add a TXT record to the _dmarc hostname with the value: v=DMARC1; p=quarantine; rua=mailto:postmaster@my.domain; ruf=mailto:postmaster@my.domain; fo=1; adkim=s; aspf=s; pct=100; rf=afrf; ri=86400; sp=quarantine

Testing

Here’s the part where you need to verify and verify again that everthing is in working order. Mostly MX Toolbox to the rescue.

MX

• https://mxtoolbox.com/MXLookup.aspx
• Format: my.domain

What you are looking for in the results is something along the lines of

• Pref: 0
• Hostname: my.domain
• IP address: 123.123.123.123

If there are more than one result in the table, you want to ensure your email server, the one with the IP of 123.123.123.123, is at the top, with a “Pref” (priority) value that is lower than any of the others.

SPF

• https://mxtoolbox.com/spf.aspx
• Format: my.domain

What you are looking for in the results is something along the lines of 4 rows, detailing the parameters of the SPF record. For instance, if the SPF record in DNS was set to: “v=spf1 include:spf.protection.outlook.com mx ~all” you want a result such as:

v

spf1

The SPF record version

include

spf.protection.outlook.com

Pass

The specified domain is searched for an ‘allow’.

mx

Pass

Match if IP is one of the MX hosts for given domain name.

~

all

SoftFail

Always matches. It goes at the end of your record.

DKIM

• https://mxtoolbox.com/dkim.aspx
• Format: my.domain:mail

What you are looking for in the results is something along the lines of 4 rows, detailing the parameters of the DKIM record. For instance, if the DKIM (mail._domainkey) was set to: “v=DKIM1; h=sha256; k=rsa; p=jasdfjidjelajdvlkjg” you want a result such as:

v

DKIM1

Version

Identifies the record retrieved as a DKIM record. It must be the first tag in the record.

h

sha256

Hash Algorithms

A colon-separated list of hash algorithms that might be used.

k

rsa (Length: 4096 bits)

Key Type

Key Type The type of the key used by tag (p).

p

jasdfjidjelajdvlkjg

Public Key

The syntax and semantics of this tag value before being encoded in base64 are defined by the (k) tag.

DMARC

• https://mxtoolbox.com/dmarc.aspx
• Format: my.domain

What you are looking for in the results is something along the lines of 10 rows, detailing the parameters of the DMARC record. For instance, if the DMARC (my.domain) was set to: “v=DMARC1; p=quarantine; rua=mailto:postmaster@my.domain; ruf=mailto:postmaster@my.domain; fo=1; adkim=s; aspf=s; pct=100; rf=afrf; ri=86400; sp=quarantine” you want a result such as:

v

DMARC1

Version

Identifies the record retrieved as a DMARC record. It must be the first tag in the list.
p quarantine Policy Policy to apply to email that fails the DMARC test. Valid values can be ‘none’, ‘quarantine’, or ‘reject’.

p

quarantine

Policy

Policy to apply to email that fails the DMARC test. Valid values can be ‘none’, ‘quarantine’, or ‘reject’.

rua

mailto:postmaster@my.domain

Receivers

resses to which aggregate feedback is to be sent. Comma separated plain-text list of DMARC URIs.

ruf

mailto:postmaster@my.domain

Forensic Receivers

Addresses to which message-specific failure information is to be reported. Comma separated plain-text list of DMARC URIs.

fo

1

Forensic Reporting

Provides requested options for generation of failure reports. Valid values are any combination of characters ’01ds’ seperated by ‘:’.

adkim

s

Alignment Mode DKIM

Indicates whether strict or relaxed DKIM Identifier Alignment mode is required by the Domain Owner. Valid values can be ‘r’ (relaxed) or ‘s’ (strict mode).

adpf

s

Alignment Mode SPF

Indicates whether strict or relaxed SPF Identifier Alignment mode is required by the Domain Owner. Valid values can be ‘r’ (relaxed) or ‘s’ (strict mode).

pct

100

Percentage

Percentage of messages from the Domain Owner’s mail stream to which the DMARC policy is to be applied. Valid value is an integer between 0 to 100.

rf

afrf

Forensic Format

Format to be used for message-specific failure reports. Valid values are ‘afrf’ and ‘iodef’.

ri

86400

Reporting Interval

Indicates a request to Receivers to generate aggregate reports separated by no more than the requested number of seconds. Valid value is a 32-bit unsigned integer.

sp

quarantine

Sub-domain Policy

Requested Mail Receiver policy for all subdomains. Valid values can be ‘none’, ‘quarantine’, or ‘reject’.

There’s also a test results table, where you want all the checkmarks to turn up green.

TLS Cert using letsencrypt

My weapon of choice is and will probably for a long time be certbot/letsencrypt. I chose to use the officially provided script in “certonly” mode. It is fully possible to acquire a certificate without involing anything else. I had a small bump in the road where I included a domain that had an A record for my HTTP services and that failed the whole process because the HTTP service wouldn’t supply the info needed for certbot to complete. The gist of the command, though, is:

$ sudo certbot certonly --standalone -d mail.my.domain1 -d mail.my.domain2 [more parameters]

and follow the instructions. Make sure to supply the mail domain, e.g. mail.my.domain and not only the root domain, as it can make things end up like it did for me in the beginning, where I don’t have a webserver that can properly reply on the ports/URL:s that certbot expects.

Now, edit the docker-compose.yml file to add a volume that points to the letsencrypt cert store path. For my setup this would result in a line under the volumes-section:

...

• /etc/letsencrypt:/etc/letsencrypt:ro ...

Don’t forget to restart your docker-mailserver, as a reload of details is required.

Catchall for everything but valid email accounts

This took some digging around. docker-mailserver isn’t really the authority here, but postfix, that’s included in docker-mailserver. I did find a couple of nice pointers in the docker-mailserver issue tracker that gave me the following understanding to get catchall to work as I expect; Catching all email to a single catchall-specific mail account, and deliver mail designated to valid accounts without ending up in both catchall and the valid account inbox:

• Valid accounts need to have aliases to themselves
• The domain, e.g. my.domain, need to have an alias pointing to the catchall account

So, for the sake of completeness, this is how would go about it:

1. Create any valid email account(s) (these will require their own passwords)
2. Create the catchall account (this will also require its own password)
3. Create an alias to the catchall account with
   $ docker-compose exec mailserver setup alias add @my.domain @catchall.my.domain
4. Edit the newly created file config/postfix-virtual.cf and modify it to the following format:

account1@my.domain account1@my.domain
account2@my.domain account2@my.domain
...
@my.domain catchall@my.domain

The order is crucial. You want the catchall to be at the bottom of this file at all times.

Multiple domains, one server

This ties into the TLS Cert section, insofar that the command to generate valid certificates is:

$ sudo certbot certonly --standalone -d mail.my.domain1 -d mail.my.domain2 [more parameters]

and you will need to secure that you have your docker-compose.yml file set up to take into account the certificates:

...
- /etc/letsencrypt:/etc/letsencrypt:ro
...

and you need to have restarted docker-mailserver to have it react to those changes.

And, you also need to have updated the mailserver.env file to tell docker-mailserver that you’ll be using letsencrypt.

Other than that, there’s a few more things needed for your second domain. It is in general much the same as your primary domain, except for a few specific details that need to be unique to your second domain. Here’s the complete list as to not miss out on anything

• MX DNS record
• SPF DNS record
• DKIM DNS record
• DMARC DNS record
• An email account tied to the second domain

Ok, that’s a lot of stuff again! But here’s the kicker; a few of them you can use as they mostly are from your primary domain, and others are super easy to set up.

The ones you can copy and easily modify from your primary domain are

• MX record – You will use the same IP address for this as the primary one
• SPF record – You use the same TXT value as the primary one
• DMARC record – You use the same TXT value as the primary one, but I also modified the postmaster email address to align with the second domain

And this is the part where you need to do a little more work

DKIM

This is very much like generating DKIM from the setup steps in this page;

1. Create an email account, e.g. postmaster@second.domain
2. Generate the DKIM DNS record details by running $ docker-compose run mailserver setup config dkim which will generate new keyfiles for your new domain (remember that the setup-script use the domain names of already created accounts to determine what domains to generate DKIM DNS records for)
3. Copy and paste the value from $ cat config/opendkim/keys/second.domain/mail.txt and create a new DNS record as explained in the DKIM-section in the setup steps on this page.

Changelog summary, 2021-10-14