Having an SMTP server installed locally can help in a lot of different projects.
Many different server types need an SMTP server configuration to relay generated emails to users. Without such a configuration, it's impossible for many of these servers to communicate with its own users.

Email is arguably the most basic form of automated electronic configuration in the modern digital world. From account activation links to password reset tokens, a lot of basic user functionality can be achieved only if the server is equipped with a proper SMTP relay in its configuration.

But why do we even need an SMTP server in the first place? Well, we don't really need one but, at the end of the day, it's very handy to have one, nonetheless.

• How does email work?
• Postfix configuration
  • Getting the right IP list
• Instructing Postfix to relay emails through Google's servers, instead
• Configuring the Postfix service such that it always restarts when the system is running low on RAM
• Opening up port 587 for SMTP traffic
• Troubleshooting issues with Postfix reachability from docker containers
• Wrapping it up

How does email work?

Email works on different levels but the general gist of it is that it all boils down to SMTP servers acting as the backbone of all email providers.
SMTP is a protocol that allows email servers to send an email from one another, either encrypted with TLS on port 25 using the STARTTLS command, or even in plaintext.
Email clients, i.e. most of the software used to access the emails from one's inbox, use retrieval protocols such as POP3 or IMAP to access the SMTP server and download the relevant emails off the server onto the local machine to display them.

This system, while archaic in many ways, is the foundation of modern day communication. All email systems boil down to these simple concepts. Any innovative feature that they may offer on top, uses these basic principles to function behind the scenes.

So why do we need an SMTP server?

Well, the SMTP server will act as an outbound gateway for all generated emails of our various services. All email needs an originating SMTP server to be sent. If we wouldn't install one, we would need to use a third-party one instead, such as a gmail server, or a hotmail server.

This would require hardcoding our user credentials for our gmail account or hotmail account into our configurations of our various services (either that or creating a technical gmail account for this specific purpose) but this is laborious and there's always the chance that these configurations will be exposed, in case of a hack, to outside entities.

Instead, relying on our own SMTP server, we don't need to hardcode our login gmail credentials to have an email sending service available. We can just install our own one!

Excited yet? Well, you should be! In this guide, I'll configure a special type of service called Postfix. Let's start!

Postfix configuration

I'll skip the installation part, as that's distro dependent. Moreover, it's been so long since I installed Postfix on my machine that I cannot remember the exact details so that I can write them down in the first place.

As such, please follow whatever online guide you can find for your particular Linux distro on how to install and initially configure Postfix. These should be very common.

For once, I will not encourage the installation of Postfix docker image because I wish for this email functionality to be available to me even if the docker systemd service is stopped, so that I can manually send emails to specific addresses even without any docker containers running or even needing for docker to be installed on the system, at all.

Once Postfix is installed properly, test it by also installing the sendmail utility and then issuing a test email to your own personal inbox to see if it works using the following command:

The address after "sendmail" can obviously be changed to any valid email address that references your inbox so that you can quickly check if it works or not.

With that said, let's now tinker with the configuration!

The main configuration file is "/etc/postfix/main.cf". On my distro, Postfix works as a systemd service that's meant to constantly run. Be sure to run the following command after installation, to make sure the service will always start up on reboot or shut down:

Once this is done, open the aforementioned configuration file with write privileges and let's start editing lines!

First, TLS configuration:

These parameters are by no means mandatory and I would generally omit them. Postfix will use these certificates in TLS communication with clients. The above configuration will allow for STARTTLS to occur from port 25 (which is normally in text mode and unencrypted only), so that the connection will be more secure.

While I generally consider encryption to be a very nice to have feature, assuming all your services are running in docker containers on the same machine as the Postfix server, this encryption channel is unnecessary as, ultimately, all communication between email clients and the Postfix server will be, effectively, to localhost, i.e. no data will be sent over the wire. As such, there's no way for unencrypted data to be intercepted.

This configuration is presented mostly for the sake of completeness.

Now, the more relevant parameters are the following:

There's a lot to unpack there!
The easiest parameter to explain is the "myhostname" one. This tells Postfix the domain it's running under. This may or may never be relevant and I believe that this option can even be omitted (maybe).

Next is the "smtpd_relay_restrictions" which has a bunch of values assigned to it. The only one relevant to talk about is "permit_mynetworks", which informs Postfix that it's fine to relay any outgoing email from the IP addresses and hosts defined in the "mynetworks" variable, without having to authenticate them with user passwords.
Basically, this means that, as long as a service connects to port 25 of the current machine from an originating IP that's listed under "mynetworks" this means that Postfix will accept whatever email that service is trying to send and relay it over to its destination.

Please note though that the above configuration still allows emails from external entities to be relayed through this server, as has happened recently to me when Gmail was sending spam through to my Postfix instance because it was trying to send an email to "pentilescu.com", a previous domain that I still own, the email wouldn't reach its destination because the SSL certificates were not configured for that specific domain anymore, the sending would get rejected, Google would automatically send a bounce email to the originator (i.e. my VPS) again, and this would cause the VPS to flood my inbox with these unwanted emails, which can be abused by those with ill intents.

"mydestination = localhost" not sure about this one?

"mynetworks" tells Postfix which machines are trusted. SMTP needs to trust sources of email before it can relay them. If you specify "permit_mynetworks" to "smtpd_relay_restrictions" then any machine whose IP is listed in this parameter can relay its email through this Postfix instance.

Basically, what this means is that, all your docker services need to have their IPs listed in this parameter for Postfix to relay their emails further. This is mandatory. If you omit any docker service's IP from this list, that service will not be able to use Postfix as its SMTP server to relay email even if the Postfix server is technically reachable by it via ICMP echo packets.

Getting the right IP list

To find the IP address for a specific docker container, please run "docker inspect <container_id>" and then look up the "IPAddress" field from the resulting output, under the "Networks" JSON property. Note: it's not the "Gateway" field, that's something else!

Please be aware, though, that docker allocates IPs dynamically. So even if a container has a specific IP at one point, it doesn't mean that it will have the same IP next time a new container is spawned from the same image (i.e. after a system reboot). As such, this can, in theory, mean that your configuration will work at one point but, after a system reboot, it won't work anymore. This would mean that you either have to specify manual static IP addresses for your docker images so that they will always take the exact same IP all the time (not recommended and it goes against the entire philosophy of docker) or, you can just do what I did and simply whitelist all the possible private IPs under "172.16.0.0/12". This basically resolves to all the 16 continous class B private IP addresses in the IPv4 address space, as seen here. Docker will, by default, use IPs in a subrange in this address space, when allocating IPs to newly spawned containers. As I have found out recently, this may not be the case. Docker can use any private IP address that it wishes to use and, as such, it's best to not rely on this.

Instead, a better means of configuring docker to respect a specific IP address range is by restricting it from its own configuration, as the administrator.

To do so, please open the docker config file (in my case, this was under "/etc/docker/daemon.json") and edit it with your favorite text editor:

You can safely omit the "storage-driver" property, as this is unnecessary for our needs.

I really can't even remember why I needed to specify it to "fuse-overlayfs". I remember doing so fixed a particular bug that I had encountered in the past, but not what the bug was.

Alas, the only relevant configuration above is the "default-address-pools". This configuration basically tells the docker daemon that, whenever it needs to spawn a container, this container's network will have an address from that addres range specified to it (in our case, the 172.16.0.0/16 network).

Very handy

This approach has the advantage that whichever IP docker will assign to a newly created container, that IP will always fall somewhere in this range, so it will already be whitelisted. Moreover, since this is a private address range, not a public one, nobody outside the current LAN of the server can impersonate it, nor can they breach the local network from the outside if proper firewall and NAT rules are set in place by the network administrator, which means there's never a risk that someone might try misusing our Postfix server from outside our network.

Finally, there's the "inet_interfaces" configuration parameter. This one specifies under which identities the current installation of Postfix will be assumed by the server. Postfix will accept all requests destined to any of these addresses as its own and will handle them.

In a docker configuration, assuming the services are using a "bridge" network driver, they will all have their own IP addresses in the aforementioned address space, and these addresses will be distinct from the proper address of the machine where Postfix is installed. As such, they need a target to resolve to reach the machine running Postfix. This target will be IP 172.16.0.1. IMPORTANT note: if you'll use TLS enabled in Postfix, please avoid using the raw IP address as mentioned here, as certain services like Nextcloud check the domain of the SMTP server against the TLS certificates being provided and, if they mismatch, it will reject the connection. This is why I added the "mail.transistor.one" hostname in there, as my TLS certificate is against all subdomains under *.transistor.one and, as such, can be verified successfully by it. When configuring each individual docker service, enter that IP as the IP of the SMTP server to use, as well as port 25, as its connection port. These should be the only parameters you should need to configure everything to work properly. 172.16.0.1 was a random address that I decided on. Really, it has no real relevance and can be changed to any private IPv4 address, whether in class B, C or A. The only point is that it should be reachable through this network driver.

Instructing Postfix to relay emails through Google's servers, instead

Assuming that running your own postfix server is a pain in the ass (especially since it's quite difficult to get DMARC or other authenticity mechanisms configured by yourself), a proper workaround for that is to simply rely on a third party emailing service to relay your emails for you.

This has the benefit of being somewhat easier to use and configure, but with the obvious downside that you will need to have an already existing email address on that service to send the emails through.

To the recipients receiving those emails, it will look like the sender of those emails is that existing email address that you'll be using.

So, how do we achieve this?

I'm personally using my Gmail account for this. The way I configured it is as follows:

The above code was appended to the "/etc/postfix/main.cf" configuration file, at the end. It specifies a bunch of necessary configurations, but the most relevant for them is:

• The relayhost: this specifies the SMTP server itself that needs to relay the email. including the port. This is Google's own SMTP address
• smtp_sasl_password_maps: this specifies the local filesystem file that contains the login credentials that need to be used to authenticate with
  Google only allows authenticated services to relay email through their own servers, so the authentication part cannot be skipped. Also, Google requires app passwords to be used when authenticating, not the actual Gmail password of your Google account. App passwords must be manually generated by the end user and, as I've recently found out, are meant to be used only once per app. The system is designed in such a way that the password gets generated once, is meant to be copy-pasted into wherever the configuration requires it, and then be left alone. As such, you will be unable to later inspect the app password that you've previously generated, even though this password may still be valid and in use.

The contents of my "/etc/postfix/config" file looks like this:
[smtp.gmail.com]:587    <username@gmail.com>:<app_password>

Once this file has been written, please run the following utility to generate the necessary files:

Afterwards, you will be required to also edit the "/etc/postfix/master.cf" file, to enable postfix to open up its own 587 port. To do so, open this file in read-write mode and uncomment the following lines:

Once this is configured and ready to go, you can safely restart the postfix service and rely on Google to relay your emails for you.

Configuring the Postfix service such that it always restarts when the system is running low on RAM

This has been a major pain in the butt for me.

Every so often, I would visit one of my services after a long period of being away and then would request for a password to be delivered to me from them, only to then find out that no email is being received.

I would then SSH into my VPS, only to discover that Postfix had been killed at some point in the past due to low RAM.

Granted, this is by no means ideal, and it can be very annoying, having to manually restart the Postfix service every single time.

To solve this, I've rewritten my Postfix service file to account for this. Its location (for me at least) was under /etc/systemd/system/multi-user.target.wants/postfix.service

From that, the only genuinely relevant changes that need to be highlighted are the last two lines (i.e. the "Restart" and "RestartSec" assignments). These tell systemd that that, in the event that the service gets killed due to an abnormality (i.e. it receives a SIGKILL system because it is running low on RAM), to automatically restart it. The second rule (i.e. "RestartSec"), tells it to wait an entire second before performing the restart, so that it gives the system the chance to finish whatever it was doing.

Opening up port 587 for SMTP traffic

Certain services refuse to accept STARTTLS traffic on port 25, as is open, by default, on Postfix (looking at you, Gitea). To account for them, we must open port 587 to attain this. To do so, we must open the master.cf configuration file (mine was under "/etc/postfix/master.cf") and add the following line:

The smtp line was already there. I only added the 587 line. This instructs Postfix to bind itself to the 587 port, such that, any services wanting to reach that port in order to start a STARTTLS connection, wil be able to do so.

Once this is done, restart the Postfix daemon with a systemctl restart command and everything should almost be done. Almost.

Ubuntu server also comes preinstalled with a firewall utility that will deny traffic towards its own port 587. This can be an impediment. As such, please allow traffic from your docker containers to be able to reach this port:

If you recall from above, 172.16.0.0/16 was the IP range we configured for our docker engine to use when assigning IPs to its container networks. So that command will effectively allow all traffic originating from docker containers to be explicitly allowed to reach the host's own 587 port, to be able to initiate a STARTTLS encrypted channel.

While you're on it, you may also do

to delete the firewall rules that allow full access to port 25. This solved an issue where Google would spam my Gmail inbox with unnecessary garbage because it was trying to relay bounced email notifications to me, which was highly annoying to say the least.

Troubleshooting issues with Postfix reachability from docker containers

If whichever docker container you're currently running doesn't seem to connect to 172.17.0.1 or to mail.transistor.one and its image contains the ping utility pre-installed in it, you can attach your current terminal session into that container and access it via "docker exec -it <docker_container_id> /bin/bash" and then simply issuing a "ping 172.17.0.1" to send ICMP echo packets to your SMTP server from inside the container itself. If there are replies, this means the container can reach your local Postfix server so the problem is most likely from Postfix dropping the requests intentionally. Alternatively, this could be a firewall misconfiguration problem but this has never happened to me before, although I recognize that it may be theoretically possible.

To further validate this, issue the following command to see the last log error reports from Postfix, including the notifications of rejected requests:

Note, you need sudo privileges to read the mail.log file, as it is owned by the syslog user and it has restricted reading privileges.

Finally, to get further in depth into this matter, you can run:

To see if port 587 on localhost is reachable from within a docker container. If it is, this utility should be able to confirm it. Otherwise it will print an error message.

Wrapping it up

That's it! As soon as you finish editing the main configuration file, please remember to restart the Postfix service afterwards so that the changes can take effect immediately (or reboot the machine).

Happy coding!