paper-thin-bits

Tech articles, mini tech blog

Email with SMTP Relay and SparkPost

XKCD email

Contents

Preface

So you just got a fresh Linux cloud server instance, you setup a web server, a database, some of your favourite services, but you’ll need a way for the server to send email. Aah! Do you now setup Postfix, qmail or maybe Exim? And how do you handle security and authentication?

Setting up a mail server, also MTA [1], is a great experience if you want to learn a ton of new things. Doing that for your weekend project though? Maybe, not that good of an idea. So what if you could shift that responsibility to some third-party service?

There are many email services available [2] and most offer free plans that allow for sending tens of thousands of emails per month without charges. This tutorial uses a way to setup an email relay using SparkPost. The integration with SparkPost’s API is quite easy and they also offer up to 100,000 emails per month for free.

Goal

We would like to have emails sent by our Linux server delivered via SparkPost to recipients. A simple communication diagram would look like this:

Communication Diagram

What we need to do first is to configure a SparkPost account and then setup the services that we use, i.e., PHP based services or services that connect directly to an SMTP server, to use SparkPost as an SMTP server.

Setup SparkPost

Go to SparkPost and create a user account. I recommend setting up a Two-Factor authentication [3]. It is a great way to get a better control of your account. After logging in, you’ll be greeted by a nice looking dashboard with basic statistics.

SparkPost Dashboard

Sending Domain

You first need to setup your a sending domain. Add the domain name that you will be assigning to your cloud server instance, e.g., mydomain.com.

SparkPost will reject emails that are not coming from a configured Sending Domain!

SparkPost Sending Domain

Use the Settings option to configure domain verification. SparkPost needs to know that you are the owner of mydomain.com, therefore you need to create a new DNS TXT record and add the DomainKeys Identified Mail [4] public key info to that record. This enables the SMTP server of the recipient to verify that the emails it gets are indeed coming from your domain, i.e., a defense against email spoofing attacks.

SparkPost Sending Domain DKIM

API Key

An API key is required in order for client services to connect and relay emails via SparkPost. Go to SMTP Relay and use Manage API Keys to configure a new API key.

SparkPost API

In general the only permission required is Send via SMTP. If you need to setup a service that uses the SparkPost HTTP API, then you need to also add the Transmission Read/Write permission.

Add your server IP address to the Allowed IPs, if your server has a static IP and you need extra security.

SparkKpost API Key

Next, add and copy the generated API key. Be advised that SparkPost will only show the newly generated API key once! Copy it to a safe location until you have your services setup.

SparkKpost API Key

Setup SMTP Clients

In this scenario we need a simple mail delivery software tool to deliver emails from our system, i.e., various mail user agents or MUA [5], to the SparkPost SMTP server. Two well known tools for this job are sSMTP [6] [7] and msmtp [9].

  • sSMTP is quite stable, but it does not seem to be maintened anymore.
  • Msmtp does seem to be maintained with the latest version released on Nov 14, 2016.

It probably makes sense to first try Msmtp and if that’s is not available then fall back to using sSMTP instead.

Msmtp

Msmtp has a great documentation [10]. Make sure you check it out, if you need a more complex setup like proxies or even GPG encryption.

Install Msmtp using the system package manager, e.g., on Debian:

apt-get install msmtp

Open /etc/msmtprc and configure the following:

# Defaults

defaults
auth           on
tls            on
tls_certcheck  on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile        ~/.msmtp.log
# Syslog logging with facility LOG_MAIL instead of the default LOG_USER.
syslog LOG_MAIL

# A system wide configuration file is optional.
# If it exists, it usually defines a default account.
# This allows msmtp to be used like /usr/sbin/sendmail.
account sparkpost
host smtp.sparkpostmail.com
port 2525
user SMTP_Injection
password 831fe901f859cdea3c5d278e6f8ecce343070115

auto_from on
#from sender@mydomain.com

# Default account
account default : sparkpost

# Aliases. (Optional)
#aliases /etc/msmtprc_aliases
  • We used the SparkPost generated API key as password. The user is always SMTP_Injection.
  • If you do not want to verify the remote server certificate, therefore the SparkPost SMTP server certificate, then comment out the tls_trust_file option and set tls_certcheck to off.
  • You can override the From field for all emails sent by specifying a hard coded sender email address. Comment out the from option above to do so. You may want to do that should there be services that do not send emails on behalf of your domain, e.g., sender@mydomain.com. Remember that SparkPost will reject emails that are not coming from a configured Sending Domain!
  • If you would like to use sender aliases, enable the aliases option.

Aliases

Your are not required to setup aliases, but it is recommended in order to have local addresses properly mapped.

Open /etc/msmtprc_aliases and configure the following:

# Replace local recipients with addresses in the aliases file.

# Send root to own address
root: my-address@example.org

# Matched if the local address is not found in the aliases file. 
default: my-address@example.org

Testing

Create a new testmail.txt file with the following sample text.

To: customer@example.org
From: no-reply@mydomain.com
Subject: This is a Test Mail

Hello there from mydomain.com!

--

Note that the From field must contain a known SparkPost Sending Domain.

Run the following to send a test email to customer@example.org. You should of course use another valid email address.

cat testmail.txt | msmtp customer@example.org

Logs are found in the the configured /root/.msmtp.log log file:

Apr 14 11:21:02 host=smtp.sparkpostmail.com tls=on auth=on user=SMTP_Injection from=no-reply@mydomain.com recipients=customer@example.org mailsize=190 smtpstatus=250 smtpmsg='250 2.0.0 OK 7C/D1-412B9-8E2C01A2' exitcode=EX_OK

sSMTP

Install sSMTP using the system package manager, e.g., on Debian:

apt-get install ssmtp

Open /etc/ssmtp/ssmtp.conf and configure the following:

#
# Config file for sSMTP sendmail
#
# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.
root=my-address@example.org

# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
#mailhub=mail

# The full hostname
hostname=mydomain

# Are users allowed to set their own From: address?
# YES - Allow the user to specify their own From: address
# NO - Use the system generated From: address
FromLineOverride=YES

# SparkPost
mailhub=smtp.sparkpostmail.com:2525
AuthUser=SMTP_Injection
AuthPass=831fe901f859cdea3c5d278e6f8ecce343070115
UseSTARTTLS=YES
AuthMethod=AUTH LOGIN

Reverse Aliases

Reverse aliases are a nice way to make sure emails intended for system users like root always go to a designated address. In the example below, all emails sent to the root user will be rerouted to my-address@example.org.

Open /etc/ssmtp/revaliases and configure the following:

# sSMTP aliases
#
# Format:       local_account:outgoing_address:mailhub
#
# Example: root:your_login@your.domain:mailhub.your.domain[:port]
# where [:port] is an optional port number that defaults to 25.
root:my-address@example.org:smtp.sparkpostmail.com:2525:2525

Testing

Prepare a test mail file just like with Msmtp and run the following to send an email to customer@example.org.

 cat testmail.txt | ssmtp customer@example.org

Logs are found the syslog file, i.e., /var/log/syslog:

Apr 14 14:56:30 vxs01 sSMTP[10249]: Creating SSL connection to host
Apr 14 14:56:30 vxs01 sSMTP[10249]: SSL connection using RSA_AES_128_CBC_SHA1
Apr 14 14:56:33 vxs01 sSMTP[10249]: Sent mail for no-reply@mydomain.com (221 2.3.0 c.server.prd.sparkpost closing connection) uid=0 username=root outbytes=123

Examples

Now that at least one MTA is configured let’s look at some examples using the MTA or the SparkPost API.

PHP

Configuring PHP is straightforward. Open the php.ini file and add the following to the sendmail_path option.

[mail function]
; For Unix only.  You may supply arguments as well (default: "sendmail -t -i").
; http://php.net/sendmail-path

; Use the following for Msmtp
sendmail_path = /usr/bin/msmtp -C /etc/msmtprc -t

; Or use the following for sSMTP
sendmail_path = /usr/sbin/ssmtp -t

Testing

You can create a simple script to test the mail [11] function.

<?php
$to      = 'customer@example.org';
$subject = 'the subject';
$message = 'hello';
$headers = 'From: no-reply@mydomain.com' . "\r\n" .
    'X-Mailer: PHP/' . phpversion();

mail($to, $subject, $message, $headers);
?>
$ php script.php

Gogs

Gogs is an awesome self-hosting Git service. To setup SparkPost to relay Gogs emails open the <gogs-installation>/custom/conf/app.ini file and configure the following:

[mailer]
ENABLED = true
HOST = smtp.sparkpostmail.com:587
FROM = gogs@mydomain.com
USER = SMTP_Injection
PASSWD = 831fe901f859cdea3c5d278e6f8ecce343070115

This simple setup does not use an MTA. Instead it talks to the SparkPost SMTP server directly.

Java API

SparkPost offers a way for developers to use the HTTP API [12] to send emails. In the following example we use Java with the java-sparkpost library.

import com.sparkpost.Client;
import com.sparkpost.exception.SparkPostException;

...

List<String> emails = ...;

Client client = new Client("831fe901f859cdea3c5d278e6f8ecce343070115");

try {
  client.sendMessage("no-reply@mydomain.com", emails,
          "Test Mail", "Hello!", null);
} catch (SparkPostException e) {
  e.printStackTrace();
}

It should be noted again that the code above would only work with API keys for which the permission Transmission Read/Write is enabled.

References

  1. Mail Transfer Agent (MTA) - en.wikipedia.org/wiki/Message_transfer_agent
  2. Mandrill HN discussion - news.ycombinator.com/item?id=11170713
  3. SparkPost Two-Factor authentication - support.sparkpost.com/customer/portal/articles/1948449
  4. DomainKeys Identified Mail (DKIM) - en.wikipedia.org/wiki/DomainKeys_Identified_Mail
  5. Mail User Agent (MUA) - en.wikipedia.org/wiki/Mail_user_agent
  6. sSMTP Debian Wiki - wiki.debian.org/sSMTP
  7. sSMTP Archlinux Wiki - wiki.archlinux.org/index.php/SSMTP
  8. sSMTP Quick-N-Dirty Guide - www.scottro.net/qnd/qnd-ssmtp.html
  9. Msmtp - msmtp.sourceforge.net
  10. Msmtp documentation - msmtp.sourceforge.net/doc/msmtp.html
  11. PHP mail - php.net/manual/en/function.mail.php
  12. SparkPost API Libraries - github.com/SparkPost