Deriving an SES SMTP password from an IAM access key in Python
The SES SMTP password is not your IAM secret key. It is a short HMAC chain over the secret, the region and a fixed message. Here is the Python for it and the script I use to rotate the key and write it to my Ghost host without printing it.
When I moved my Ghost blog's mail to Amazon SES, I had an IAM user that was allowed to send email and nothing else. Next I needed SMTP credentials for it. The SMTP username is just the access key ID. The password is not the secret access key though. If you paste the secret key into your mail client, SES rejects the login with a 535.
The password is derived from the secret key. AWS documents the steps, and it's only a few lines of Python. In this blog we will look at the derivation, how I check that it works, and the small script I use to rotate the key and drop the result into Ghost's .env on the server without it ever showing on my screen.
Why not just click the button
The SES console has a "Create SMTP credentials" button. It makes a new IAM user for you, attaches a send policy and shows you the username and password once.
I didn't wanted that. My IAM user and its policy are in Terraform, locked down to ses:SendRawEmail from my own domain (that part is in my post on sending Ghost emails through SES with Terraform). The console button would create a second user that Terraform doesn't know about. And then I'd be copying a password out of a browser window into a terminal, which is exactly the thing I was trying to stop doing.
I also didn't want Terraform to make the access key. An aws_iam_access_key resource puts the secret into the state file. So the key gets created by a script, outside Terraform, and goes straight to the server.
The derivation
It's the same HMAC chain that SigV4 request signing uses, with fixed values in place of the date and the request.
- Start with the string
AWS4followed by the secret access key, as the HMAC key. - HMAC-SHA256 the date. For SMTP it is always the literal string
11111111. - HMAC that result with the region, like
ap-south-1. - Then with the service,
ses. - Then with the terminal string,
aws4_request. - Then with the message
SendRawEmail. - Put a single byte
0x04in front of the final 32 bytes and base64 encode the lot.
The 0x04 is a version byte. There was an older scheme with a different version byte that wasn't tied to a region. AWS deprecated that one, so just use this.
Here it is in Python, with nothing outside the standard library.
import base64
import hashlib
import hmac
def _sign(key: bytes, msg: str) -> bytes:
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def ses_smtp_password(secret_access_key: str, region: str) -> str:
sig = _sign(("AWS4" + secret_access_key).encode("utf-8"), "11111111")
for part in (region, "ses", "aws4_request", "SendRawEmail"):
sig = _sign(sig, part)
return base64.b64encode(b"\x04" + sig).decode("utf-8")
The output is always 44 characters, because 33 bytes base64 encode to exactly 44 characters with no padding. That's a quick sanity check if you ever see a password that looks off.
pw = ses_smtp_password("not-a-real-secret", "ap-south-1")
print(len(pw)) # 44
Notice the region is baked in. A password made for ap-south-1 will not log in to email-smtp.us-east-1.amazonaws.com. If you move regions, you derive again. Same secret key, different password.
Checking it against SES
The derivation is easy to get slightly wrong (a wrong order, encoding the wrong thing) and the only real test is logging in. Python's smtplib does that in a few lines. It reads the credentials from the environment so I don't type them anywhere.
import os
import smtplib
host = os.environ.get("MAIL_HOST", "email-smtp.ap-south-1.amazonaws.com")
with smtplib.SMTP(host, 587, timeout=15) as smtp:
smtp.starttls()
smtp.login(os.environ["MAIL_USER"], os.environ["MAIL_PASS"])
print("login ok")
On the Ghost host I run it with the .env loaded into the environment.
cd /opt/ghost
set -a; . ./.env; set +a
python3 smtp_check.py
If the login fails with 535 Authentication Credentials Invalid, it's usually one of three things. Wrong region in the derivation. A brand new key that IAM hasn't spread around yet, give it a minute and try again. Or the secret key got pasted in place of the derived password, which is an easy mistake at midnight.
A successful login only proves the credentials. It doesn't prove the IAM policy lets you send from your From address, that's checked when you actually send.
Rotating the key without seeing it
This is the script I actually use. It does four things:
- remembers the IAM user's current access key IDs
- creates a new key and derives the SMTP password from it
- pipes the result over SSH into
/opt/ghost/.envon the server - deletes the old keys, only after the new one is in place
#!/bin/bash
# Rotate the SES SMTP credential for an IAM user and write it into
# /opt/ghost/.env on the Ghost host. Nothing secret is printed or saved locally.
# Usage: ./ses-smtp-credentials.sh ubuntu@203.0.113.10 [extra ssh options...]
set -euo pipefail
TARGET=${1:?usage: $0 user@host [ssh opts]}; shift
REGION=${AWS_REGION:-ap-south-1}
USER_NAME=ghost-ses-smtp
OLD_KEYS=$(aws iam list-access-keys --user-name "$USER_NAME" \
--query 'AccessKeyMetadata[].AccessKeyId' --output text)
aws iam create-access-key --user-name "$USER_NAME" --output json | REGION="$REGION" python3 -c '
import sys, json, hmac, hashlib, base64, os
k = json.load(sys.stdin)["AccessKey"]
def sign(key, msg): return hmac.new(key, msg.encode(), hashlib.sha256).digest()
sig = sign(("AWS4" + k["SecretAccessKey"]).encode(), "11111111")
for part in (os.environ["REGION"], "ses", "aws4_request", "SendRawEmail"):
sig = sign(sig, part)
print("MAIL_USER=" + k["AccessKeyId"])
print("MAIL_PASS=" + base64.b64encode(b"\x04" + sig).decode())
' | ssh -o BatchMode=yes "$@" "$TARGET" "REGION=$REGION bash -c '
set -e; umask 077; cd /opt/ghost
new=\$(cat)
grep -v -E \"^(MAIL_USER|MAIL_PASS|MAIL_HOST|MAIL_PORT)=\" .env > .env.tmp || true
printf \"%s\nMAIL_HOST=email-smtp.\$REGION.amazonaws.com\nMAIL_PORT=587\n\" \"\$new\" >> .env.tmp
mv .env.tmp .env
'"
for k in $OLD_KEYS; do
aws iam delete-access-key --user-name "$USER_NAME" --access-key-id "$k"
done
echo "SES SMTP credential written to $TARGET:/opt/ghost/.env (old keys removed: ${OLD_KEYS:-none})"
The JSON from create-access-key holds the secret, it only ever exists inside the pipe. Python reads it from stdin, prints two lines, and those go straight into ssh. On the server, new=$(cat) reads them and the old mail lines get swapped out. Nothing touches my laptop's disk and nothing ends up in my terminal scrollback.
A few details that matter more than they look.
umask 077 before writing .env.tmp means the temp file is created as mode 600. Without it, there is a short window where a file with the password in it is readable by other users on the box. Small, but free to fix.
And the quoting is ugly, honestly. The remote command is inside double quotes on my side and single quotes on the server side, so every $ that should run remotely needs a backslash. One missing backslash and the variable expands on your laptop instead, usually to nothing. Test this kind of thing against a throwaway .env first.
IAM allows two access keys per user. Because the script deletes the old keys at the end, there is normally one key, the rotation makes it two, and then it goes back to one. If a run dies halfway you can end up at two, and the next run would fail at create-access-key. Then you delete the stale one by hand and run again.
The script needs AWS credentials in the environment that can create and delete access keys for that user. It never reads a credentials file itself, it just uses whatever the shell has.
Picking up the new password
Ghost reads the mail settings when it starts. Docker Compose sees that the interpolated environment changed and recreates the container.
cd /opt/ghost
docker compose up -d ghost
Then I run the smtplib check from above and send myself staff invite from Ghost admin. If that mail arrives, the rotation worked end to end.
That's it. The whole thing is maybe forty lines, and I don't have to know my own SMTP password, which is how I like it.