rmail

Outbound Relay Internals

Direct MX delivery

When relay.smarthost is empty, rmail resolves MX records for the recipient domain using a raw DNS UDP implementation in relay/mx.py. It reads the nameserver from /etc/resolv.conf and falls back to 8.8.8.8. Responses are cached by TTL (minimum 60 seconds). If no MX records are found, the domain itself is used as the mail host per RFC 5321 §5.1.

Opportunistic STARTTLS

The SMTP client in relay/client.py advertises its hostname, checks EHLO for STARTTLS, and upgrades if available. If the remote server does not support STARTTLS, delivery continues over plaintext. There is no option to require TLS for outbound connections in the current implementation.

Retry schedule

Failed messages are kept in an in-memory list with scheduled retry times. The scheduler wakes every 30 seconds and re-enqueues messages that are due. Retry delays between attempts:

DSN bounces

After all retries are exhausted, or on a permanent failure (5xx response code), a Delivery Status Notification is written directly to the original sender's INBOX via the backend. The bounce includes the failure reason, status code, and original message headers. Bounce-of-bounce suppression prevents loops: messages with empty senders or mailer-daemon@ as sender are never bounced. Bounces are only generated for local senders (domain in config.domains).

Smarthost configuration

To route all outbound mail through a relay host (e.g. when port 25 is blocked by the hosting provider):

{
  "relay": {
    "enabled": true,
    "smarthost": "smtp.sendgrid.net",
    "smarthost_port": 587,
    "smarthost_username": "apikey",
    "smarthost_password": "your-api-key"
  }
}

In smarthost mode, MX resolution is skipped entirely. All messages are delivered to the configured host with AUTH PLAIN credentials.

TLS Management

Auto-generated certificates

On first start with tls.cert_file: null, rmail calls openssl req -x509 to generate a 2048-bit RSA self-signed certificate valid for 3650 days. The certificate is written to {mail_root}/tls/cert.pem and the key to {mail_root}/tls/key.pem. Subsequent starts reuse the existing files.

Certificate renewal

To replace a certificate without a full restart, update the files on disk and send SIGHUP:

kill -HUP $(cat /var/run/rmail.pid)

Configuration reload re-reads tls.cert_file and tls.key_file. New connections use the new certificate; existing connections continue until closed.

Security behaviour

When TLS is configured, AUTH is omitted from EHLO responses and AUTH=PLAIN / AUTH=LOGIN from IMAP CAPABILITY on cleartext connections. Clients must issue STARTTLS and repeat EHLO/CAPABILITY before credentials are accepted. This prevents credential transmission over unencrypted channels.

Quota Enforcement

Each user has a quota_mb field in user.json (default 1024 MiB). Quotas are enforced on all append paths:

• SMTP delivery — messages to recipients over quota are logged as delivery failures and silently dropped (the sending server receives a 250 OK; rejection happens post-DATA)
• IMAP APPEND — returns NO [ALERT] Quota exceeded
• Exchange API send — over-quota recipients appear in the failed array
• Exchange API append — returns HTTP 413

To update a user's quota, edit quota_mb in {mail_root}/domains/{domain}/users/{username}/user.json.

Scaling

The shard-based storage model supports multi-process deployments sharing a single mailstore on a network filesystem:

• JWT tokens are validated by any process sharing the same exchange_api.token_secret
• All flag changes use os.rename(), which is POSIX-atomic
• UID allocation uses fcntl.flock() with a sub-millisecond hold time
• Directory listing is os.scandir() — eventually consistent, no shared cache

Storage Migration

Earlier versions used per-mailbox index.json files with flat message directories. To migrate to the current shard-based format:

python run.py migrate

This converts each mailbox from index.json + msg-{uid}.eml to status.json + shard directories with flag-encoded filenames. The original index.json is renamed to index.json.migrated and preserved.

Diagnostics

The 8-phase diagnostic suite creates a temporary test user, exercises all server components through real protocol connections, and cleans up afterward:

sudo python run.py diagnose

Logging

rmail uses Python's standard logging module. Logger names follow the module hierarchy:

• rmail.smtp — SMTP connections, auth, delivery
• rmail.imap — IMAP connections and commands
• rmail.delivery — local delivery queue
• rmail.relay.queue — relay queue, retries, bounces
• rmail.relay.mx — DNS MX resolution
• rmail.relay.client — outbound SMTP client
• rmail.api — Exchange HTTP API
• rmail.config — configuration loading

Set log_level to DEBUG in config.json for verbose protocol-level output. This includes individual SMTP commands, IMAP exchanges, MX query results, and relay attempt details.

Custom Storage Backend

MailStoreBackend in rmail/backend/base.py is an abstract base class defining the full storage interface. All protocol code interacts only with this interface. To implement an alternative backend (e.g. database-backed or S3-backed), subclass MailStoreBackend and implement all abstract methods, then pass an instance to SMTPServer, IMAPServer, and ExchangeAPI in main.py.