SOGo 6 Mail Outgoing

Here is explained how SOGo 6 handles mail sending.

Draft and mail writing: Preamble

Even if sending a mail could be as easy as a single endpoint, SOGo 6 has mechanisms to support mail writing by user. It means that no user will just "send" a mail. A user will open a composer, take their time to write the mail, correct it, add attachments that could be heavy…​

SOGo 6 has several endpoints to both support mail being written and sent by a human and scripts (or robots).

From the point of view of a script, sending a mail should be straightforward: uploading the attachments and sending the content.

The send endpoint is respectful of our REST API rules. Technically, it is possible to make only one endpoint to send a mail with attachments. However, an attachment can be heavy, thus SOGo 6 has an endpoint to upload the attachment first, with the proper HTTP method for it, before sending.

But, from the point of view of a human, sending a mail takes time: The user writes their mail, modifies it, wants it to be saved, will add an attachment, will check the attachment, maybe remove it and so on. And they do not want to lose anything in progress. That’s why SOGo 6 has ways to make this action as smooth as possible.

Just saving the mail in the Draft folder is not enough.

Draft vs mail in progress

When Draft is mentioned, it represents the actual mail file being stored in the mail server. The "mail in progress" represents the mail composer where it is currently written. To make the link between the two of them, SOGo 6 uses a database table to store info until the mail is sent, deleted or is paused by the user.

The paused state means that the user has ended their "mail editing" session. They will retrieve their mail in the Draft folder from that point on.

With SOGo 6 there is a database table that manages the "mail in progress" state, making the link between what the user is doing and the current draft stored on the mail server. To be sure that table is kept clean, any user or robot has a limited number of "mail in progress" that they can have at once. The limit is set to avoid scripts that would purposely create "mail in progress" entries and overload the database. The limit does not affect real human usage, as users will naturally write and then send or delete their emails in progress.

tmp_draft

tmp_draft is the database table that represents "mail in progress". It contains the following columns:

Column Type Description

key

string (unique)

A unique hex hash that identifies this tmp_draft entry. Used as the key in all related API endpoints.

owner

string (indexed)

The UID of the user who owns this entry. Every operation checks this against the authenticated user.

mail_server_uid

string

The IMAP UID of the draft stored in the mail server’s Drafts folder. Empty string when no IMAP draft has been saved yet (e.g. a brand-new locked row before the first save).

lock_state

boolean

True means a request is currently modifying the draft. Other requests must wait or return 409.

headers

dict (nullable)

Optional RFC 5322 headers to inject at send time. Used by the reply and edit flows to carry threading headers such as In-Reply-To and References.

last_updated

integer (nullable)

Unix timestamp of the last modification of this row.

The table uses a composite primary key on id and key, and has an index on owner because the WHERE clause will be either on key (to get a specific tmp_draft) or on owner (to list all tmp_drafts for a user).

Lock mechanism

The lock ensures that two concurrent requests (e.g. a save and an upload) do not corrupt the IMAP draft simultaneously.

The TmpDraftManager class (app/module/mail/model/TmpDraftManager.py) centralises all tmp_draft database operations and exposes a locked() context manager that automatically unlocks the row on error.

  • Immediate fail (save, delete): If the row is locked, return 409 immediately.

  • Wait-and-retry (upload): Poll every 100 ms for up to 2 s. If still locked after the timeout, return 409.

Threading headers

The headers column stores a JSON dict of RFC 5322 headers that must be injected when the mail is finally sent. This is populated by the reply and edit workflows:

  • reply stores {"In-Reply-To": "<original-message-id>", "References": "…​"}.

  • edit does not need threading headers (it copies the original mail as-is).

At send time (/send), the interface layer reads these headers from the tmp_draft row and passes them to ModuleMailOutgoing.send_mail() as extra_headers. They are injected only if the header is not already present in the message, and they can never overwrite From, To, Subject, Message-ID or Date.

Mail send API endpoints

The base URL prefix for all mail compose/send endpoints is:

/mailboxes/<account_id>/mail

account_id is "0" for the user’s main account, or the hash identifier of an external account.

The key (when present) is always part of the URL path, not a query parameter.

Endpoint overview

Method & URL HTTP method Description

POST /mailboxes/<account_id>/mail/send

POST

Send a mail immediately (no existing tmp_draft).

POST /mailboxes/<account_id>/mail/<key>/send

POST

Send a mail linked to an existing tmp_draft. Attachments stored in the IMAP draft and threading headers are injected automatically before sending.

POST /mailboxes/<account_id>/mail/save

POST

Create a new tmp_draft and save the mail as a draft in the Drafts folder.

PUT /mailboxes/<account_id>/mail/<key>/save

PUT

Update an existing draft. Accepts an optional ?close=true query parameter to delete the tmp_draft row after saving (the IMAP draft is kept).

POST /mailboxes/<account_id>/mail/attachments

POST

Upload an attachment, creating a new tmp_draft entry.

POST /mailboxes/<account_id>/mail/<key>/attachments

POST

Upload an attachment to an existing tmp_draft.

GET /mailboxes/<account_id>/mail/<key>/attachments/<filename>

GET

Download an attachment from an existing tmp_draft.

DELETE /mailboxes/<account_id>/mail/<key>/attachments/<filename>

DELETE

Delete an attachment from an existing tmp_draft.

DELETE /mailboxes/<account_id>/mail/<key>

DELETE

Delete the IMAP draft and the tmp_draft row (the mail is permanently lost).

GET /mailboxes/<account_id>/mail/current

GET

List all tmp_draft entries currently owned by the authenticated user.

Key in the path

When a <key> segment is present in the URL, the request targets an already-existing "mail in progress". If no <key> is given (e.g. POST /save, POST /attachments, POST /send), SOGo 6 creates a new tmp_draft (after checking the per-user limit; 429 if the limit is reached).

The key is validated at interface level:

  • 404 if the key does not exist.

  • 401 if the authenticated user is not the owner of the tmp_draft.

  • 409 if the row is locked (behaviour depends on the endpoint, see below).

POST /mailboxes/<account_id>/mail/send

Send a mail without any pre-existing tmp_draft.

  1. Build the EmailMessage from the request body.

  2. Call ModuleMailOutgoing.send_mail().

  3. Save a copy to the Sent folder via IMAP APPEND.

No tmp_draft is created or consumed.

POST /mailboxes/<account_id>/mail/<key>/send

Send a mail linked to an existing tmp_draft.

  1. Validate the key (404 / 401 / 409).

  2. Retrieve threading headers from the tmp_draft row (e.g. In-Reply-To, References).

  3. Retrieve attachments stored in the IMAP draft and merge them with any attachments sent in the request body.

  4. Call ModuleMailOutgoing.send_mail() with the merged attachments and threading headers.

  5. Save a copy to the Sent folder.

  6. Delete the tmp_draft row (and the associated IMAP draft) on success.

POST /mailboxes/<account_id>/mail/save

Create a new tmp_draft and save the mail body as a draft in the Drafts folder.

  1. Check the per-user limit (429 if reached).

  2. Insert a new locked tmp_draft row with an empty mail_server_uid.

  3. Build the EmailMessage from the request body and call IMAP APPEND (save_draft).

  4. Update the tmp_draft row with the new IMAP UID and unlock it.

  5. Return the tmp_draft key and the parsed draft data.

If the row is locked when the request arrives, return 409 immediately (no wait).

PUT /mailboxes/<account_id>/mail/<key>/save

Update an existing draft identified by <key>.

The existing IMAP draft is fetched first so that any attachments already uploaded are preserved. Only the text body and the RFC 5322 headers (From, To, Subject, Cc, Bcc, …) are updated.

The optional query parameter close=true can be passed to delete the tmp_draft row after saving. The IMAP draft in the Drafts folder is kept, and the user can resume from the Draft folder later.

If the row is locked when the request arrives, return 409 immediately.

POST /mailboxes/<account_id>/mail/attachments

Upload an attachment without an existing tmp_draft.

  1. Check the per-user limit (429 if reached).

  2. Insert a new locked tmp_draft row.

  3. Create a new EmailMessage, attach the uploaded file, and call IMAP APPEND.

  4. Update the tmp_draft row and unlock it.

  5. Return the key and the resolved filename.

The file must be sent as multipart/form-data with a field named file. If a file with the same name already exists in the draft, the filename is automatically suffixed with (n) before the extension to avoid collisions.

POST /mailboxes/<account_id>/mail/<key>/attachments

Upload an attachment to an existing draft.

Behaviour is the same as the create variant above except:

  • The existing IMAP draft is fetched first to preserve already-uploaded attachments and the current mail body.

  • If the row is locked, the request waits up to 2 s (polling every 100 ms) before returning 409. This allows concurrent save + upload operations to resolve gracefully.

GET /mailboxes/<account_id>/mail/<key>/attachments/<filename>

Download (or preview) an attachment from the draft identified by <key>.

  1. Validate the key (404 / 401).

  2. Fetch the raw EML from the Drafts folder.

  3. Walk the MIME tree and return the first part whose filename matches <filename>.

  4. Return 200 with the file content and the correct Content-Type, or 404 if the attachment is not found.

The file is returned as an attachment (Content-Disposition: attachment).

DELETE /mailboxes/<account_id>/mail/<key>/attachments/<filename>

Delete an attachment from the draft identified by <key>.

  1. Validate the key (404 / 401 / 409 if locked).

  2. Fetch the raw EML from the Drafts folder.

  3. Remove the first MIME part whose filename matches <filename> from the MIME tree (in-place, preserving all other parts).

  4. Save the modified message back to Drafts (IMAP REPLACE).

  5. Update the tmp_draft row with the new IMAP UID and unlock it.

  6. Return 204 on success, 404 if the attachment is not found.

DELETE /mailboxes/<account_id>/mail/<key>

Delete the IMAP draft and the tmp_draft row. The mail is permanently lost.

  1. Validate the key (404 / 401).

  2. If the row is locked, return 409 immediately.

  3. Permanently delete the IMAP draft from the Drafts folder (if a mail_server_uid is present).

  4. Delete the tmp_draft row.

  5. Return 204.

GET /mailboxes/<account_id>/mail/current

Return all tmp_draft entries owned by the authenticated user.

Returns a list of objects, each with:

  • key: the tmp_draft key

  • mail_server_uid: the IMAP UID of the linked draft (may be empty for brand-new rows)

  • locked: boolean lock state

  • last_updated: Unix timestamp of the last modification

Mail compose entry points (edit & reply)

These two endpoints live under the mail folder API (/mailboxes/<account_id>/folders/<folder_name>/mails) and serve as entry points into the compose workflow. They create a tmp_draft entry so the user can continue using the standard compose endpoints afterward.

GET /mailboxes/<account_id>/folders/<folder_name>/mails/<mail_uid>/edit

Open an existing mail for editing.

  1. Check the per-user tmp_draft limit (429 if reached).

  2. Fetch the raw EML of <mail_uid> from <folder_name>.

  3. IMAP APPEND the raw EML to the Drafts folder (creating a new draft copy).

  4. Insert a new tmp_draft row linking to the new IMAP draft UID.

  5. Permanently delete the original mail from <folder_name> (best-effort; failure is logged but does not abort the operation).

  6. Return the parsed draft data augmented with the key field.

The client should use the returned key to continue editing via the compose endpoints (PUT /<key>/save, POST /<key>/attachments, etc.).

GET /mailboxes/<account_id>/folders/<folder_name>/mails/<mail_uid>/reply

Prepare a reply draft for the mail identified by <mail_uid>.

  1. Check the per-user tmp_draft limit (429 if reached).

  2. Fetch the raw EML of <mail_uid> and extract Message-ID and References headers.

  3. Compute threading headers according to RFC 5322:

    • In-Reply-To = the original mail’s Message-ID.

    • References = the original References + the original Message-ID (or just the Message-ID if there were no prior references).

  4. Create a new empty draft in the Drafts folder via IMAP APPEND.

  5. Insert a new tmp_draft row with mail_server_uid pointing to the empty draft and headers set to the computed threading headers.

  6. Return the parsed empty-draft data augmented with key and headers fields.

When the user later calls POST /<key>/send, the threading headers stored in the tmp_draft row are automatically injected into the outgoing message.

Flow examples

Sending a mail without attachment (script / robot)

POST /mailboxes/<account_id>/mail/send

Sending a mail with an attachment (script / robot)

POST /mailboxes/<account_id>/mail/attachments

Returns the key for the new tmp_draft.

POST /mailboxes/<account_id>/mail/<key>/send

A user writes a mail and pauses to finish it later

POST /mailboxes/<account_id>/mail/save

Returns the key. The user adds an attachment:

POST /mailboxes/<account_id>/mail/<key>/attachments

The user resumes writing and saves again:

PUT /mailboxes/<account_id>/mail/<key>/save

The user decides to continue later and closes the composer. The tmp_draft row is deleted but the IMAP draft is kept:

PUT /mailboxes/<account_id>/mail/<key>/save?close=true

A user writes an email and decides to cancel it

POST /mailboxes/<account_id>/mail/save

Returns the key. The user cancels — the IMAP draft and the tmp_draft row are both deleted:

DELETE /mailboxes/<account_id>/mail/<key>

A user replies to a mail

GET /mailboxes/<account_id>/folders/INBOX/mails/<mail_uid>/reply

Returns the key and pre-filled threading headers. The user writes the reply body:

PUT /mailboxes/<account_id>/mail/<key>/save

The user sends the reply. Threading headers are injected automatically:

POST /mailboxes/<account_id>/mail/<key>/send

A user edits a sent or draft mail

GET /mailboxes/<account_id>/folders/Sent/mails/<mail_uid>/edit

Returns the key. The original mail is moved to Drafts. The user edits and sends:

PUT /mailboxes/<account_id>/mail/<key>/save
POST /mailboxes/<account_id>/mail/<key>/send