IMAP History Backfill Guide
This guide covers the IMAP history backfill API — a one-time import of a connected mailbox's historical emails, delivered to your webhook. You can import a whole mailbox (or a date window), target a single conversation with one participant, or import everything exchanged with a list of up to 50 contacts.
Overview
When a mailbox is connected, the live poller only forwards mail that arrives after connection. History backfill imports the historical emails that predate (or surround) that point, so a newly connected mailbox can build a complete communication timeline.
How it works:
- You start a backfill for a connected mailer. The platform walks the mailbox newest-first and delivers each email individually to the mailer's configured webhook — the same webhook that receives live incoming mail.
- Delivery is at-least-once: a slice may be retried (on transient errors, redelivery, or resume), so the same email can be delivered more than once. Your webhook must dedupe by
Message-Id. - The job runs asynchronously; poll the status endpoint for progress, or abort it at any time.
Supported connections
History backfill works on any IMAP-capable mailer that has a webhook configured. The connection you used to add the mailbox changes how the mailbox is searched and how fast it imports, but not how this API behaves.
| Connection | Added via | Mailer id prefix | Webhook event | Backfill |
|---|---|---|---|---|
| Google / Gmail OAuth | /v1/oauth connect flow | oauth_imap_, oauth_smtp_imap_ | IMAP_OAUTH | Supported |
| Microsoft / Outlook OAuth | /v1/oauth connect flow | oauth_imap_, oauth_smtp_imap_ | IMAP_OAUTH | Supported |
| Basic IMAP (username + password) | POST /v1/imapbasic/verify | basic_imap_ | BASIC_IMAP | Supported |
| Basic SMTP (send only) | POST /v1/smtpbasic/verify | basic_smtp_ | — | Not eligible |
A mailer whose config_type is smtp has no mailbox to read from, so there is nothing to import and the request returns 422. To be eligible, a basic mailer needs a config_type of imap or smtp_imap, working IMAP credentials, and a registered webhook. Adding IMAP to an existing send-only mailer means creating it through POST /v1/imapbasic/verify.
Behaviour that is identical across all supported connections: the request and response shapes, newest-first walk order, folder selection, participant scoping, at-least-once delivery, the lifecycle events, and the delivered payload shape. Per-connection differences are collected under Notes & limits.
Prerequisites
- A connected IMAP-capable mailer (
mailerID) — Gmail OAuth, Outlook OAuth, or basic IMAP. See Supported connections. - A webhook configured on that mailer (backfilled emails are delivered there). A mailer with no webhook cannot run a backfill. A participants backfill also needs that webhook to be active, or the request returns 422.
- A valid API token (the same auth used for other
/v1endpoints).
Endpoints
| Method | Path | Purpose |
|---|---|---|
POST | /v1/imap/{mailerID}/history | Start a history backfill job |
GET | /v1/imap/{mailerID}/history/{jobId} | Get job status / progress |
POST | /v1/imap/{mailerID}/history/{jobId}/abort | Cancel an in-flight job |
GET | /v1/imap/{mailerID}/messages?refId=… or ?messageId=… | Get one email by its ref-id or Message-Id |
The pre-rename paths /v1/{mailerID}/backfill[...] still work as aliases to the same handlers, so existing integrations keep functioning. New integrations should use the /v1/imap/{mailerID}/history paths.
See the API Reference → IMAP History Backfill section for the full generated schema.
Scope: full mailbox, single conversation, or participants
There is no separate "mode" flag — the fields you send decide what gets imported:
| Scope | How to select | What it imports |
|---|---|---|
| Full import | omit participant and participants (default) | Every email in the window (and/or up to count), across the selected folders. |
| Single conversation | set participant | Only emails exchanged with that one address. Use this to pull one conversation's history. |
| Participants | set participants | Every email exchanged with up to 50 contacts, across the mailbox's whole history. See Participants backfill. |
The participant match is delegated to the mailbox, so it differs slightly by connection:
| Connection | Matches on |
|---|---|
| Gmail OAuth | from: OR to: |
| Outlook OAuth | Graph $search across the message's participants |
| Basic IMAP | FROM OR TO OR CC |
Start a history backfill
POST /v1/imap/{mailerID}/history
| Field | Type | Required | Notes |
|---|---|---|---|
participant | string (email) | optional | A single bare email address, e.g. x@gmail.com. Provide it to import only the conversation with that address; omit for a full import. |
window_start | string (RFC3339) | see below | Oldest point to walk back to, e.g. 2025-01-01T00:00:00Z. |
window_end | string (RFC3339) | optional | Newest point. Defaults to now. |
count | integer | optional | Cap on number of emails (newest-first). 0/omitted = no cap; bounded only by the window. Clamped to a server maximum (default 1000). |
folders | object | optional | Include non-INBOX folders: { "inbox": true, "sent": false, "spam": false, "trash": false }. Defaults to inbox-only. |
When is window_start required?
- Full import — provide at least one of
window_startorcount. - Single conversation (
participantset) —window_startis optional. If omitted, it defaults to the last 1 year. (window_endstill defaults to now.)
Examples
Full backfill of a date window (inbox + sent):
curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"window_start": "2025-01-01T00:00:00Z",
"count": 1000,
"folders": { "sent": true }
}'
Single conversation with one participant (defaults to the last year):
curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"participant": "x@gmail.com"
}'
{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }
Idempotent per mailer: only one active full or single-conversation backfill per mailer at a time. A second request while one is in flight returns 409 with the existing jobId:
{ "success": false, "error": "active history backfill already exists", "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }
Check status
GET /v1/imap/{mailerID}/history/{jobId}
{
"success": true,
"jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA",
"status": "processing",
"processed": 250,
"target": 1000,
"percent": 25,
"startedAt": "2026-06-03T10:07:23Z",
"endedAt": ""
}
status—pending→processing→done(orfailed/cancelled). A participants job can also end asdone_with_gaps; see Participants backfill.percent— time-based progress over the requested window (clamped 0–100;100when done).processed/target— emails delivered so far / the count cap (0= uncapped).endedAt— set once the job reaches a terminal state.
The job must belong to the addressed mailerID; a job from another mailer (or another tenant) returns 404.
Abort a history backfill
POST /v1/imap/{mailerID}/history/{jobId}/abort
Cancels an in-flight job; the worker stops cleanly at its next slice, leaving the job cancelled. Already-finished jobs (done/failed/cancelled) return 409.
{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA", "status": "cancelled" }
After a job is cancelled (or finished), you can start a new backfill for the same mailer. Emails already delivered before cancellation stay delivered (dedupe by Message-Id).
Webhook delivery & dedupe
- Each historical email is delivered as a normal incoming-email webhook, with the full email content inline including attachments. The payload
eventisINCOMING_EMAILfor every connection type;IMAP_OAUTHandBASIC_IMAPare webhook registration types, not delivered event names. - The payload shape is identical to live incoming mail for the same connection, so an existing inbound handler needs no changes.
- Backfill deliveries carry an
X-History-Backfill-Job-Idheader so you can distinguish them from live mail. - Because delivery is at-least-once, dedupe by
Message-Idon your side — this is required, not optional.
Lifecycle events (optional)
Alongside the per-email deliveries, a full or single-conversation job emits small lifecycle events to the same webhook so you can track it end-to-end:
HISTORY_BACKFILL_STARTED— when the first slice begins.HISTORY_BACKFILL_PROGRESS— throttled progress updates (~every 10%).HISTORY_BACKFILL_COMPLETED— on success.HISTORY_BACKFILL_FAILED— on terminal failure.
Participants jobs do not emit lifecycle events; poll the status endpoint instead.
Participants backfill
Use this when you already know who a mailbox talks to — for example, the contacts a CRM found in the last few months of sent mail — and you want every email exchanged with them, not just a recent window.
POST /v1/imap/{mailerID}/history with participants:
| Field | Type | Required | Notes |
|---|---|---|---|
participants | array of strings (email) | yes | 1 to 50 bare email addresses. Matching is case-insensitive and repeated addresses are ignored. |
participants cannot be combined with participant, count, window_start or window_end; sending any of them returns 400. folders is ignored, because a participants job always reads every folder except spam, trash and drafts.
What gets imported:
- Who: any email where a listed contact appears in
From,To,CcorBcc. - When: the mailbox's whole history, up to the moment the job was created. There is no window.
- Where: every folder except spam, trash and drafts.
- Skipped and counted, not delivered:
- emails the platform already delivered for this mailer recently (by live inbound or an earlier backfill — roughly the last 7 days);
- emails this mailbox sent through this platform, which you already have from the send API (counted in
sentViaEdith). Emails it received from another mailer that sends through this platform are imported like any other email.
- The mailbox's own address and aliases are dropped from the list and returned as
ignoredContacts. A request that lists only those addresses returns 400.
curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"participants": ["a@acme.com", "b@acme.com", "c@globex.com"]
}'
{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }
Parallel jobs: you can run several participants jobs on the same mailer at once, for example one request per batch of 50 contacts. An email shared by two jobs is delivered once. A participants job and a full or single-conversation job cannot run on the same mailer at the same time; the second request returns 409 with the running jobId.
Participants status
GET /v1/imap/{mailerID}/history/{jobId} returns the usual fields plus per-contact counts:
{
"success": true,
"jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA",
"status": "processing",
"processed": 300,
"target": 420,
"percent": 71.4,
"startedAt": "2026-11-03T10:07:23Z",
"endedAt": "",
"provisional": true,
"totals": {
"listed": 420, "processed": 300, "delivered": 250, "duplicates": 30,
"sentViaEdith": 15, "alreadyInJob": 3, "unmatched": 1, "failed": 1, "gone": 0
},
"contacts": [
{ "contact": "a@acme.com", "found": 120, "delivered": 100, "duplicates": 15, "sentViaEdith": 5 }
],
"ignoredContacts": ["owner@acme.com"]
}
targetis the number of emails found so far andprocessedthe number handled. Contacts are searched in groups, sotargetgrows while the job runs andpercentcan move backwards.provisionalstaystrueuntil the job ends.totalscounts each email once:delivered— sent to your webhook.duplicates— already delivered recently, not sent again.sentViaEdith— sent by this mailbox through this platform, not sent again.alreadyInJob— seen twice in this job (for example in two folders), sent once.unmatched— a search hit whose headers name none of the contacts.failed— could not be downloaded or parsed.gone— deleted from the mailbox before it could be downloaded.
contactscounts per contact. An email that names three contacts counts once for each of them, so the per-contact numbers can add up to more thantotals.done_with_gapsmeans the job finished but not everything was read: some contacts or emails still failed after repeated retries, the mailbox stopped being readable (for example, the connection was revoked), or Outlook returned only part of the search results.reasonsays why in plain text, for exampleOutlook returned only part of the search results on busy days, so some emails may be missing for: a@acme.com (3 days, 2026-10-07 to 2026-10-12). All other emails were imported.It lists up to 5 contacts with the number of affected days and their date range, then "and N more contacts". Don't parse it. A participants job never ends asfailed.
Abort works the same as for other jobs: emails delivered before the abort stay delivered.
Get one email by ref-id or Message-Id
Fetch a single email of a connected mailbox, for example one a backfill counted as a duplicate or one your system knows only by its Message-Id. Give exactly one of:
| Query | Use it when | How it is found |
|---|---|---|
refId | You have the ref-id from an INCOMING_EMAIL webhook | Read from the copy this platform stored. If the stored copy has expired, the call returns 404; look it up by messageId instead. |
messageId | You know the email's Message-Id (with or without <>) | The stored copy is returned if there is one; otherwise the email is fetched from the mailbox, stored the same way as live inbound, and returned. |
curl "https://<your-host>/v1/imap/{mailerID}/messages?messageId=%3Cabc%40mail.acme.com%3E" \
-H "Authorization: Bearer <API_KEY>"
{
"success": true,
"refId": "01M6G4QWRZ1YJACS8TVPNE5AYJ",
"source": "stored",
"email": { "event": "INCOMING_EMAIL", "details": { "...": "same fields as the webhook" }, "success": true }
}
emailhas exactly the shape of theINCOMING_EMAILwebhook body, so you can reuse your webhook parser.sourceisstoredwhen this platform already had the email,mailboxwhen it was fetched from the mailbox for this call.- No webhook is sent for an email returned here. Once fetched, a later backfill counts it as a duplicate.
- URL-encode the
Message-Id(<→%3C,>→%3E,@→%40). ref-ids from full or single-conversation backfills are the provider's own message ids, not stored copies, sorefIdreturns404for them; usemessageId.ref-ids from live inbound and participants backfills work withrefId.
| Status | Meaning |
|---|---|
400 | Neither or both of refId and messageId given |
404 | Mailer not found, email not in the mailbox, or the refId copy is no longer stored |
409 | Mailer is inactive |
422 | Provider not supported, or the mailbox can't be read (for example the connection was revoked) |
429 | The mail provider is throttling this mailbox; retry after the Retry-After header |
Notes & limits
All connections
- Send-only mailers return 422. See Supported connections.
- Window precision is rounded to whole seconds. Every mailbox's date filter is day-granular, so the platform widens the query and then filters on each message's real timestamp.
- Throughput is bounded by bytes, not message count. A mailbox of 200 large attachments takes far longer than one of 2,000 short replies, because every message is transferred from the mailbox and then posted to your webhook in full. Use the status endpoint rather than estimating from message counts.
- Your webhook is part of the critical path. A slow or failing endpoint slows or drops deliveries, so keep it fast and return 2xx quickly.
Gmail OAuth
- Throughput is additionally capped by Gmail's per-user API quota, so a large mailbox is paced rather than pulled at full speed.
Outlook OAuth
- Microsoft returns at most about 275 search results per folder. Single-conversation and participants searches split the date range in half whenever they reach that limit, so large histories are still covered. A single day can't be split further: if one contact has more matching emails than the limit on one day in one folder, the extra emails are skipped and the platform logs a warning.
- Participants search runs folder by folder. When a day is still at the limit for a group of contacts, it searches each contact on their own for that day. If one contact is still over the limit on a single day in one folder, the extra emails for that day are skipped and the job ends as
done_with_gaps, with areasonnaming the contact and the affected days.
Basic IMAP
- Port 993 with SSL only. Other port and security combinations are rejected when the mailer is created.
- One connection per job. Messages are fetched sequentially over a single IMAP session, so basic IMAP is slower per byte than the OAuth connections, which fetch in parallel.
- Shared session limits. Many hosted IMAP servers cap concurrent sessions per account. If the live poller holds the only available session, the backfill cannot connect and the job retries. Running a backfill during quiet hours avoids this.
- Participants search uses the server's all-mail folder when it has one, otherwise it searches each folder except spam, trash and drafts. On servers whose all-mail folder also holds drafts (Gmail over IMAP, for example), drafts addressed to a contact can be included.
- Folder names are mapped, not assumed. The platform discovers the mailbox's real folders and maps them onto
INBOX,SENT,SPAMandTRASH, so server-specific names such as[Gmail]/Sent Mailresolve correctly.