Skip to main content

IMAP History Backfill Guide

This guide covers the IMAP history backfill API — a one-time import of a connected mailbox's historical emails, delivered to your webhook. You can import a whole mailbox (or a date window), target a single conversation with one participant, or import everything exchanged with a list of up to 50 contacts.

Overview​

When a mailbox is connected, the live poller only forwards mail that arrives after connection. History backfill imports the historical emails that predate (or surround) that point, so a newly connected mailbox can build a complete communication timeline.

How it works:

  • You start a backfill for a connected mailer. The platform walks the mailbox newest-first and delivers each email individually to the mailer's configured webhook — the same webhook that receives live incoming mail.
  • Delivery is at-least-once: a slice may be retried (on transient errors, redelivery, or resume), so the same email can be delivered more than once. Your webhook must dedupe by Message-Id.
  • The job runs asynchronously; poll the status endpoint for progress, or abort it at any time.

Supported connections​

History backfill works on any IMAP-capable mailer that has a webhook configured. The connection you used to add the mailbox changes how the mailbox is searched and how fast it imports, but not how this API behaves.

ConnectionAdded viaMailer id prefixWebhook eventBackfill
Google / Gmail OAuth/v1/oauth connect flowoauth_imap_, oauth_smtp_imap_IMAP_OAUTHSupported
Microsoft / Outlook OAuth/v1/oauth connect flowoauth_imap_, oauth_smtp_imap_IMAP_OAUTHSupported
Basic IMAP (username + password)POST /v1/imapbasic/verifybasic_imap_BASIC_IMAPSupported
Basic SMTP (send only)POST /v1/smtpbasic/verifybasic_smtp_—Not eligible
Send-only mailers are not eligible

A mailer whose config_type is smtp has no mailbox to read from, so there is nothing to import and the request returns 422. To be eligible, a basic mailer needs a config_type of imap or smtp_imap, working IMAP credentials, and a registered webhook. Adding IMAP to an existing send-only mailer means creating it through POST /v1/imapbasic/verify.

Behaviour that is identical across all supported connections: the request and response shapes, newest-first walk order, folder selection, participant scoping, at-least-once delivery, the lifecycle events, and the delivered payload shape. Per-connection differences are collected under Notes & limits.


Prerequisites​

  1. A connected IMAP-capable mailer (mailerID) — Gmail OAuth, Outlook OAuth, or basic IMAP. See Supported connections.
  2. A webhook configured on that mailer (backfilled emails are delivered there). A mailer with no webhook cannot run a backfill. A participants backfill also needs that webhook to be active, or the request returns 422.
  3. A valid API token (the same auth used for other /v1 endpoints).

Endpoints​

MethodPathPurpose
POST/v1/imap/{mailerID}/historyStart a history backfill job
GET/v1/imap/{mailerID}/history/{jobId}Get job status / progress
POST/v1/imap/{mailerID}/history/{jobId}/abortCancel an in-flight job
GET/v1/imap/{mailerID}/messages?refId=… or ?messageId=…Get one email by its ref-id or Message-Id
Legacy paths

The pre-rename paths /v1/{mailerID}/backfill[...] still work as aliases to the same handlers, so existing integrations keep functioning. New integrations should use the /v1/imap/{mailerID}/history paths.

See the API Reference → IMAP History Backfill section for the full generated schema.


Scope: full mailbox, single conversation, or participants​

There is no separate "mode" flag — the fields you send decide what gets imported:

ScopeHow to selectWhat it imports
Full importomit participant and participants (default)Every email in the window (and/or up to count), across the selected folders.
Single conversationset participantOnly emails exchanged with that one address. Use this to pull one conversation's history.
Participantsset participantsEvery email exchanged with up to 50 contacts, across the mailbox's whole history. See Participants backfill.

The participant match is delegated to the mailbox, so it differs slightly by connection:

ConnectionMatches on
Gmail OAuthfrom: OR to:
Outlook OAuthGraph $search across the message's participants
Basic IMAPFROM OR TO OR CC

Start a history backfill​

POST /v1/imap/{mailerID}/history

FieldTypeRequiredNotes
participantstring (email)optionalA single bare email address, e.g. x@gmail.com. Provide it to import only the conversation with that address; omit for a full import.
window_startstring (RFC3339)see belowOldest point to walk back to, e.g. 2025-01-01T00:00:00Z.
window_endstring (RFC3339)optionalNewest point. Defaults to now.
countintegeroptionalCap on number of emails (newest-first). 0/omitted = no cap; bounded only by the window. Clamped to a server maximum (default 1000).
foldersobjectoptionalInclude non-INBOX folders: { "inbox": true, "sent": false, "spam": false, "trash": false }. Defaults to inbox-only.

When is window_start required?

  • Full import — provide at least one of window_start or count.
  • Single conversation (participant set) — window_start is optional. If omitted, it defaults to the last 1 year. (window_end still defaults to now.)

Examples​

Full backfill of a date window (inbox + sent):

curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"window_start": "2025-01-01T00:00:00Z",
"count": 1000,
"folders": { "sent": true }
}'

Single conversation with one participant (defaults to the last year):

curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"participant": "x@gmail.com"
}'
{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }

Idempotent per mailer: only one active full or single-conversation backfill per mailer at a time. A second request while one is in flight returns 409 with the existing jobId:

{ "success": false, "error": "active history backfill already exists", "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }

Check status​

GET /v1/imap/{mailerID}/history/{jobId}

{
"success": true,
"jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA",
"status": "processing",
"processed": 250,
"target": 1000,
"percent": 25,
"startedAt": "2026-06-03T10:07:23Z",
"endedAt": ""
}
  • status — pending → processing → done (or failed / cancelled). A participants job can also end as done_with_gaps; see Participants backfill.
  • percent — time-based progress over the requested window (clamped 0–100; 100 when done).
  • processed / target — emails delivered so far / the count cap (0 = uncapped).
  • endedAt — set once the job reaches a terminal state.

The job must belong to the addressed mailerID; a job from another mailer (or another tenant) returns 404.


Abort a history backfill​

POST /v1/imap/{mailerID}/history/{jobId}/abort

Cancels an in-flight job; the worker stops cleanly at its next slice, leaving the job cancelled. Already-finished jobs (done/failed/cancelled) return 409.

{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA", "status": "cancelled" }

After a job is cancelled (or finished), you can start a new backfill for the same mailer. Emails already delivered before cancellation stay delivered (dedupe by Message-Id).


Webhook delivery & dedupe​

  • Each historical email is delivered as a normal incoming-email webhook, with the full email content inline including attachments. The payload event is INCOMING_EMAIL for every connection type; IMAP_OAUTH and BASIC_IMAP are webhook registration types, not delivered event names.
  • The payload shape is identical to live incoming mail for the same connection, so an existing inbound handler needs no changes.
  • Backfill deliveries carry an X-History-Backfill-Job-Id header so you can distinguish them from live mail.
  • Because delivery is at-least-once, dedupe by Message-Id on your side — this is required, not optional.

Lifecycle events (optional)​

Alongside the per-email deliveries, a full or single-conversation job emits small lifecycle events to the same webhook so you can track it end-to-end:

  • HISTORY_BACKFILL_STARTED — when the first slice begins.
  • HISTORY_BACKFILL_PROGRESS — throttled progress updates (~every 10%).
  • HISTORY_BACKFILL_COMPLETED — on success.
  • HISTORY_BACKFILL_FAILED — on terminal failure.

Participants jobs do not emit lifecycle events; poll the status endpoint instead.


Participants backfill​

Use this when you already know who a mailbox talks to — for example, the contacts a CRM found in the last few months of sent mail — and you want every email exchanged with them, not just a recent window.

POST /v1/imap/{mailerID}/history with participants:

FieldTypeRequiredNotes
participantsarray of strings (email)yes1 to 50 bare email addresses. Matching is case-insensitive and repeated addresses are ignored.

participants cannot be combined with participant, count, window_start or window_end; sending any of them returns 400. folders is ignored, because a participants job always reads every folder except spam, trash and drafts.

What gets imported:

  • Who: any email where a listed contact appears in From, To, Cc or Bcc.
  • When: the mailbox's whole history, up to the moment the job was created. There is no window.
  • Where: every folder except spam, trash and drafts.
  • Skipped and counted, not delivered:
    • emails the platform already delivered for this mailer recently (by live inbound or an earlier backfill — roughly the last 7 days);
    • emails this mailbox sent through this platform, which you already have from the send API (counted in sentViaEdith). Emails it received from another mailer that sends through this platform are imported like any other email.
  • The mailbox's own address and aliases are dropped from the list and returned as ignoredContacts. A request that lists only those addresses returns 400.
curl -X POST "https://<your-host>/v1/imap/{mailerID}/history" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"participants": ["a@acme.com", "b@acme.com", "c@globex.com"]
}'
{ "success": true, "jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA" }

Parallel jobs: you can run several participants jobs on the same mailer at once, for example one request per batch of 50 contacts. An email shared by two jobs is delivered once. A participants job and a full or single-conversation job cannot run on the same mailer at the same time; the second request returns 409 with the running jobId.

Participants status​

GET /v1/imap/{mailerID}/history/{jobId} returns the usual fields plus per-contact counts:

{
"success": true,
"jobId": "01JC3BBW8S9YGX2VNKG5MD7BTA",
"status": "processing",
"processed": 300,
"target": 420,
"percent": 71.4,
"startedAt": "2026-11-03T10:07:23Z",
"endedAt": "",
"provisional": true,
"totals": {
"listed": 420, "processed": 300, "delivered": 250, "duplicates": 30,
"sentViaEdith": 15, "alreadyInJob": 3, "unmatched": 1, "failed": 1, "gone": 0
},
"contacts": [
{ "contact": "a@acme.com", "found": 120, "delivered": 100, "duplicates": 15, "sentViaEdith": 5 }
],
"ignoredContacts": ["owner@acme.com"]
}
  • target is the number of emails found so far and processed the number handled. Contacts are searched in groups, so target grows while the job runs and percent can move backwards. provisional stays true until the job ends.
  • totals counts each email once:
    • delivered — sent to your webhook.
    • duplicates — already delivered recently, not sent again.
    • sentViaEdith — sent by this mailbox through this platform, not sent again.
    • alreadyInJob — seen twice in this job (for example in two folders), sent once.
    • unmatched — a search hit whose headers name none of the contacts.
    • failed — could not be downloaded or parsed.
    • gone — deleted from the mailbox before it could be downloaded.
  • contacts counts per contact. An email that names three contacts counts once for each of them, so the per-contact numbers can add up to more than totals.
  • done_with_gaps means the job finished but not everything was read: some contacts or emails still failed after repeated retries, the mailbox stopped being readable (for example, the connection was revoked), or Outlook returned only part of the search results. reason says why in plain text, for example Outlook returned only part of the search results on busy days, so some emails may be missing for: a@acme.com (3 days, 2026-10-07 to 2026-10-12). All other emails were imported. It lists up to 5 contacts with the number of affected days and their date range, then "and N more contacts". Don't parse it. A participants job never ends as failed.

Abort works the same as for other jobs: emails delivered before the abort stay delivered.


Get one email by ref-id or Message-Id​

Fetch a single email of a connected mailbox, for example one a backfill counted as a duplicate or one your system knows only by its Message-Id. Give exactly one of:

QueryUse it whenHow it is found
refIdYou have the ref-id from an INCOMING_EMAIL webhookRead from the copy this platform stored. If the stored copy has expired, the call returns 404; look it up by messageId instead.
messageIdYou know the email's Message-Id (with or without <>)The stored copy is returned if there is one; otherwise the email is fetched from the mailbox, stored the same way as live inbound, and returned.
curl "https://<your-host>/v1/imap/{mailerID}/messages?messageId=%3Cabc%40mail.acme.com%3E" \
-H "Authorization: Bearer <API_KEY>"
{
"success": true,
"refId": "01M6G4QWRZ1YJACS8TVPNE5AYJ",
"source": "stored",
"email": { "event": "INCOMING_EMAIL", "details": { "...": "same fields as the webhook" }, "success": true }
}
  • email has exactly the shape of the INCOMING_EMAIL webhook body, so you can reuse your webhook parser.
  • source is stored when this platform already had the email, mailbox when it was fetched from the mailbox for this call.
  • No webhook is sent for an email returned here. Once fetched, a later backfill counts it as a duplicate.
  • URL-encode the Message-Id (< → %3C, > → %3E, @ → %40).
  • ref-ids from full or single-conversation backfills are the provider's own message ids, not stored copies, so refId returns 404 for them; use messageId. ref-ids from live inbound and participants backfills work with refId.
StatusMeaning
400Neither or both of refId and messageId given
404Mailer not found, email not in the mailbox, or the refId copy is no longer stored
409Mailer is inactive
422Provider not supported, or the mailbox can't be read (for example the connection was revoked)
429The mail provider is throttling this mailbox; retry after the Retry-After header

Notes & limits​

All connections​

  • Send-only mailers return 422. See Supported connections.
  • Window precision is rounded to whole seconds. Every mailbox's date filter is day-granular, so the platform widens the query and then filters on each message's real timestamp.
  • Throughput is bounded by bytes, not message count. A mailbox of 200 large attachments takes far longer than one of 2,000 short replies, because every message is transferred from the mailbox and then posted to your webhook in full. Use the status endpoint rather than estimating from message counts.
  • Your webhook is part of the critical path. A slow or failing endpoint slows or drops deliveries, so keep it fast and return 2xx quickly.

Gmail OAuth​

  • Throughput is additionally capped by Gmail's per-user API quota, so a large mailbox is paced rather than pulled at full speed.

Outlook OAuth​

  • Microsoft returns at most about 275 search results per folder. Single-conversation and participants searches split the date range in half whenever they reach that limit, so large histories are still covered. A single day can't be split further: if one contact has more matching emails than the limit on one day in one folder, the extra emails are skipped and the platform logs a warning.
  • Participants search runs folder by folder. When a day is still at the limit for a group of contacts, it searches each contact on their own for that day. If one contact is still over the limit on a single day in one folder, the extra emails for that day are skipped and the job ends as done_with_gaps, with a reason naming the contact and the affected days.

Basic IMAP​

  • Port 993 with SSL only. Other port and security combinations are rejected when the mailer is created.
  • One connection per job. Messages are fetched sequentially over a single IMAP session, so basic IMAP is slower per byte than the OAuth connections, which fetch in parallel.
  • Shared session limits. Many hosted IMAP servers cap concurrent sessions per account. If the live poller holds the only available session, the backfill cannot connect and the job retries. Running a backfill during quiet hours avoids this.
  • Participants search uses the server's all-mail folder when it has one, otherwise it searches each folder except spam, trash and drafts. On servers whose all-mail folder also holds drafts (Gmail over IMAP, for example), drafts addressed to a contact can be included.
  • Folder names are mapped, not assumed. The platform discovers the mailbox's real folders and maps them onto INBOX, SENT, SPAM and TRASH, so server-specific names such as [Gmail]/Sent Mail resolve correctly.