Both describe a full offline mail replica with a persistent cursor-based sync
engine. That scope was dropped in favour of "a SQLite index we can prompt
against" - see the notes prepended to each file for what shipped instead
(lib/mail-index/** + app/api/offline/{reindex,search}).
Kept rather than deleted because several findings are still accurate and still
load-bearing: the SQLCipher binding investigation, the PRAGMA-key
silent-no-op landmine, the safeStorage Linux basic_text hazard, the
hosted-deployment gate, and the codebase survey.
The review's note also records the disposition of every CRITICAL/HIGH finding.
Most became MOOT rather than fixed - C2, C3, C4, H1 and H2 were all
consequences of a long-lived worker holding credentials, and the new shape has
no worker. C1 (the Docker build breakage) and H2's env-vs-fd point were fixed
as specified, and the review's two corrections to the design (the
cipher_version check needing a non-empty string, getSelectedStorageBackend
being Linux-only) are both in the shipped code.
Also recorded: two things the design got wrong beyond the scope change - its
claim that the chosen process needs no new secret handling (the review was
right) and its assumption that Next's file tracing would carry the native
module (it does not).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
93 KiB
⚠️ SUPERSEDED — this is not what was built
This document designs a full offline mail replica: a persistent background sync engine with JMAP
Foo/changescursors, three state machines, a retry ladder, reconcile/sweep logic and an epoch-fenced multi-account registry. That scope was dropped. After the adversarial review (ELECTRON-OFFLINE-ENGINE-REVIEW.md), the human narrowed the requirement to "a SQLite index we can prompt against" — retrieval to feed an LLM, refreshed on each delivery/change event.What was actually built:
lib/mail-index/**+app/api/offline/{reindex,search}— an encrypted SQLite/FTS5 index over mail, calendar, contacts and file metadata, written by an ordinary request-scoped API route that the renderer's existing live JMAP push connection calls when something changes. No background worker, no cursors, no resident credentials. Staleness between refreshes is acceptable by design.Most of the review's CRITICAL and HIGH findings stopped existing rather than being fixed: C2, C3, C4, H1 and H2 were all consequences of a long-lived worker holding credentials, and there is no worker.
Still accurate and still worth reading here:
- §3 — the SQLite/SQLCipher binding investigation.
@signalapp/sqlcipheris what shipped, for the reasons given, and thePRAGMA keysilent-no-op landmine is real (the shipped code assertscipher_versionreturns a non-empty string, per the review's correction).- §6 —
safeStorage, including the Linuxbasic_texthazard. Shipped as described, withgetSelectedStorageBackend()correctly guarded to Linux only (a review finding).- §1 — the codebase survey (auth model, push pipeline, CSP, account model). All verified.
- §2.4's hosted-deployment gate (
VNCMAIL_DESKTOP_STORE_DIR) — shipped, and now covered by a test.- §14 — what was and was not empirically verified.
Wrong in hindsight, beyond the scope change: §2.1's claim that Option A needs no new secret handling (the review's C2 is right — credentials are request-scoped, not resident); and §2.1's assumption that Next's output file tracing would carry the native module (it does not — the standalone build needs an explicit copy step, now in
scripts/assemble-standalone.mjs).
Electron Offline Engine — Design
Status: superseded design, never implemented. See the note above. Nothing outside this file was
changed by the pass that wrote it; electron/main.ts, electron/preload.ts and
lib/jmap/client.ts were untouched at that time (main.ts has since gained the index's store-dir
and key-channel wiring, which is a small fraction of what this document describes).
Repo: brvncde-dotcom/vncmail-plus, branch claude/electron-offline-design, worktree
~/worktrees/vncmail-electron-sqlite. Based on claude/electron-desktop (the working desktop
shell + RFC 8887 WebSocket push), HEAD b15098a6.
Companion documents:
~/worktrees/vncmail-native-sync-impl/docs/DELTA-SYNC-ENGINE-DESIGN.md(revision 3, 2012 lines) — the finalized, twice-adversarially-reviewed, implemented and real-device-verified JMAP delta-sync design for this program's React Native client. This document is an adaptation of that one, not a replacement for it. Read it first; it is the normative source for everything marked [reused] below. Cited as M§n throughout.~/worktrees/vncmail-native-sync-impl/docs/DELTA-SYNC-DESIGN-REVIEW.md— the adversarial review that produced M's revision 2 (findings S1–S16). Cited as MR.docs/VNCMAIL-NATIVE-BUILD-MANUAL.md— program narrative; §4 decision log, §5 (what the Electron shell already has), §8 known landmines.~/.claude/skills/VNCprodbuild/SKILL.md— the build plan this belongs to.
Normative references, cited by section: RFC 8620 (JMAP core) and RFC 8621 (JMAP Mail), exactly as enumerated in M's preamble. This document does not re-derive any RFC reading; every protocol-level claim is M's, verified there.
0. How to read this document
The JMAP delta-sync problem is platform-independent. M solved it, was attacked twice over it, and shipped it. Re-deriving it here would produce a second, subtly different set of invariants for the same protocol — which is how one client silently loses mail the other doesn't.
So every section is tagged:
| Tag | Meaning |
|---|---|
| [reused] | Adopted from M unchanged. The cited M section is normative; this document only records that it applies and any Electron-specific naming. Do not re-litigate. |
| [adapted] | M's decision holds but its mechanism doesn't, because Electron's runtime differs. The difference is stated explicitly. |
| [new] | No M counterpart, or M's answer is actively wrong here. Designed from scratch in this document. |
The genuinely new work is §2 (which process hosts the engine), §3 (SQLCipher from day one), §6 (key storage), and the parts of §5/§7/§8 that follow from those. Everything else is M.
0.1 Scope
In scope: where the engine and its SQLite file live; which SQLite binding; whether encryption ships on day one and how its key is stored; how the store keys against this repo's account model; what the existing renderer-side push pipeline must and must not do once the engine exists; the schema; triggering; the staged rollout with its verify-first gate.
Out of scope, deliberately:
- FTS5 index population (VNCprodbuild step 9). §7.5 reserves the hook. Note §3.4: FTS5 is empirically present in every candidate binding, so this is not a binding-selection input.
- Offline compose/outbox. Unlike the mobile app, this repo has no outbox and no optimistic mutation layer at all (§1.6) — so M§5.6's read-time overlay has nothing to overlay. v1 desktop offline is read-only. This is a scope decision, recorded in §5.4, not an oversight.
- Attachment blob storage (M§9.4's second bullet applies verbatim when it lands).
- Calendar / Contacts / Files delta sync.
- Shared/group ("delegated") mail. Account-scoped primary keys are in place from day one (M§9.3, MR S3) so adding it later is inserting rows.
- Code signing (VNCprodbuild step 9). It is referenced in §6.3 because it interacts with macOS key storage, but it is not resolved here.
Non-goal: compatibility with anything on disk today. There is nothing on disk today (§1.6).
1. What exists today, verified
File:line references are to this worktree at b15098a6. Everything in this section was read, and
every runtime claim in §3/§6 was executed against the Electron binary actually pinned by
package.json — see §3.1 for the transcript summary.
1.1 The desktop shell
electron/main.ts (225 lines) boots the same Next.js output: "standalone" artifact the
Dockerfile ships (next.config.ts's output: "standalone"), as a child process:
getStandaloneServerEntry()(:26-31) —process.resourcesPath/standalone/server.jswhen packaged,.next/standalone/server.jsin dev.startStandaloneServer()(:70-105) — allocates a random free localhost port (:33-48), spawnsprocess.execPathwithELECTRON_RUN_AS_NODE: "1"(:85-94) so no system Node is required, then polls until reachable (:50-68).createMainWindow()(:114-143) —BrowserWindowwithcontextIsolation: true,nodeIntegration: false,sandbox: true, preload atdist-electron/preload.js, loadinghttp://127.0.0.1:<port>.- The notification bridge:
ipcMain.handle("vnc:show-notification", …)(:155-176), reached fromelectron/preload.ts:16-27'scontextBridge.exposeInMainWorld("vnc", …), wrapped bylib/electron-bridge.ts'sisElectronShell()/showElectronNotification(), and called fromapp/(main)/[locale]/page.tsx:1198-1222. This is the existing IPC pattern — oneipcMain.handle+ onecontextBridgemethod, no channel registry, no streaming.
Packaging (electron-builder.config.js): the standalone server ships as extraResources copied
from: ".next" with a standalone/**/* filter — deliberately, to dodge app-builder-lib
unconditionally dropping a copy-root directory literally named node_modules (documented in that
file, and in the manual §5 as a bug found by actually launching a --dir build). Targets: macOS
dmg+zip x64 and arm64, Windows nsis x64, Linux AppImage+deb x64. Unsigned.
scripts/build-electron.mjs bundles electron/*.ts with esbuild, CJS, external: ["electron", "electron-updater"].
CI: .github/workflows/electron-build.yml, matrix macos/windows/ubuntu, Node 22 on the runner
(note: not Electron's Node — see §3.3), npm run test:electron as a required gate before packaging.
1.2 The JMAP client, and where the credentials actually are
lib/jmap/client.ts (7413 lines) is a renderer-side class. JMAPClient (:542) holds
serverUrl, username, password and an authHeader built in the constructor as
Basic ${btoa(username:password)} (:579-585), or Bearer … via static withBearer (:586-598).
authenticatedFetch (:672) is plain browser fetch() straight to the mail server. There is no
JMAP proxy route in front of it for normal traffic.
But the credentials are also recoverable server-side, and that is the load-bearing fact for §2.
app/api/auth/session/route.ts:
POSTstoresencryptSession(serverUrl, username, password)— AES-256-GCM underSESSION_SECRET(lib/auth/crypto.ts:23-33) — in an httpOnly cookiejmap_session[_<slot>](lib/auth/session-cookie.ts), one per account slot,MAX_ACCOUNT_SLOTS = 50(lib/account-utils.ts).GETreturns only{serverUrl, username};PUTreturns the full credentials for session restoration, gated onSec-Fetch-*headers proving a same-origin browserfetch().- OAuth/TOTP accounts instead park a refresh token in an httpOnly cookie
(
app/api/auth/token/route.tsPOST), andPUTon that route mints a fresh access token from it — rotating the stored refresh token whenever the server returns a new one (:104-106). Remember that; §2.5 has to keep two independent refreshers from existing.
So: any code running in the standalone server process can, for any account slot, obtain either a Basic auth header (decrypt the session cookie) or a bearer token (refresh-token grant) without a single new credential path, IPC message, or storage location. This is not true of Electron's main process, which sees none of those cookies.
1.3 The push pipeline as shipped, and the wall it hit
setupPushNotifications() (:6130) prefers RFC 8887 JMAP-over-WebSocket
(getWebSocketUrl(), :3803-3809, reading the urn:ietf:params:jmap:websocket capability off the
session — never hardcoded), falling back to SSE, then polling. Tight reconnect ladder (200 ms base /
5 s cap), 30 s heartbeat, and a 3-consecutive-handshake-failure circuit breaker
(wsPermanentlyDisabled, :6060ff).
The committed comment at :6038-6059 records the empirical outcome, and it is the single most
important existing finding for this design:
stalwart.sandbox.vnc.de's/jmap/wsrequires the same HTTPAuthorizationheader as every other JMAP endpoint on the WebSocket UPGRADE request itself. The browserWebSocketconstructor cannot attach custom headers (a WHATWG restriction; credentials-in-URL are rejected too). So from the renderer, every attempt fails the handshake and the circuit breaker correctly falls back to SSE.
And the alternative it explicitly declined (:6055-6059):
opening it from Electron's main process via a header-capable client like the
wspackage "would mean piping raw credentials from the renderer to the main process over IPC, which is a materially bigger security-sensitive change than what was scoped here."
That objection is correct for the main process and inapplicable to the standalone server — which already holds the credentials (§1.2) and would pipe nothing. §2.5 acts on this.
The transport-agnostic "genuine new mail" signal is email-store.newEmailNotification
(stores/email-store.ts:3082-3086, set by refreshCurrentMailbox), consumed once in
app/(main)/[locale]/page.tsx:1198-1222. It already fires identically over WS, SSE and polling.
The engine must not add a second notification path. §2.5 states the rule.
1.4 Multi-account model (differs from mobile — check, don't assume)
stores/account-store.ts — a Zustand persist store named account-registry, holding
AccountEntry[] with:
id:`${username}@${new URL(serverUrl).hostname}`vialib/account-utils.ts generateAccountId(). Same shape as mobile'sLocalAccountId— a genuine coincidence worth stating, because it means M§3.1'sLocalAccountIdtype carries over verbatim.cookieSlot: number— new relative to mobile. The index into the per-slot cookie namespace of §1.2, assigned bygetNextCookieSlot()(first free integer, reused after removal).serverIdentifiers?: string[]— server-confirmed account-id forms captured at login, used by the account-switch guard so a short login name canonicalized by the server is still recognized.activeAccountId,defaultAccountId; capsMAX_ACCOUNTS_HTTP1 = 5(HTTP/1.1 SSE-connection budget) lifting toMAX_ACCOUNT_SLOTS = 50once h2/h3 is observed.
Two consequences for the schema (§7):
- The durable key is
accountId(username@host), nevercookieSlot— slots are recycled bygetNextCookieSlot(), so a slot number is a transport detail with a shorter lifetime than the data. Aslot → accountIdconfusion is a cross-account data-mixing bug of exactly M's D6 shape. - Any API surface addressed by slot (as §1.2's routes are) must resolve slot → accountId and re-verify against the session's confirmed username before touching the store. §5.3.
1.5 CSP — a hard constraint on option C
proxy.ts:88-141 builds the CSP. In production:
script-src 'self' 'nonce-<n>' # no 'unsafe-eval', no 'wasm-unsafe-eval'
connect-src 'self' https: wss: # 'wss:' was added for §1.3's WS push
'unsafe-eval' exists only for isDev and the plugin-sandbox path. WebAssembly compilation
requires 'wasm-unsafe-eval' or 'unsafe-eval' under CSP3. §2.3.
1.6 What does not exist here (and does in the mobile repo)
This is the inverse of M§1.1/§1.2, and it is mostly good news:
mobile (vncmail-native) |
here | |
|---|---|---|
| Existing offline cache | offline-sync.ts + offline-cache-store.ts, carrying defects D1–D8 |
nothing. No IndexedDB mail cache, no offline list, no offline read path. lib/plugin-storage.ts uses IndexedDB but only for plugin assets. |
Email/changes / Mailbox/changes wrappers |
already present, already driving an incremental list path | none. The only occurrence in the repo is a mock in app/api/dev-jmap/[...path]/route.ts:1949. Greenfield. |
| Outbox / optimistic mutations | outbox-store.ts, full-state idempotent queue |
none. Mutations go straight to the server. |
| Push transport | SSE + FCM relay | WS (blocked, §1.3) → SSE → polling, all renderer-side |
| Stalwart integration fixture | in a sibling repo — MR S16 costed this as real cross-repo CI work | in this repo: integration/docker-compose.yml + 11 specs incl. 11-electron-notification.spec.ts, which logs in against real Stalwart, injects mail over real SMTP, and asserts the native notification. Free to extend. |
Therefore M§1.3's defect list D1–D8 does not apply here. There is no legacy cache to inherit
bugs from, no patchCache() write-through to delete, no D4 cursor fast-forward in shipped code, and
M§14.1's "discard, don't migrate" is vacuous. What does carry over is the class of each defect
as a thing not to introduce — which is what M's invariants I1–I13 are for (§4.3).
One inherited defect shape is worth naming, because this repo has it too: stores/file-store.ts
and others use try { localStorage.setItem(...) } catch { /* ignore */ } in a dozen places — M's D2
pattern. Banned in the sync path (M I4). §7.2.
2. Decision 1 — which process hosts the engine [new]
M has no counterpart: React Native has one JS context. Electron has three candidate homes, and this codebase makes the choice non-obvious in both directions.
2.1 Candidate A — engine + SQLite inside the standalone Next.js server process
The renderer reaches cached data through new app/api/** routes, exactly as it reaches everything
else server-side today.
For:
- The credentials are already there, encrypted, per account (§1.2). No new credential path, no
IPC carrying secrets, no second copy of the TOTP/refresh state machine. Every other candidate has
to solve this, and B can only solve it by doing the thing
client.ts:6055-6059explicitly declined. - It unlocks real WS push, which the renderer structurally cannot have (§1.3). A Node process
can set
Authorizationon a WebSocket upgrade (wspackage). This is not a side benefit: it converts a documented dead end into a working transport, on the server this program actually targets, with no Stalwart-side change. §2.5. - No new IPC surface at all.
window.vncstays a one-method bridge. Nothing aboutcontextIsolation: true/sandbox: truehas to be relaxed or extended. - Native module packaging is already solved for this process. The standalone server ships as
extraResourcesoutsideapp.asar, with its own tracednode_modules— the exact copy path whose one footgun is already found, fixed and documented (electron-builder.config.js). A.nodebinary in an unpacked directory needs noasarUnpackreasoning at all. - Blocking is cheapest here.
better-sqlite3and@signalapp/sqlcipherare synchronous (§3.2). Blocking this event loop delays local-cache HTTP responses; it does not block the renderer's paint (React runs in the renderer) and does not block window/menu/IPC handling (that's the main process). It is the least latency-critical of the three loops. - Reads are trivially observable — the existing integration suite drives the app over HTTP and can assert on new routes without any Electron-specific harness.
Against:
- This process is also what a hosted, multi-user Docker deployment runs. Unconditional offline routes would have a shared server start caching every user's mail into a server-side SQLite file. This is the strongest argument against A and it must be closed by construction, not by convention — §2.4.
- Next.js output-file-tracing vs. a native module.
serverExternalPackages(already used foresbuild,next.config.ts) plus NFT must actually carryprebuilds/**/*.nodeinto.next/standalone/node_modules.node-gyp-build's resolution is directory-scan-based, which NFT handles specially but not infallibly. Verify-first, §12 Stage A. - The DB path must be handed in:
app.getPath('userData')is a main-process API, somain.tsmust pass it as an env var on spawn (:85-94already builds the env). One line, but it is a coupling. - Server-process lifetime is
window-all-closed/before-quit(main.ts:210-219), so an in-flight cycle is killed by process death rather than by a cooperative abort. M's crash-recovery design (I1, M§6.3) already makes that safe — cost is one page — but a graceful-shutdown IPC is worth adding later.
2.2 Candidate B — engine + SQLite in electron/main.ts, over contextBridge/IPC
For:
- The standalone server's code stays byte-identical to a hosted deployment's. A's §2.1-against-1 simply does not arise: there is no Electron-only server code to accidentally ship in Docker.
- Mirrors the existing notification bridge, so the pattern is familiar.
electron-builderalready auto-unpacks**/*.nodefrom the asar, and productionnode_modulesare collected regardless of the narrowfiles: ["dist-electron/**/*", "package.json"](that's whyelectron-updaterisexternalinbuild-electron.mjsand still ships). Low packaging friction.safeStorage(§6) lives in the main process natively — no bridging for the key.
Against:
- It requires exactly the thing
lib/jmap/client.ts:6055-6059refused. The engine needs credentials. The main process has none: thejmap_session/ refresh-token cookies belong to the renderer's origin. So either the renderer ships the Basic header / bearer token over IPC (the declined "materially bigger security-sensitive change"), or the main process learns to read Electron's cookie jar (session.defaultSession.cookies) and re-implementdecryptSession— which means shippingSESSION_SECRETinto the main process too. Both are net-new secret handling to reach a place that currently, deliberately, holds no secrets. - A large new IPC surface. The offline read path is not one fire-and-forget notification; it is
list queries, single-message reads, per-account status, settings changes and abort signals — each
an
ipcMain.handlereturning structured data acrosscontextIsolation. Every one is a new trust boundary in a window that is currentlysandbox: truewith a 12-line preload. - Blocking hurts most here. Synchronous SQLite on the main process's loop is jank in window
dragging, menu response and IPC dispatch. Mitigable with
worker_threads, but then B is A's complexity plus IPC. - The renderer's offline read path becomes Electron-only by construction, so the web/PWA deployment can never share it. That may be acceptable — but it is a fork, and A's routes would work in both.
2.3 Candidate C — engine in the renderer, WASM SQLite over OPFS
Investigated, not assumed. Findings:
- The official WASM build is
@sqlite.org/sqlite-wasm(3.53.0-build1), whichsqlocal(0.18.0) wraps for OPFS. Neither has any encryption — SQLCipher is a fork of SQLite's source, not a loadable extension, so an official build cannot have it. - Encrypted WASM builds do exist on npm:
@7mind.io/sqlcipher-wasm(1.2.0, "production-ready WebAssembly build of SQLCipher with real OpenSSL-based encryption") and@aztec/sqlite3mc-wasm(5.1.0, SQLite3MultipleCiphers 2.3.5 as WASM). So the honest answer to "does a WASM SQLCipher genuinely exist?" is yes, but only from small third-party publishers — not from the SQLite project, not from a vendor with a desktop-mail-scale user base. For a component whose failure mode is "the user's whole mailbox is readable on a stolen laptop", that provenance is the finding. - The CSP problem is decisive independently of encryption. §1.5: production
script-srcis'self' 'nonce-…'. WASM compilation needs'wasm-unsafe-eval'. Adding it inproxy.tswidens the CSP for every deployment of this product, including the hosted web one, to buy a desktop-only feature. That is a security regression with the wrong blast radius. - Even granted both, the encryption key would live in the renderer's JS heap — the same context that renders untrusted HTML mail bodies and hosts the plugin sandbox. A/B keep it in a Node process the renderer cannot address.
- Secondary, verify-first if C is ever revisited: the official
opfsVFS usesSharedArrayBuffer+Atomics.waitand therefore needs COOP/COEP headers;opfs-sahpooldoes not. Neither is configured inproxy.tstoday.
Against, summarised: requires a product-wide CSP widening, puts the key in the most exposed context, and its only encrypted backends are unvetted third-party WASM builds. For: no IPC, no native module, no packaging story, works identically in the browser PWA — a real benefit, and the reason to keep C on record rather than dismiss it. If the offline store were unencrypted and browser-first, C would be the right answer. It isn't either.
2.4 Decision: A, with the hosted-deployment gate as part of the design
The engine and the SQLite file live in the standalone Next.js server process. Rationale in priority order: it is the only candidate where credentials are already present and correctly scoped (§2.1-for-1); it is the only candidate that makes RFC 8887 push actually work (§2.1-for-2); it adds no IPC and no preload surface; and its native-module packaging path is the one already exercised and debugged in this repo.
A's one serious objection — the same process serves hosted multi-user deployments — is closed structurally, not by convention. Three layers, all required:
- A desktop marker env var.
main.ts's spawn env (:85-94) gainsVNCMAIL_DESKTOP_STORE_DIR=<app.getPath('userData')>/offline. Absent or empty ⇒ the engine module is never constructed, and this also supplies §2.1-against-3's path. One variable does both jobs, so they cannot drift apart. - Every new route refuses to run without it.
app/api/offline/**returns404(not 403 — nothing should learn the routes exist) when the marker is unset. This mirrors the existing "routes 503-on-misconfig" habit elsewhere in the program. - A single-user assertion. With the marker set, the engine asserts at open time that the store
directory is per-OS-user (it is, being under
userData) and records the resolvedserverUrl+usernameof every account it materialises. A store whose recorded account set doesn't match the requesting session's is a purge trigger (§5.5), not a merge.
Additionally, and non-negotiably: the engine runs on a worker_threads Worker inside the server
process, never on the request event loop. Synchronous SQLite plus JMAP page application is exactly
the workload that turns a shared event loop into a latency problem, and M's own I11 (jobs strictly
sequential within an account, M§3.4) is naturally expressed as "one worker per account, one job at a
time" rather than as a hand-rolled mutex. API routes talk to the worker via postMessage and never
touch the database handle. This also localises §2.1-against-4: the worker gets an explicit
terminate path.
Rejected explicitly, for the record: B, because it can only be built by moving credentials into a
process that today holds none — the change client.ts already declined on its merits, and nothing
about an offline store makes that trade better. C, because it needs a product-wide CSP widening and
its only encrypted backends are unvetted.
2.5 Consequence for the already-working WS-push renderer code
This is the question that must not be answered by accident.
The renderer's push pipeline stays exactly as it is. No line of lib/jmap/client.ts changes for
v1. Its SSE/polling path is the renderer's own liveness for the visible list, it is working, and
it is what feeds newEmailNotification → showElectronNotification (§1.3). The engine does not
replace it and does not read from it.
Three rules, in order of how easy they are to get wrong:
- The engine gets its own push connection, and it is the header-capable one. In the server
process the engine opens
wss://…/jmap/wswith anAuthorizationheader (viaws), which is the connection the renderer cannot open (§1.3). It subscribes withWebSocketPushEnable, and treats the resultingStateChangeexactly as M§10.4 specifies: a wake signal, never a cursor. M's two load-bearing rules (a pushednewStateis never written as a cursor; state-equality against our cursor is a cheap safe dedupe) apply verbatim. [reused: M§10.4] - The engine never fires a notification.
newEmailNotificationremains the single source of the OS notification, in the renderer, via the existing bridge. An engine-side notification would double-notify on the common path (both connections see the same delivery) and diverge on the uncommon one. What the engine may do is expose "account X changed" on its status channel; the renderer decides whether to refresh, exactly as M§5.7 specifies the engine→email-store direction (and only that direction). - Duplicate work is bounded and acceptable; duplicate state is not. Yes, two connections to
the same server per account, and both wake on the same delivery. That is deliberate, and it is
M§5.7's argument transplanted: the renderer's list cursor and the engine's
/changescursors page differently, invalidate differently, and one being wrong must not corrupt the other. The engine must never read the renderer'slastStates(client.ts:571) and the renderer must never read the engine's cursors. The cost is one extra socket per account; the cap isMAX_ACCOUNTS_HTTP1 = 5today, and the engine's socket is server→server, so it does not consume the browser's per-origin HTTP/1.1 connection budget that cap exists to protect.
Deferred, and worth stating so it isn't done silently: once the engine's WS connection is proven,
the renderer's transport could be retired in favour of the engine pushing "account changed" down an
SSE/EventSource from the local server — one server-side socket per account instead of two, and the
renderer's circuit-breaker-into-SSE path becomes dead code. That is a follow-up, gated on the
engine's connection being verified against real Stalwart, not part of v1. Doing it in v1 would make
a working notification path depend on an unproven one.
3. Decision 2 — SQLite binding, and SQLCipher on day one [new]
M deferred useSQLCipher for one specific reason: Expo Go cannot load it, so a plaintext-first phase
was the only way to keep the day-to-day dev workflow (M§9.2, M§14.3 step 3.1, MR S4/V4). Electron
has no Expo Go. The deferral's entire justification is absent, so the question is genuinely open
here and has to be answered on the evidence.
3.1 What was measured, and how
All of the following was executed against the Electron binary this repo pins
(electron@43.2.0, resolved from ~/worktrees/vncmail-electron/node_modules — this worktree has no
node_modules installed), both as the main process and under ELECTRON_RUN_AS_NODE=1 (the mode the
standalone server actually runs in, main.ts:88):
| Fact | Result | How |
|---|---|---|
| Electron 43.2.0's bundled Node | 24.18.0, ABI modules=148, napi=10 |
process.versions |
| Its bundled SQLite | 3.53.1, with ENABLE_FTS5 |
pragma compile_options |
node:sqlite present and working |
yes; exports DatabaseSync, StatementSync, Session, constants, backup; no ExperimentalWarning observed |
require('node:sqlite') |
node:sqlite encryption |
none. compile_options has no codec. PRAGMA key='…' is silently accepted and does nothing — the file was written with a SQLite format 3 header and a plaintext canary string recoverable with grep |
wrote a real file, read the bytes back |
node:sqlite stability (Node 24) |
1.2 — Release Candidate (RC since v24.15.0; no longer behind --experimental-sqlite), not stability-2 stable |
Node 24 docs |
better-sqlite3@13.0.2 |
installs with zero build step; ships in-tarball N-API prebuilds for darwin-{arm64,x64}, linux-{arm64,x64}, linuxmusl-{arm64,x64}, win32-{arm64,x64}; loads in Electron 43 both as main process and under ELECTRON_RUN_AS_NODE; FTS5 available; PRAGMA key silently a no-op |
npm install + load in Electron |
better-sqlite3 12.x vs 13.x |
12.x used install: prebuild-install || node-gyp rebuild (per-ABI downloads). 13.0.0 dropped that for gypfile: false + in-tarball prebuilds — i.e. moved to ABI-stable N-API. This is why no electron-rebuild is needed |
npm metadata for 13.0.2 vs 12.11.1 |
better-sqlite3-multiple-ciphers |
latest is 12.11.1 (2026-06-18) — on the old 12.x prebuild-install model. Its GitHub release carries 98 Electron prebuilds, ABIs 121…146. Electron 43 needs ABI 148 — absent. So it would fall through to node-gyp rebuild: a C++ toolchain + Python + Electron headers on every contributor machine and every CI runner |
npm metadata + GitHub releases API |
@signalapp/sqlcipher@4.0.3 |
N-API (prebuildify --strip --napi, node-gyp-build), in-tarball prebuilds for darwin-{arm64,x64}, linux-{arm64,x64}, win32-{arm64,x64}. Loads in Electron 43 with no rebuild, both process modes. Real SQLCipher 4.10.0 community; PRAGMA cipher_version reports it; file header is ciphertext, canary absent from the bytes, wrong key rejected with SQLITE_NOTADB, right key reads the row back; FTS5 available; better-sqlite3-shaped synchronous API (db.exec, db.prepare().run()/all(), db.pragma()); AGPL-3.0-only, matching this repo's own licence |
npm install + full round-trip in Electron main process |
Two of those deserve to be called out as landmines rather than table rows:
PRAGMA keyfailing silently is the worst possible ergonomics. On bothnode:sqliteand plainbetter-sqlite3, setting a key "works", the database works, and the mail is on disk in cleartext. There is no error to notice. Whatever binding ships, the store's open path must assert encryption positively — readPRAGMA cipher_versionand refuse to proceed if it is empty — and a test must assert a canary string is absent from the raw file bytes. Both are in §11.better-sqlite3-multiple-cipherslags Electron by roughly one to two majors (ABI 146 vs 148, and its 12.x base trails better-sqlite3's 13.x). That lag is structural, not a one-off: with per-ABI prebuilds, every Electron major bump re-opens the question.@signalapp/sqlcipher's N-API prebuilds are immune to Electron majors by construction.
3.2 Recommendation: ship SQLCipher on day one, via @signalapp/sqlcipher
The friction the mobile design was avoiding does not exist here, and the evidence is unusually
clean: a package built and maintained specifically to run SQLCipher inside an Electron desktop
application, N-API so it needs no rebuild against Electron 43, prebuilds covering every platform this
repo actually packages (§1.1: darwin x64+arm64, win32 x64, linux x64 — all present), FTS5 already
compiled in, an API close enough to better-sqlite3 that the backend is the same code either way,
and a licence identical to this repo's.
Verified working, in this environment, against this Electron version. Not inferred.
So: no plaintext-first phase. M§14.3's plain-then-encrypted staging exists to protect a dev workflow that has no analogue here; importing it would mean deliberately shipping a plaintext mailbox on disk for a phase, plus building and then discarding the store-format-migration machinery of M§8.4.1 to get out of it. Both costs, no benefit.
Corollaries:
- The abstraction boundary of M§9.1 (
SyncStore/SyncTxn) stays exactly as M specifies. It is what makes this reversible: if@signalapp/sqlcipherever becomes untenable,store-sqlite.tsis the only file that changes. Do not skip it on the grounds that the binding question is now settled — the boundary is also what makesstore-memory.tspossible, and M§13's contract-tests-against-two-backends is most of the test plan's value. - M§8.4.1's out-of-band store-format marker is still required, for the reason V4 gives, minus one:
schemaVersionstill lives inside a file that a future format change could make unreadable, so it must be mirrored outside. What day-one encryption removes is only the plain→cipher transition that would otherwise be the marker's first customer. Keep the marker; it costs a JSON file. node:sqliteis rejected, on two independent grounds: no encryption at any price, and stability 1.2 (RC) for a component holding the user's mail. Its one advantage — zero dependencies — is worth nothing once encryption is a requirement. Worth re-evaluating only if a decision is ever taken to ship unencrypted.- Plain
better-sqlite3@13.0.2is the fallback, not the plan: adopt it only if Stage A (§12) finds@signalapp/sqlciphercannot be packaged, and in that case the human decides between "unencrypted desktop store" and "no desktop store yet" (§13, open question 1).
3.3 Friction that does exist, stated plainly
Not zero, just small — and the human should see it rather than have it smoothed over:
- A ~6 MB native dependency with 6 platform prebuilds in the tarball, entering
dependencies. Install size andnpm citime grow for everyone, including web-only contributors who will never run Electron. - NFT tracing is the real unknown (§2.1-against-2).
serverExternalPackagesplus a verification thatprebuilds/**/*.nodereached.next/standalone/node_modulesis Stage A's first task. If NFT won't carry it, the fallback is a copy step inscripts/assemble-standalone.mjs— which already exists precisely to patch up what standalone output omits, so this is a known-shaped fix. - CI runs Node 22 while Electron bundles Node 24 (§1.1). Harmless for an N-API module (the
prebuild is selected by platform+arch, not ABI) — but it would have been fatal for a
per-ABI package, which is worth recording as another reason the N-API choice matters. Any
npm testthat loads the binding under the runner's own Node exercises a different Node than production; the binding load assertion must therefore run inside Electron (npm run test:electron), not only in vitest. - Cross-arch macOS packaging. CI builds both x64 and arm64 dmg/zip on one macOS runner.
In-tarball prebuilds ship all platforms, so this works — whereas
prebuild-installdownloads only the host's, and would have broken the cross-arch target. Verify in Stage A that the arm64.nodeis what ends up in the arm64 build and vice versa. linuxmuslis not covered by@signalapp/sqlcipher(better-sqlite3does cover it). Irrelevant for AppImage/deb (glibc); relevant if an Alpine-based container ever wants the engine — which, per §2.4, it must not.
3.4 What is not an input to this decision
FTS5 (VNCprodbuild step 9) is present in all three candidates — Electron's own bundled SQLite,
better-sqlite3 13, and @signalapp/sqlcipher (which additionally ships Signal's FTS5 segmenting
extension and an initTokenizer()). So the search step cannot be used to argue for a binding.
Recorded so a later session doesn't relitigate the choice on those grounds.
4. The sync engine itself — mostly M, verbatim
Everything in this section is [reused] unless marked otherwise. The M section cited is normative; what follows is a map, not a restatement, so that a reader can tell reuse from re-derivation at a glance.
4.1 Architecture: three state machines [reused: M§2, M§3.4 I11]
Per account: A delta (Mailbox/changes then Email/changes, one cursor each), B coverage
(the envelope-window enumeration; /changes structurally cannot deliver pre-existing mail, so
coverage owns history and is also the bootstrap), C1 body-queue drain, C2 body backfill
(MR S9 — without it, widening body retention silently does nothing for already-covered envelopes).
Logically independent state, operationally serialised (I11). Here that serialisation is structural rather than disciplinary: one worker thread per account, one job at a time (§2.4). M's warning stands regardless — "run bodies in parallel, it's separate state" is forbidden, and F48 (a body landing for an envelope destroyed in the same cycle) is what it costs.
Module layout: M§2.2 verbatim, relocated. src/sync/** in the mobile repo becomes lib/sync/**
here (this repo has no src/): engine.ts, cursor.ts, apply.ts, coverage.ts, bodies.ts,
retention.ts, errors.ts, states.ts, store.ts, store-sqlite.ts, store-memory.ts. M's hard
requirement that apply.ts be pure (no network, no storage, no store access) carries over
unchanged and is the single highest-leverage constraint in the document: it is what turns M§11's
failure-mode table into a vitest suite. overlay.ts is not ported — see §5.4.
4.2 Two record tiers, two retention windows [reused: M§2.1]
Envelope tier = EMAIL_LIST_PROPERTIES — which in this repo is
lib/jmap/client.ts:139-154: id, threadId, mailboxIds, keywords, size, receivedAt, from, to, cc, subject, preview, hasAttachment, blobId. (Note the extra blobId, present so list rows can serve
drag-out to the filesystem as .eml; it belongs in the envelope tier here.) Body tier =
bodyStructure, textBody, htmlBody, bodyValues, attachments, bcc, replyTo, sentAt.
Independent retention: offlineEnvelopeDays ≫ offlineBodyDays, MB cap on bodies only. The
human decision M records — widen envelopes well beyond bodies, so a message never falls out of the
offline list over a body-size cap — is a program-level decision and applies here identically.
Concrete numbers remain open (§13).
4.3 Cursors, provenance and invariants [reused: M§3]
Adopted without modification:
SyncCursor/CoverageState/BodyQueueEntry/AccountSyncStateas M§3.1 defines them, including per-cursor failure counters (MR S6),sweepFloor+deferredTargetFrom(MR S2), andgapMarkers.- Branded state types (M§3.2):
ChangesStatevsSnapshotState,advanceCursor(key, next: ChangesState)as the delta path's only cursor write,seedCursor(key, commitment: EnumerationCommitment)for bootstrap/reconcile, andEnumerationCommitmentmade genuinely unforgeable by an unexported realSymbol()tag — including M's implementation note thatdeclare const … : unique symbolemits no runtime value and throwsReferenceErroras a computed key. That note came out of M's actual build; it would have been re-discovered here otherwise. - The ordering rule as I2, not the false "only a changes state, ever" of M's revision 1.
- I1–I13 in full (M§3.4). Cursor-last; provenance-as-ordering; monotonic-or-invalidated; no silent write loss; idempotent application; account containment; deletion provenance; no clock-dependent cursors; bounded work; no wedge; sequential execution; field-level state writes; corrupt-state-blob ⇒ resync.
- The "not a cursor" list (M§3.3):
Email/query'squeryState, a pushedStateChange.newState,sessionState, Thread state,EmailDelivery.
Two Electron notes on I12/I4, both simplifications rather than changes: with a real SQLite
BEGIN…COMMIT in place from day one (§3.2), M's per-account mutex and read-merge-write discipline
become belt-and-braces rather than load-bearing (M§9.2 anticipated exactly this), and I4's "every
write either succeeds or raises" is the binding's default behaviour rather than something to enforce
against a fire-and-forget storage API.
Cursor keying: (LocalAccountId, JmapAccountId, CursorType), all three required, for M§3.1's
reasons. LocalAccountId is this repo's AccountEntry.id (§1.4) — never cookieSlot.
4.4 Bootstrap [reused: M§4]
Replace-the-code-keep-the-shape does not apply (there is no runOfflineSync here), but M§4.1's
mandatory order does, and it is the one thing in this document most likely to be "optimised" into
a permanent data hole:
- Capture both cursors first, in one JMAP request (
Mailbox/get {ids: []}+Email/get {ids: []}), andseedCursorthem inside oneEnumerationCommitmentthat in the same transaction writescoverage {phase:'scanning', targetFrom, sweepFloor: targetFrom}. - Full
Mailbox/get→ upsert every mailbox row. - The seeded cursors are live from here: each cycle runs A1, A2, then B.
- Scan reaches
targetFrom⇒coveredFrom = sweepFloor,phase = 'complete'. Bootstrap has no delete sweep; only reconcile sweeps.
The cursor is deliberately older than the data, so the first delta cycle re-delivers some changes we already have. That is I5 working. The cheaper opposite order silently loses mail.
4.5 Change application [reused: M§5.1–§5.5]
Order within a cycle (A1 → A2 → B → C1 → C2) and M's timeline argument for why delta-before-coverage
is safe given I11 — the resurrection hazard, and the fact that the unsafe configuration is
concurrency, not ordering. Mailbox/changes updatedProperties count-only optimisation (RFC 8621
§2.2). Email/changes: created ⇒ envelope fetch + conditional body enqueue; updated present
⇒ a 3-property Email/get {id, keywords, mailboxIds} and never a body (RFC 8621 §4.1 — the two
mutable properties); updated absent ⇒ unconditional no-op with the ids filtered out before
the fetch is issued (MR S16); destroyed ⇒ delete envelope + body + membership + queue row.
Create-then-update-then-destroy ordering within a page. notFound is normal, not an error.
Mailbox/Email transient inconsistency tolerated, never repaired (I7: no deletion by inference).
4.6 Pagination [reused: M§6]
Ascending keyset walk on receivedAt with calculateTotal: false; after is spec-inclusive
(RFC 8621 §4.4.1, MR S14) so boundary re-delivery is normal and deduped by id; forward progress needs
strictly-greater max(receivedAt); the no-progress guard is anchor/anchorOffset first and only
then, on anchorNotFound, a +1 ms advance with a WARN and a durable gap marker. Position-based
paging is rejected for M§6.2's reason. Budgets per M§6.4 — but see §8.2 for the desktop numbers.
4.7 Errors, retry, reconcile, anti-wedge [reused: M§7]
The seven-class taxonomy (Transport / RateLimit / ServerTransient / RequestLimit / Auth / Fatal /
StateInvalid), with exactly one class moving a cursor and unrecognised method errors defaulting
to ServerTransient. Full-jitter backoff. "Offline is not an error." Partial-failure semantics inside
a page (records may commit partially; the cursor may not). The eight cursor-advance rules of M§7.5.
cannotCalculateChanges handled as RFC-mandated without blanking the UI, with the pinned
sweepFloor (MR S2) and with the freshly-seeded cursor live immediately so a wide-window
rebuild doesn't stall incoming mail (MR S9). oldState mismatch re-issued once before escalating,
plus the ≤4-reconciles-per-24 h ceiling (MR S10). The monotonically shrinking maxChanges ladder
with every rung clamped to rung 0 (MR S7 + V2), and per-cursor counters with "any job failed ⇒ cycle
failed for escalation purposes" (MR S6).
One Electron-specific input: this repo's client already has RateLimitError with Retry-After
parsing (client.ts:54-62, authenticatedFetch's 429 branch) and a client-wide rate-limit gate.
The engine's own JMAP layer (§10.1) must reproduce that behaviour rather than inherit it, since it
will not be using JMAPClient.
5. Multi-account isolation, account identity, lifecycle [adapted]
Requirement confirmed at program level (manual §4: an offline cache must isolate per account, including per-account keys). M§8 is the design; what changes is the identity plumbing, because this repo's account model differs (§1.4).
5.1 Namespacing [reused: M§8.1, adapted paths]
<VNCMAIL_DESKTOP_STORE_DIR>/
registry.json # ONLY: account ids present, purge tombstones,
# monotonic epochs, store-format markers (M§8.4.1)
accounts/<sha256(accountId)>.db # one SQLCipher file per account: mailbox, envelope,
# email_mailbox, body, body_queue, sync_state
Filenames are hashed, not username@host, so the directory listing is not a plaintext account
inventory on disk. No cursor, coverage row, record or resync flag lives outside an account's own
file — M§8.1's forward-compatibility requirement, which here is load-bearing on day one rather than
later, because §5.5's purge deletes the key and the file together and a cursor surviving that would
be advanced against a freshly-empty store.
registry.json is deliberately plaintext and M§8.1's accepted-limitation argument transfers with
one improvement: mobile's justification was that account-store already persists usernames to plain
AsyncStorage. Here, account-store.ts's persist (:219-227, name account-registry) already puts
username and email for every account into renderer localStorage, so the registry adds no new
exposure — and hashing the filenames means the registry is the only place the account list appears
in the store directory. It must be readable before any key exists (that is the whole point of
M§8.4.1's format marker), so it cannot itself be encrypted.
epoch lives in the registry, outside the per-account namespace, because it must be monotonic
across a purge (M§8.3). Owner: SyncStoreFactory. Not writable from SyncTxn; a transaction
reads it to validate itself and rejects with EpochMismatchError.
5.2 JMAP-level accounts within one login [reused: M§8.2, M§9.3]
Cursors and every SQL primary key carry jmap_account_id (MR S3: JMAP ids are unique only within
an account). v1 syncs the primary mail account only; delegated/shared accounts stay online-only,
as they effectively are today.
Note this repo already carries the same evidence mobile did: client.ts:388's
namespaceMailboxIds() prefixes ids when returning emails for a non-active account (five call sites:
:632, :1271, :2138, :2185, :2323). Same collision, same workaround, same conclusion —
account-scoped keys from day one.
5.3 Slot → account resolution [new]
The one piece of identity plumbing with no mobile counterpart, and the one most likely to produce a cross-account write.
app/api/offline/** routes are addressed the way every other authenticated route here is: by
?slot=N (§1.2). The resolution rule, in order, all steps required:
- Read
jmap_session[_slot];decryptSession⇒{serverUrl, username}. accountId = generateAccountId(username, serverUrl)— the server-confirmed username from the cookie, not a client-supplied one.- Open the store for
accountId. Never derive a path fromslot. Slots are recycled bygetNextCookieSlot(), so a stale slot number pointing at a re-added different account is an ordinary occurrence, not an edge case. - Cross-check against the JMAP session's own
username(client.ts:3822'sgetSessionUsername(), which exists precisely because a short login name may be canonicalized server-side — and whichAccountEntry.serverIdentifierswas added to handle). A mismatch is a hard error, not a best-effort match. - Every commit re-validates
(accountId, epoch)(I6), and every network call re-verifies that the engine's JMAP session still serves that account — not only at cycle start, because a cycle is long-lived. This is M§8.3's generalisation ofjmapClientServesActiveAccount, and it is what makes M's D6 (persisted cross-account contamination) unreachable here rather than merely unlikely.
5.4 Local mutations: not applicable in v1, and why that is a decision [new]
M§5.6 makes the outbox the sole durable record of local intent and composes it into reads via a pure
overlay.ts; M§5.6.1 then requires fixing the outbox's fire-and-forget persistence, because that
promotion made its durability load-bearing (V1).
None of that machinery exists here (§1.6): no outbox, no optimistic mutation queue, no
patchCache(). Mutations go straight to the server and fail when offline.
Decision: v1 desktop offline is read-only. The durable store holds server-derived state only — which is M§5.6's core property, reached by having no write path at all rather than by removing one. Consequences, stated so they are chosen rather than discovered:
- Marking a message read while offline does not work at all (rather than working locally and syncing later). That is today's behaviour; the engine does not regress it.
- M§5.6.2's two explicit non-coverages (unread badge counts read a server-maintained
Mailbox.unreadEmailsscalar and cannot be overlaid; SQL/FTS predicates see server truth) are moot in v1 and become live the moment an outbox is added. - When offline mutations are added later, M§5.6 and §5.6.1 are the design — including the
durability requirement. Do not invent a write-through into
envelope/body; that is the failure mode M removed rather than guarded (MR S11).
5.5 Logout, account removal, disable, purge [reused: M§8.4]
purgeAccount(accountId, reason: 'logout' | 'removed' | 'feature-disabled' | 'store-format-change'):
1. registry: { accountId, purgePending: true } # durable intent, crash-safe
2. epoch++ # in-flight commits now rejected
3. delete the SQLCipher key from safeStorage-protected key file -- FIRST
4. delete accounts/<hash>.db (+ -wal, -shm)
5. registry: remove the entry, KEEP the epoch
Ordering 3-before-4 is the security property: an interrupted purge must leave unreadable data.
Crash between 1 and 5 ⇒ the next launch completes the purge before any cycle starts. Triggers:
account-store.removeAccount, logout (single or all), the offline-cache setting being turned off
(MR S13 — purge, with a confirming Settings copy, since re-enabling costs a full bootstrap), and a
store-format/schema marker mismatch (M§8.4.1). AuthenticationError during a cycle is not a purge
signal — a server hiccup returning 401 must never delete a user's offline mail.
Lazy materialisation (M§9.5, MR S13) is if anything more important here than on mobile: read
paths check the setting for that account before calling open(), and open() on a
non-materialised account returns an empty read-only store and creates no file and no key. A user
who never enables offline mail must not end up with an encrypted database and a keychain entry
materialised by a read path.
6. Where the encryption key lives [new]
Mobile used expo-secure-store (OS-keychain backed). Electron's equivalent is safeStorage.
6.1 What safeStorage actually is, verified
Measured in Electron 43.2.0 on macOS (main process, after app.whenReady()):
isEncryptionAvailable() === true, encryptString/decryptString round-trip correct, ciphertext
prefixed v10 (Chromium's OSCrypt format). No Keychain prompt appeared.
Per Electron's documented behaviour (docs/latest/api/safe-storage):
- macOS: Keychain-backed. "Access to the system Keychain is required and these calls can block the current thread to collect user input."
- Windows: DPAPI; requires the
readyevent. - Linux:
isEncryptionAvailable()returns true even when no secret store exists, in which case items are "encrypted via hardcoded plaintext password" andgetSelectedStorageBackend()returnsbasic_text. Real backends aregnome_libsecret,kwallet/kwallet5/kwallet6;unknownmeans it was called beforeready.setUsePlainTextEncryption()forces an in-memory password on Linux and is a no-op elsewhere.
6.2 Decision: safeStorage, in the main process, with an explicit Linux gate
safeStorage (built in, no dependency) over keytar (unmaintained). Per-account key, generated
once as 32 random bytes, wrapped with safeStorage.encryptString() and written to
<store dir>/keys/<sha256(accountId)>.bin.
The awkward part, stated rather than hidden: safeStorage is a main-process API, and §2.4 put the
engine in the server process. Options, and the choice:
Give the server process its own key wrapping (e.g. a file with 0600 perms)— rejected: that is a key protected by nothing but filesystem permissions, i.e. materially weaker than the OS keychain the rest of the desktop ecosystem uses, and it silently discards the one thingsafeStoragebuys.- Chosen: the key crosses the existing IPC bridge, in one direction, once per account per app
launch.
main.tsgains a singleipcMain.handle("vnc:offline-key", …)-shaped path that unwraps the per-account key and hands it to the server process — not to the renderer. Mechanically this means main.ts fetches/creates+wraps the key and passes it to the standalone server over a small local channel established at spawn time (astdioextra fd, or a one-shot loopback request authenticated by a nonce also passed in the spawn env). The renderer is never in the path andwindow.vncgains nothing.
This is a real cost of choosing A over B — B would have had the key and the database in the same process — and it is the one place where B is genuinely simpler. It is outweighed by §2.1-for-1/2: moving the engine to main to co-locate the key would drag the credentials there too, which is a much larger secret-handling change (§2.2-against-1). Moving 32 bytes once per launch is the smaller of the two.
Mechanism is deliberately left open as an implementation choice between the extra-fd and nonce-authenticated-loopback variants; both are small, and Stage A should pick whichever proves cleaner against the packaged build. What is not open: the renderer must never see the key, and the key must never be written unwrapped.
6.3 Caveats to design around, not discover
- Linux
basic_textis the important one. On a Linux desktop with no keyring daemon — an AppImage on a minimal WM, a container, a headless CI box —isEncryptionAvailable()returns true while the key is protected by a hardcoded password that is public knowledge. That is worse than an honest failure, because it looks like it worked. Rule: at key-creation time,getSelectedStorageBackend()must be consulted, andbasic_textmust not silently proceed. Recommended behaviour: refuse to materialise a store, surface "offline mail can't be stored securely on this system (no OS keyring available)", and offer an explicit opt-in that records the downgrade. The decision on whether that opt-in exists at all is a human one (§13, open question 2). readyordering.safeStoragemust not be touched beforeapp.whenReady(), andgetSelectedStorageBackend()returnsunknownif it is.main.ts:205-208already does its work insidewhenReady().then(...), so the key path must sit there — and, since the server spawn happens insidecreateMainWindow(), the key must be resolved before or as part of the spawn.- macOS Keychain vs. unsigned builds — flagged, not resolved. Keychain ACLs are tied to app
identity. Builds are currently unsigned (
electron-builder.config.js,hardenedRuntime: false, VNCprodbuild step 9 open). Whether an ad-hoc-signed Electron app retains Keychain access across anelectron-updaterupgrade, or prompts, or silently loses the item — could not be verified in this environment and is not documented by Electron either way. The failure mode if it does lose access is not data loss but "offline mail must re-bootstrap after every update", which §5.5's purge-on-unreadable path handles gracefully. Stage A must test this on a real packaged build across a simulated update. It is also an argument for step 9 (signing) being a soft prerequisite for shipping the encrypted store to users, not merely a nice-to-have. - A lost key is a purge, never a prompt. If the wrapped key cannot be unwrapped, or the database
opens but
PRAGMA cipher_versionis empty, or the key fails (SQLITE_NOTADB), the response ispurgeAccount(..., 'store-format-change')and a fresh bootstrap. Never a "enter your password to recover" flow — the key was never derived from a user secret, so there is nothing to enter.
7. Storage interface and schema
7.1 Interface [reused: M§9.1]
SyncStore / SyncTxn / SyncStoreFactory exactly as M§9.1 defines them, including:
field-level state patches only and no whole-struct AccountSyncState write (I12, MR S1);
advanceCursor(key, next: ChangesState) and seedCursor(key, commitment);
putBodyIfEnvelopeExists (F48); enqueueBodies insert-or-ignore that never resets attempts
(MR S12, F41); listBodiesForEviction reading body.received_at from the body table alone;
listOrphanBodies; clearRecords() clearing records and the body queue while not nulling
cursors; loadAccountState() throwing CorruptStateError so the caller applies I13; the
StoreFormatMarker read/write pair and completePendingPurges() running once at launch before any
cycle.
The engine imports SyncStore and nothing else about persistence — no SQL, no binding import, no
path strings outside store*.ts. Two backends: store-sqlite.ts (@signalapp/sqlcipher) and
store-memory.ts (unit tests, and the second implementation that proves the boundary).
One Electron addition: SyncStoreFactory.open() must assert encryption positively —
PRAGMA cipher_version non-empty — and throw otherwise. §3.1's silent-PRAGMA key landmine makes
this the difference between an encrypted store and a plaintext one.
7.2 Backend notes [adapted: M§9.2]
M§9.2's staging question (AsyncStorage vs expo-sqlite, plain vs cipher) is closed here by §3.2:
one backend, encrypted, from the first commit. M's contingency section does not apply — there is no
key-value fallback worth building in a process that has a filesystem.
Concrete choices for this binding:
PRAGMA journal_mode = WALandsynchronous = NORMAL. WAL means the-wal/-shmsiblings must be included in every delete path (§5.5 step 4) — a classic leak.PRAGMA keyis set as the first statement after open, before any other statement, thencipher_versionis asserted (§7.1).- Synchronous API on a worker thread (§2.4), so a long
BEGIN…COMMITcannot stall an HTTP response. transaction()is a realBEGIN…COMMIT, so cursor-last (I1) is enforced by the database rather than by write ordering — the payoff M§9.2 predicted for shipping SQLite before the engine.void setItem(...).catch(warn)andtry { … } catch { /* ignore */ }around a store write are banned in the sync path (I4; §1.6's note aboutfile-store.ts).
7.3 Schema [reused: M§9.3]
M§9.3 verbatim: mailbox, envelope, email_mailbox, body, body_queue, sync_state, all
primary keys (jmap_account_id, id) per MR S3; envelope_received and envelope_nobody indexes
(the latter being job C2's driver); email_mailbox_by_mailbox; body.received_at present so
eviction is a single-table ordered scan (MR S12); deliberately no foreign keys and no cascades
(M§5.5's transient inconsistency is normal; a cascade on mailbox delete would delete mail, violating
I7).
One field to add for this repo: envelope.blob_id, since blobId is in this codebase's
EMAIL_LIST_PROPERTIES (§4.2).
sync_state living in the same file as the records is what makes §5.5's atomic wipe work.
7.4 What the renderer reads, and how
New routes under app/api/offline/, all gated per §2.4 and resolved per §5.3:
| Route | Backs |
|---|---|
GET /api/offline/emails?slot&mailboxId&limit&before |
the offline mailbox list (indexed queryEnvelopes) |
GET /api/offline/email/:id?slot |
a single cached message incl. body |
GET /api/offline/status?slot |
phase/progress/coverage/error for the UI |
POST /api/offline/sync?slot |
user-initiated "sync now" (coalesces, never aborts — M§10.3, D7) |
DELETE /api/offline/store?slot |
clear cache / purge (§5.5) |
The engine→UI direction only, per M§5.7: the engine notifies "account X changed"; the renderer decides whether to refresh. The engine never reads renderer state.
7.5 Reserved hooks [reused: M§9.4]
FTS5 (step 9) hangs off upsertEnvelopes / putBodyIfEnvelopeExists as the only write paths for
indexable content — no engine change. §3.4: FTS5 is compiled in. Attachment blobs are out of scope;
when added, their deletion belongs in deleteEmails and purge so they cannot leak past an account
wipe.
8. Triggering [adapted: M§10]
The trigger model is M's; the trigger set is not, because a desktop app has different lifecycle
events than a mobile one (no AppState backgrounding, no OS-governed background budget, but real
window minimise/hide, system sleep/wake, and a process that can outlive its window on macOS).
8.1 Triggers
| # | Trigger | Jobs | Throttle | vs. M |
|---|---|---|---|---|
| T1 | Server process ready + an account's credentials resolvable | A, B, C | 2 s delay | M T1 |
| T2 | Window shown / focused (BrowserWindow focus, via a small IPC ping) |
A, C | min 30 s since last cycle | M T2 (AppState → active) |
| T3 | User "sync now" (POST /api/offline/sync) |
A, B, C | none; coalesces into a running cycle, never aborts it | M T3, closes M's D7 |
| T4 | Network regained | A, C | 3 s debounce + per-account jitter | M T4 |
| T5 | StateChange on the engine's own WS/SSE connection (§2.5) |
A, C | 2 s debounce + M§10.4's state-equality check | M T5 |
| T6 | Retention setting changed | B (envelope widen), C2 (body widen), eviction only (narrow) | none | M T6 |
| T9 | Unfinished work: previous cycle partial, or any cursor drainPending, or coverage.phase ∈ {scanning, reconciling}, or a non-empty body queue |
the unfinished job(s) | 5 s, subject to §8.3's chaining rule | M T9 (MR S8) |
| T10 | Offline caching disabled for an account | abort + purge (§5.5) | none | M T10 (MR S13) |
| T11 | System resume from sleep (powerMonitor resume), and unlock-screen |
A, C | 5 s debounce; treat as network-uncertain, so T4's logic applies | [new] — no mobile analogue; a laptop lid closed for a day is the single most common way a desktop cursor gets far behind |
| T12 | App quit requested | none — cooperative stop | n/a | [new] — see §8.4 |
Explicitly not triggers: a periodic timer; opening a mailbox; opening a message; scrolling. The engine must never be on the critical path of a UI interaction (M§10.2) — if it is, its budgets and backoff become user-visible latency.
M's T8 (OS background refresh) has no counterpart: on desktop the process simply keeps running, so
partial+T9 covers it.
8.2 Budgets [adapted: M§6.4]
M's foreground/background split is replaced by a window-visible / window-hidden split. A hidden window on a plugged-in laptop is not the constrained environment a backgrounded phone is, so the hidden column is lower for politeness to the server and the user's battery, not because an OS will kill us:
| Bound | Window visible | Window hidden / minimised |
|---|---|---|
| Pages per cycle, per cursor | 40 | 20 |
| Wall clock per cycle | 90 s soft deadline, checked between pages | 60 s |
| Body queue items per cycle (C1+C2) | 200 | 100 |
| Coverage pages per cycle | 25 | 15 |
Exceeding a budget is a normal outcome (partial), not an error (M§6.4): the cursor stands at
the last committed page, drainPending stays true, T9 resumes. This is also the answer to a server
whose hasMoreChanges never goes false (F14).
8.3 Single-flight, coalescing, chaining [reused: M§10.3]
Per LocalAccountId: a second trigger during a cycle sets wakePending and awaits the same promise
— it never aborts (M's D7). Chained cycles continue only while madeProgress is true, so fixing
M's stall (MR S8) does not create a hot loop.
Abort triggers here: logout/purge, offline caching disabled (T10), the account being removed, network loss, a budget deadline, and app quit (T12). All leave a committed cursor and resumable state.
Cross-account: M is limited to the active account because jmapClient is a renderer singleton.
That constraint does not exist here — the engine constructs its own per-account JMAP layer from
per-slot credentials (§5.3), so it can sync all logged-in accounts, active or not, with a worker
per account. This is a genuine capability gain from choosing A, and one of the few places this design
is more capable than M. It is also a new load consideration: up to 5 accounts × (1 WS + delta
traffic) against one Stalwart. M§7.2's jitter is what keeps T4/T11 from producing a synchronised
stampede, and it becomes more important here than there.
8.4 Process lifetime [new]
M§10.5's headless-callability constraint holds trivially — the engine has no React, no store, no component dependency by construction (§2.4). Two Electron-specific rules:
main.tscurrently kills the server onwindow-all-closedandbefore-quit(:210-219) withserverProcess.kill()— SIGTERM, no coordination. A cycle dies mid-page. That is safe (I1: the cursor is the last fully-applied page; cost is one page's refetch) but wasteful, and it is worth T12: abefore-quitthat asks the engine to stop at the next page boundary, with a short timeout before falling through to the existing kill. Small change, and it must not be allowed to delay quit perceptibly.- macOS keeps the app alive with no windows.
window-all-closeddoes notapp.quit()on darwin (:210-215) yet does stop the server. So on macOS today, closing the window stops sync and reopening restarts it. Acceptable for v1; worth revisiting if "sync while closed" is ever wanted, because that is the only configuration where a desktop mail client can usefully sync with no UI.
9. Failure modes [reused + new rows]
M§11's table (F1–F49) applies in full and is not reproduced here. Every row is a JMAP-protocol or
engine-state scenario, and none of them changes because the host process changed. The ones most worth
re-reading before implementing: F1 (kill mid-drain), F3 (kill mid-bootstrap), F4 (kill mid-purge),
F9 (cannotCalculateChanges), F26 (updated for an id we don't hold), F37 (concurrent write vs.
commit), F38 (retention widened during reconcile — M's worst potential data-loss bug), F44
(clock jump), F47 (one cursor healthy, one wedged), F48 (body for a destroyed envelope).
Electron-specific additions:
| # | Scenario | Rule |
|---|---|---|
| E1 | Server child process SIGTERM'd on window close / quit (main.ts:210-219) |
Same class as M's F1: the cursor is the last fully-applied page (I1), drainPending survives, T1+T9 resume on next launch. Cost ≤1 page. T12 (§8.4) reduces it to ~0 but is not required for correctness. |
| E2 | Native module fails to load in the packaged build (NFT dropped prebuilds/, asar, wrong arch) |
Engine never constructs; /api/offline/** returns 404 exactly as in a hosted deployment; the app is fully functional online-only. Must never be a launch failure. This is also why Stage A verifies packaging before any engine code exists. |
| E3 | safeStorage reports basic_text (Linux, no keyring) |
Do not materialise a store. Surface "offline mail can't be stored securely here". Optional recorded opt-in (§6.3.1, §13 q2). Never silently encrypt with the public hardcoded password. |
| E4 | Wrapped key unwraps but the DB rejects it (SQLITE_NOTADB), or cipher_version is empty |
purgeAccount(..., 'store-format-change') + fresh bootstrap. Never a user-facing recovery prompt (§6.3.4). |
| E5 | Keychain item lost across an electron-updater upgrade of an unsigned build |
Same as E4 — re-bootstrap, one full sync. Cost is bandwidth, not data. Verify empirically (§12 Stage A); it is an argument for code signing. |
| E6 | The desktop marker env var is absent (hosted Docker deployment, or a dev next dev run) |
Engine module never constructed; routes 404. No SQLite file is created anywhere. The single most important non-failure in the document (§2.4). |
| E7 | Two app instances launched against the same userData |
Second instance's SQLite open fails or blocks on the WAL lock. Handle by requesting Electron's single-instance lock (app.requestSingleInstanceLock()) in main.ts — not currently requested, and worth doing on its own merits regardless of this engine. |
| E8 | Slot reused: account A removed, account B added into A's freed cookieSlot |
§5.3's resolve-by-cookie-then-verify-against-session makes this a no-op: B's cookie yields B's accountId, so B's store opens. A's store is already gone via §5.5's removeAccount purge. This row exists because resolving by slot would have been the natural shortcut and would have merged two accounts' mail. |
| E9 | Engine's WS connection succeeds while the renderer's fails (the expected steady state, §1.3) | Correct and intended. Renderer keeps SSE for its list; engine uses WS for its cursors; neither reads the other's state (§2.5 rule 3). No notification is fired by the engine (§2.5 rule 2). |
| E10 | Both connections wake on the same delivery | Both do their own work; the renderer refreshes the visible list, the engine advances its cursors. Duplicate fetches, never duplicate writes — they own disjoint state (M§5.7). |
| E11 | Engine and renderer both refresh an OAuth access token, and the server rotates refresh tokens (app/api/auth/token/route.ts:104-106) |
Real hazard. Two independent refreshers can invalidate each other's grant and log the user out. Rule: the engine never refreshes independently. It obtains tokens only through the existing PUT /api/auth/token route, in-process, so there is exactly one refresher and one rotation writer — the route. If that proves insufficient under concurrency, serialise it with a per-slot lock in the route itself. |
| E12 | Worker thread crashes (OOM on a huge body, native fault) | Cycle counts as failed, cursor unchanged (M§7.1 — a crash is not StateInvalid), worker respawned with backoff, escalation ladder applies via the cursor's counters. Never a purge. |
10. Required changes outside the engine
10.1 A server-side JMAP layer [new]
The engine cannot use lib/jmap/client.ts: it is a browser-fetch renderer class holding
credentials in memory (§1.2), and importing it server-side would drag the whole 7413-line surface
into the server bundle. It needs a small, focused JMAP client of its own under lib/sync/jmap/,
with only what M§12.1/§12.2 specify:
- Typed results, not
null-collapsing:JmapResult<T>,JmapMethodErrorwith M§12.1'sJmapMethodErrorTypeunion including'unknown'defaulting to ServerTransient. M's D5 is the bug that caused its D4; building the taxonomy in from the first commit is how it never exists here. getEmailChangesResult/getMailboxChangesResultreturning brandedChangesState, plusupdatedProperties: string[] | nullon the Mailbox result (RFC 8621 §2.2).getEmailProperties(ids, properties, accountId)returning itsstateas aSnapshotState, so M's D4 shape is a compile error rather than a code review question.getMailboxPropertiesfor theupdatedPropertiespatch path.queryEmailWindow({after, before, limit, sort, anchor, anchorOffset})for §4.6's keyset scan, surfacinganchorNotFounddistinctly.captureStates(accountId)— the one-requestMailbox/get{ids:[]}+Email/get{ids:[]}pair of §4.4, returning brandedSnapshotStates.- Request-level error parsing (RFC 8620 §3.6.1
application/problem+json,urn:…:error:limitwithlimit: maxSizeRequest | maxCallsInRequest | maxConcurrentRequests | rateLimit), anAbortSignal, and a per-request timeout. Noteclient.ts's plain-fetchpath has no timeout today, so a hung socket hangs a cycle — do not reproduce that. - A header-capable WebSocket (
ws) for §2.5 rule 1, withWebSocketPushEnable. RateLimitError+Retry-Afterhandling equivalent toclient.ts:54-62.
No as ChangesState / as SnapshotState cast may exist outside this layer's response parsers
(M§6.3). Worth an eslint no-restricted-syntax rule.
10.2 Settings
stores/settings-store.ts gains, per account: offlineCacheEnabled (default off, per the
program decision M records), offlineEnvelopeDays, offlineBodyDays, offlineMaxMB. Enabling and
disabling both need confirming copy — disabling purges (§5.5).
10.3 Electron shell
electron/main.ts: pass VNCMAIL_DESKTOP_STORE_DIR on spawn (§2.4); resolve/create the per-account
wrapped key after whenReady() and hand it to the server process (§6.2); add T11's powerMonitor
hooks and T12's cooperative before-quit; request the single-instance lock (E7). electron/preload.ts
gains nothing — the renderer talks to the engine over HTTP, not IPC.
10.4 UI
New: an offline-status surface (phase, coverage, last error, storage used, "sync now",
"clear offline mail") reading GET /api/offline/status; and an offline read path in the mail list /
message view that falls back to /api/offline/emails and /api/offline/email/:id when the JMAP
request fails and the account has a materialised store. stores/email-store.ts is the natural place
for the fallback, mirroring where the mobile app does it — and, per M§9.5, it must check
offlineCacheEnabled before calling anything that could materialise a store.
10.5 Packaging
serverExternalPackages: ['@signalapp/sqlcipher'] in next.config.ts; verification (and if needed a
copy step in scripts/assemble-standalone.mjs) that prebuilds/**/*.node reaches
.next/standalone/node_modules; a CI assertion that the packaged app can open an encrypted store on
every matrix OS.
11. Test plan
The [QA] gate. apply.ts being pure is what makes most of it cheap — that is why it is a hard
requirement (§4.1).
Unit, no network (vitest, already the repo's runner): M§13's unit list in full — every M§11 row
expressible as apply(localState, page, fetched) → mutations; the cursor state machine's eight
rules; classify() over the whole taxonomy including the unknown-type default; the escalation ladder
asserted monotonically non-increasing across a range of maxObjectsInGet including values below
250 and below 25 (MR V2); backoff monotonic/jittered/capped with Retry-After override; retention
F23/F23B/F24/F24B/F25 and the F44 clock-jump guard; reconcile floor pinning (F38) and the reconcile
ceiling (F39). Minus the outbox-durability tests, which have no subject here (§5.4).
Type-level, compiled by npm run typecheck (M§13's insistence that a type test only earns its
keep if a regression fails the build): advanceCursor rejects a SnapshotState; a plain object
literal is rejected where EnumerationCommitment is expected; no as ChangesState/as SnapshotState
cast exists outside the JMAP layer.
SyncStore contract tests against both backends (store-memory, store-sqlite), including
M's S1 lost-update sequence (F37) and clearRecords clearing the body queue (F35).
Encryption, new and non-negotiable (§3.1's landmine):
- After a write-and-close, the raw
.dbbytes contain no canary string and the header is notSQLite format 3. - Opening with a wrong key fails; with the right key succeeds.
open()throws ifPRAGMA cipher_versionis empty — i.e. the assertion of §7.1 actually fires if someone swaps in a non-cipher binding.- The format marker: mismatched/stale/absent ⇒
purgeAccount('store-format-change')at launch before any cycle; a crash between materialising a store and writing its marker leaves a mismatch (safe), not a false match (M§8.4.1).
Integration against real Stalwart — cheap here, unlike mobile. integration/docker-compose.yml
integration/tests/already exist in this repo with a real Stalwart, real SMTP injection and an Electron spec (§1.6). Extend with M§13's integration list, all of which apply:
- Bootstrap → deliver mail during the coverage scan → assert the first delta cycle picks it up. M calls this the highest-value test in the list and it is the §4.4 ordering test.
- Multi-page drain with
maxChangesforced to 2; kill the server child between pages; relaunch; assert convergence with no duplicates or omissions (F1/E1). - Flag toggle from a second client → assert the envelope's
keywordsupdate and no body refetch (a network assertion, not just a state assertion — this is the §4.5 3-property rule). - Mailbox delete with
onDestroyRemoveEmailsboth true and false (F7). - Force
cannotCalculateChanges→ assert reconcile runs, records stay readable throughout, delta keeps flowing during the enumeration (F49), and the sweep deletes exactly the server-absent ids. - Widen retention mid-reconcile → assert nothing in the gap is deleted (F38). M calls this the test for its worst potential data-loss bug.
- Two-account isolation, plus an explicit regression for M's D6: interleave account switching with
in-flight fetches, assert no row lands under the wrong account. Add E8: remove an account, add a
different one that lands in the freed
cookieSlot, assert no bleed. - Purge: kill mid-purge, relaunch, assert no records and no surviving cursor (F4/F22).
- E6, the hosted-deployment gate: boot the standalone server without the marker env var, hit
every
/api/offline/**route, assert 404 and assert no file was created anywhere. - E9/E10: with the engine's WS connection live, assert exactly one OS notification per delivery and that the renderer's path is the one that fired it.
Electron-level (npm run test:electron, the existing required CI gate): the packaged app opens
an encrypted store on each matrix OS (E2 negative case: a build with the binding deliberately
removed still launches and works online-only); the Linux runner asserts the basic_text refusal path
(E3) since a GitHub Linux runner has no keyring — a free, realistic test of the exact configuration
§6.3.1 is about.
Property/fuzz (M§13, cheap and high yield): generate random legal change pages with M§5.4's permitted overlaps and random kill points; assert the store converges to the same state as a from-scratch bootstrap.
12. Rollout, and the verify-first gate
M§14's shape, with M's own lesson applied: its V4 finding was that a whole staging decision rested on
an untested premise (expo-sqlite works in Expo Go). The premises here have been tested (§3.1) —
except the packaging ones, which cannot be tested without installing into this repo and building.
So Stage A exists for exactly those.
Stage A — packaging and key storage, before a line of engine code. All of it is verification; none of it is engine logic. If any item fails, the design changes before it is built, not after.
- Add
@signalapp/sqlcipher+serverExternalPackages. Runnpm run build:standaloneand assertprebuilds/<platform>-<arch>/@signalapp+sqlcipher.nodeis present under.next/standalone/node_modules. If NFT dropped it, add the copy step toassemble-standalone.mjs. - Open an encrypted database from an
app/api/**route in a packaged (--dir) build on macOS, and confirm the canary/header assertions of §11 against the real file. Repeat on Windows and Linux in CI. - Confirm cross-arch macOS packaging ships the right
.nodein each of the x64 and arm64 outputs (§3.3.4). - Resolve a
safeStorage-wrapped key inmain.tsand get it into the server process (§6.2); pick between the extra-fd and nonce-loopback variants on what actually proves cleaner. - On Linux, assert
getSelectedStorageBackend()and thatbasic_texttakes the refusal path (E3). - Simulate an
electron-updaterupgrade of an unsigned build and check Keychain access survives (E5, §6.3.3). This is the one item that could plausibly change the shipping plan — if an unsigned build loses its key on every update, the encrypted store should probably wait for step 9 (signing), and the human should be told so rather than shipping a store that re-bootstraps monthly.
Stage B — pure logic, no engine. states.ts (with M's Symbol() note), errors.ts, apply.ts,
retention.ts, fully unit-tested. Type-level tests wired into npm run typecheck.
Stage C — store. SyncStore + store-memory.ts + store-sqlite.ts + the format marker + the
contract tests against both backends + the §11 encryption tests.
Stage D — JMAP layer (§10.1), with the taxonomy and branded returns. Includes the header-capable
WebSocket, tested against the real fixture — this is where §2.5 rule 1 gets proven or disproven, and
if the fixture's /jmap/ws behaves like the sandbox's (§1.3) this is where we find out that
server-side WS works.
Stage E — cursors + delta drain (A1/A2). Stage F — coverage + bootstrap (B). Stage G — bodies (C1, C2). Stage H — triggers, routes, UI.
Feature flag: offlineCacheEnabled, default off, per account (§10.2). It gates route registration
and trigger registration, not just the engine body.
The Electron smoke gate (npm run test:electron) and the integration suite must stay green at every
stage.
13. Summary of key decisions
- The engine and the SQLite file live in the standalone Next.js server process (option A),
on a
worker_threadsWorker, not on the request loop. Decisive reasons: the per-account credentials are already there in httpOnly encrypted cookies (§1.2), so nothing secret crosses a process boundary; and a Node process can put anAuthorizationheader on a WebSocket upgrade, which is the exact thing that makes RFC 8887 push unreachable from the renderer today (client.ts:6038-6059). Option B was rejected because it can only be built by moving credentials into a process that currently holds none — the changeclient.tsexplicitly declined. Option C was rejected because WASM SQLite needs'wasm-unsafe-eval'added to the product-wide CSP (proxy.ts), and its only encrypted backends are small third-party WASM builds. - The renderer's push pipeline does not change. The engine gets its own header-capable
connection;
StateChangeis a wake signal and never a cursor (M§10.4); the engine never fires a notification —newEmailNotificationstays the single source; and the two cursors never read each other's state. Two sockets per account is the deliberate price of that isolation. Collapsing to one (renderer listening to the local server instead of Stalwart) is a follow-up, gated on the engine's connection being proven. - SQLCipher ships on day one, via
@signalapp/sqlcipher@4.0.3. Verified in this environment against Electron 43.2.0: N-API prebuilds load with no rebuild in both the main process andELECTRON_RUN_AS_NODE, real SQLCipher 4.10.0, encrypted file header, wrong key rejected, FTS5 present, AGPL-3.0-only matching this repo. The mobile design's plaintext-first phase existed solely because Expo Go cannot load SQLCipher; that constraint has no Electron analogue, and importing the staging anyway would mean shipping a plaintext mailbox for a phase plus building and discarding a migration path to leave it. node:sqliterejected (no encryption at any price —PRAGMA keyis a silent no-op that leaves the mailbox in cleartext; and stability 1.2/RC in Node 24, which is what Electron 43 bundles).better-sqlite3-multiple-ciphersrejected: newest release has Electron prebuilds up to ABI 146, Electron 43 needs 148, so it means a C++ toolchain on every machine — and that lag recurs at every Electron major by construction. Plainbetter-sqlite3@13.0.2is the fallback only.- Every store open asserts
PRAGMA cipher_versionis non-empty, and a test asserts a canary is absent from the raw file bytes. Silent-plaintext is the sharpest landmine found in this investigation and it has no error to notice. - The hosted-deployment gate is part of the design, not a convention. The same server process
runs in Docker for many users. One env var (
VNCMAIL_DESKTOP_STORE_DIR, set only bymain.ts) both enables the engine and supplies its path; the routes 404 without it; a test asserts no file is created without it (E6). - Keys:
safeStorage(built in, nokeytar), per account, wrapped and written under the store directory; the unwrapped key goes main → server process only, never to the renderer. ConsultinggetSelectedStorageBackend()is mandatory: Linux returnsisEncryptionAvailable() === truewhile using a public hardcoded password (basic_text), which is worse than an honest failure. - Account identity is
AccountEntry.id(username@host), nevercookieSlot. Slots are recycled; resolution is cookie →decryptSession→generateAccountId→ cross-check against the session's confirmed username, and every commit re-validates(accountId, epoch)(E8). - v1 desktop offline is read-only. This repo has no outbox and no optimistic mutation layer, so
M§5.6's read-time overlay has nothing to overlay. When offline mutations land, M§5.6/§5.6.1 is the
design — including its durability requirement — and a write-through into
envelope/bodyremains forbidden. - The engine syncs all logged-in accounts, not just the active one — a capability the mobile engine lacks because its JMAP client is a renderer singleton. Consequence: M§7.2's jitter matters more here, since T4/T11 fire for every account at once.
- Everything else is the mobile design, deliberately unchanged: the three state machines with
sequential execution (I11), independent envelope/body retention tiers, cursor provenance as an
ordering rule with branded types and an unforgeable
EnumerationCommitment, capture-cursors- before-enumerate, the 3-propertyupdatedfetch, cursor-last, the seven-class error taxonomy with exactly one class moving a cursor, the pinned reconcile sweep floor, the monotonically shrinkingmaxChangesladder with per-cursor counters, no-deletion-by-inference, account-scoped primary keys, the purge ordering (key before file), lazy materialisation, and the F1–F49 failure table.
Open questions for the human — resolved 2026-08-04
- If Stage A item 6 shows an unsigned build loses its Keychain item across updates — left open deliberately; this is an empirical Stage A outcome, not something to decide blind. Resolve once Stage A actually runs and reports what happens.
- Linux with no keyring (
basic_text, §6.3.1 / E3): refuse outright. Decided — matches this design's own "cheapest wrong answer" philosophy (§ discussion ofsafeStorage's silent-downgrade landmine): an honest failure beats a store that looks encrypted but isn't. No opt-in path in v1. - Concrete retention defaults: same as the mobile design's defaults. Decided —
offlineEnvelopeDays: 365,offlineBodyDays: 30, MB cap on bodies matching mobile's default (§ mobile M§2.1's shipped numbers). Consistent behavior across platforms for v1; revisit only if desktop usage patterns actually show it's too narrow — don't widen speculatively. - §2.5 single-socket follow-up: not scheduled for v1. Stays a gated follow-up as designed — don't collapse to one connection until the engine's own connection is proven in production.
- Stage A as its own PR ahead of the offline engine: not decided separately — proceed with the implementation sequence as designed (Stage A is the first step regardless of PR boundaries); the human can split the PR at merge time if desired, this doesn't block starting work.
Also decided: this design goes through an independent adversarial review next, matching the mobile engine's process (design → review → implementation), before any implementation code is written — same reasoning as before: this is the same class of highest-stakes, hardest-to-undo component (a persistent, encrypted, multi-account local store), and the review process caught real critical bugs in the mobile design that this document's own author-side verification did not.
None of these blocks starting the review.
14. What was verified, and what was not
Stated explicitly, in M's spirit — its V4 finding was precisely that an untested premise had been presented as settled.
Verified by execution in this environment, against electron@43.2.0: Electron's bundled Node
(24.18.0) and ABI (148/napi 10); its bundled SQLite (3.53.1, FTS5 on); node:sqlite's presence,
absence of encryption, and the silent no-op of PRAGMA key including the plaintext canary in the
file bytes; better-sqlite3@13.0.2's in-tarball N-API prebuilds and successful load in both process
modes; @signalapp/sqlcipher@4.0.3's load in Electron, SQLCipher 4.10.0, encrypted header, absent
canary, wrong-key rejection, right-key read-back, and FTS5; safeStorage.isEncryptionAvailable(),
round-trip and v10 ciphertext prefix on macOS.
Verified by reading published metadata: better-sqlite3 13's move off prebuild-install;
better-sqlite3-multiple-ciphers' Electron ABI coverage (121–146, no 148) and release cadence;
the existence and provenance of @7mind.io/sqlcipher-wasm and @aztec/sqlite3mc-wasm; node:sqlite's
stability index (1.2, RC) in Node 24; safeStorage's Linux basic_text fallback and
getSelectedStorageBackend() values.
NOT verified — flagged for Stage A, in descending order of how much they could change the design:
- Whether an unsigned Electron app retains its macOS Keychain item across an
electron-updaterupgrade (§6.3.3, E5). Not documented by Electron either way. Could change when the encrypted store ships. - Whether Next.js output file tracing carries
@signalapp/sqlcipher'sprebuilds/into.next/standalone/node_modules(§2.1-against-2, §3.3.2). Fallback is a copy step in a script that already exists for exactly this class of omission. - Whether the cross-arch macOS build ships the correct per-arch
.node(§3.3.4). - Whether the integration fixture's Stalwart
/jmap/wsaccepts a header-authenticated upgrade — the sandbox's does require the header (§1.3), which is what makes the server-side connection work in principle, but it has not been driven from Node here. Stage D. - The choice of mechanism for getting the unwrapped key from
main.tsinto the server process (extra fd vs. nonce-authenticated loopback) — deliberately left to whichever proves cleaner against a packaged build (§6.2).