Files
SRCmail/docs/VNCMAIL-NATIVE-BUILD-MANUAL.md

22 KiB
Raw Permalink Blame History

VNCmail+ Native & Desktop Client — Build Manual

Status: living document, last updated 2026-08-04. This is the canonical reference for the program that takes VNCmail+ (Bulwark) beyond the hosted webmail: an Electron desktop client, a React Native mobile client, and a self-hosted push relay, working toward true offline mail with an encrypted local index. It consolidates everything decided and built so far across three repositories, so nothing lives only in chat history or a session's memory.

Companion documents:

  • docs/OFFLINE-CLIENT-ARCHITECTURE.md (in ~/vncmail-plus) — the original gap analysis this program is based on.
  • ~/.claude/skills/VNCprodbuild/SKILL.md — the step-by-step build plan this manual reports progress against. That file is the operational checklist; this file is the narrative reference.

1. Why this program exists

Bulwark/VNCmail+ is a Next.js JMAP webmail app. As shipped, it has zero offline capability: the service worker caches nothing by design, there's no local mail store, no local search index, and no mobile or desktop native client. The goal of this program is to change that — ship a desktop app, a mobile app, real push notifications, and (eventually) a true offline-first local data layer with an encrypted search index — without re-deriving work that already exists upstream or duplicating effort across repos.

The single most important strategic fact discovered along the way: an upstream React Native mobile client already exists and already solves most of what looked like the hardest problems (auth, multi-account, device pairing, Android push). Building a second mobile client from scratch (e.g. wrapping the webmail in Capacitor) would have thrown that away for no reason. The whole shape of this program reflects that discovery — see §4.

2. Repository map

All three repos are AGPL-3.0 forks of the upstream Bulwark project (bulwarkmail on GitHub), owned by brvncde-dotcom:

Repo Forked from Purpose Local path
vncmail-plus bulwarkmail/webmail The Next.js webmail app itself — mail, calendar, contacts, files, admin, plugins. Deploys as a container on microk8s at vncmail.sandbox.vnc.de. ~/vncmail-plus (⚠️ shared checkout — see §8)
vncmail-native bulwarkmail/native React Native/Expo mobile client (Android + iOS). Beta/WIP upstream. ~/vncmail-native
vncmail-relay bulwarkmail/relay Push notification relay — terminates JMAP PushSubscription pushes, forwards to FCM (mobile) or Web Push (PWA/desktop). Self-hosted per the decision in §4. ~/vncmail-relay

The webmail's Electron desktop work happens in a dedicated worktree, not the shared checkout directly: ~/worktrees/vncmail-electron, branch claude/electron-desktop, based off vncmail-plus's dev branch. This will eventually become a PR into dev.

Backend: Stalwart Mail Server (stalwartlabs/mail-server), sandbox instance at stalwart.sandbox.vnc.de, speaking JMAP (mail/calendar/contacts), SMTP, IMAP, and ManageSieve.

3. Architecture recap

The core blocker for "true offline": lib/jmap/client.ts in the webmail is pure fetch(), zero Node dependencies — it's portable into any WebView, Electron renderer, or React Native context unchanged. But everything around it in the webmail — auth-cookie encryption (lib/auth/crypto.ts, uses Node's node:crypto), the push relay wiring, and every app/api/** route — is server-dependent. A native shell that just points a WebView at a bundled static export of the webmail won't work without either:

  • Option A — remote shell. The native wrapper loads the hosted URL. Fast, gets native push and an installable binary, but requires connectivity for every screen — not offline.
  • Option B — true offline-first. The client authenticates and syncs JMAP data directly against Stalwart, keeps a local encrypted store, and only needs connectivity to sync, not to read.

For desktop, this fork-in-the-road barely matters: Electron can bundle the webmail's own standalone Next.js server (the same artifact the Dockerfile already produces for the Docker image) inside its Node runtime and point a BrowserWindow at localhost. That's Option A and B at the same time, practically for free — see §5.

For mobile, the fork-in-the-road is real, which is why §4's discovery mattered so much: it meant Option A was already mostly done upstream, letting the plan skip straight to figuring out what Option B (the real offline engine) needs — instead of re-building Option A from scratch in Capacitor first.

4. Decision log

Every entry here was an explicit [DECISION] gate in the VNCprodbuild skill — resolved either by direct research/verification or by explicit user sign-off. Dates are when each was resolved.

Date Decision Resolution Why
2026-08-04 Does an offline cache need to support multiple accounts per device? Yes The webmail already has an account-registry store; the mobile offline cache must isolate per-account, including per-account SQLCipher keys later.
2026-08-04 Does an upstream React Native app already solve push/pairing? Yes — bulwarkmail/native has multi-account JMAP auth, full QR cross-device pairing, and Android FCM push via bulwarkmail/relay. Forked to vncmail-native. Duplicating working auth/pairing/push code in a fresh Capacitor wrapper has no upside. This flipped the entire mobile strategy — see §6.
2026-08-04 Self-host the push relay, or depend on upstream's shared hosted instance? Self-host. Forked bulwarkmail/relayvncmail-relay. Keeps push metadata (FCM tokens, timing) on VNC infrastructure rather than a third party's.
2026-08-04 Which SQLite library for the eventual SQLCipher-backed local index, and what Expo workflow? expo-sqlite's official useSQLCipher config-plugin option (verified via web search — Android/iOS/macOS support, docs), not a third-party binding. Stay Continuous Native Generation (don't commit ios/android, let expo prebuild regenerate them) rather than going fully bare. The official plugin already covers this. SQLCipher is unusable in Expo Go — day-to-day development necessarily moves to a custom dev client, which the user explicitly accepted. Going bare would turn every future merge from upstream bulwarkmail/native into a native-project merge conflict, for no offsetting benefit.
2026-08-04 Electron's background/foreground notification strategy: JMAP WebSocket push, polling, or quit-to-tray? JMAP WebSocket push, implemented with automatic SSE fallback — see caveat below. Confirmed live and available: the Stalwart sandbox's JMAP session resource advertises urn:ietf:params:jmap:websocket with supportsPush: true (wss://stalwart.sandbox.vnc.de/jmap/ws), independently confirmed by two separate build agents. Lower latency, no polling/backoff logic to write, and it's not hypothetical — it's live on the server this build already targets.
2026-08-04 Code-signing: Apple Developer ID? Windows cert? Apple: yes (also unblocks Phase 2's iOS push) — human must actually enroll, no agent can create the account or pay the ~$99/yr fee. Windows: yes, eventually — but ship unsigned for now during this build phase. Signing is a purchase/account-creation action, categorically outside what an agent can do.

5. Phase 1 — Electron desktop client

Location: ~/worktrees/vncmail-electron, branch claude/electron-desktop, 7+ commits ahead of dev as of this writing. Not pushed, no PR yet — review the worktree directly first.

What exists

  • electron/main.ts — boots the exact standalone Next.js server artifact the Dockerfile already produces, as a child process on a random localhost port; opens a BrowserWindow pointed at it. No parallel server-bundling approach was invented.
  • electron/preload.tscontextBridge exposing window.vnc.isElectron and window.vnc.showNotification(title, options), wired to Electron's native Notification API in the main process.
  • e2e/electron-smoke.spec.ts + playwright.electron.config.ts — the regression gate, using Playwright's _electron.launch(). Run via npm run test:electron. Verified green, including against a real packaged (--dir) build, not just the dev skeleton.
  • electron-builder.config.js + scripts/assemble-standalone.mjs + scripts/build-electron.mjs — packaging for macOS (dmg/zip, x64+arm64), Windows (nsis), Linux (AppImage/deb). Currently unsigned.
  • electron-updater wired against GitHub Releases (brvncde-dotcom/vncmail-plus), defensively wrapped so a failed update check never crashes the app.
  • .github/workflows/electron-build.yml — CI matrix (mac/win/linux) with npm run test:electron as a required gate before packaging/upload.

How to build and run it locally

cd ~/worktrees/vncmail-electron
npm install
npm run electron:dev          # dev loop against the local Next dev server
npm run build:standalone      # produces the standalone server artifact (same as Docker uses)
npm run build:electron        # packages via electron-builder (unsigned)
npm run test:electron         # the smoke-test regression gate

Two real bugs found and fixed while building this (worth knowing about)

  1. Repo-wide eslint gap. vnc/plugins/smime (an independent sub-package) was missing from the eslint ignore list, so npm run lint / the husky pre-commit hook failed for any commit touching that path on dev, regardless of what changed. Fixed alongside repos/**/ examples/**. Flag this for whoever reviews the eventual PR — it's a shared-config fix unrelated to Electron, worth landing on dev on its own merits.
  2. electron-builder extraResources footgun. electron-builder's resource-copy step unconditionally drops any directory literally named node_modules when copying extraResources — it was silently stripping the bundled standalone server's dependencies and crashing on launch with Cannot find module 'next'. Caught only because the build was actually launched and tested, not just configured. Worth remembering for any future electron-builder work generally, not just this project.

Still open

  • JMAP WebSocket push implementation (skill steps 6-7) — DONE. getWebSocketUrl() discovers the endpoint from the session's own capability object (never hardcoded), with exponential-jitter reconnect (200ms base / 5s cap / 3-attempt circuit breaker) and a 30s heartbeat, falling back to the existing SSE/polling chain on failure. A real end-to-end integration test (integration/tests/11-electron-notification.spec.ts) logs into the actual Stalwart docker fixture, injects mail over real SMTP, and asserts the native notification fires — not a mocked path. Two real bugs were found and fixed building this: production CSP blocked wss: outright (the feature was completely inert in any production build until fixed), and the original backoff timing had a window where a real delivery could be silently missed during a retry cycle. Caveat, found empirically against the real sandbox server: stalwart.sandbox.vnc.de's /jmap/ws endpoint requires the same HTTP Authorization header as every other JMAP endpoint on the WebSocket handshake itself — which the browser WebSocket API cannot attach (browsers don't allow custom headers on the handshake request). Against this specific server, the client will therefore always fail the WS handshake and fall back to SSE — correctly, by design, but it means "live WebSocket push" is currently unreachable in practice from a browser/Electron client, not just theoretically available. Fixing this for real would need a server-side accommodation (e.g. a short-lived token passed as a WS subprotocol or query parameter) — that's a Stalwart-side change, out of scope for this client work. Functionally nothing is broken (SSE fallback works), but don't expect WS to actually engage against this sandbox until that's addressed.
  • Code signing — blocked on the human actually enrolling in the Apple Developer Program (§4). Once done, wiring the signing identity + notarization into electron-builder and CI secrets is a config change, not a rewrite — the current config is structured for it.
  • App icon — using the 512×512 PWA icon as a stand-in. public/branding/Bulwark_Icon_App.svg should be rasterized at 1024×1024+ for a proper icon; no SVG rasterization tooling was available in-agent.
  • Internal dogfood gate — a human should install an unsigned build locally and sign off on UX before this goes any further (wider rollout, PR, etc.).

6. Phase 2 — Native mobile client + push relay

6.1 vncmail-native — what it already had vs. what this program added

Upstream bulwarkmail/native (forked as-is, no rewrite) already ships:

  • Multi-account JMAP sign-in against any server.
  • Full QR-code cross-device pairing (src/screens/LoginScreen.tsx, QrScanModal, redeemPairingCode/OAuth handoff in src/lib/oauth.ts).
  • Android push notifications via FCM, dispatched through bulwarkmail/relay.
  • A basic offline mail cache (src/lib/offline-sync.ts, src/stores/offline-cache-store.ts) — bulk-downloads the last N days of mail via Email/query+Email/get into AsyncStorage, with a size cap and eviction. Not the delta-sync/SQLCipher/FTS engine §7 describes — a periodic bulk re-download, not incremental sync, plain JSON not an encrypted database.
  • Android + iOS release pipelines already working (release-android.yml sideloads an APK from GitHub Releases; release-ios.yml + docs/ios-release.md ship to TestFlight) — iOS builds already work, just without push (Android-only so far per its own README).

This program's first pass (2026-08-04) added, without touching any of the above:

  • Verified npm install, typecheck, and the existing test suite all pass cleanly (429/430 tests; one pre-existing, unrelated transform failure in src/stores/__tests__/auth-store.test.ts, not introduced by this work — worth a look eventually, not urgent).
  • Confirmed live reachability to stalwart.sandbox.vnc.de (HTTP 307 → /jmap/session, valid JMAP session JSON returned) — and incidentally re-confirmed the WebSocket push capability from §4/§5.
  • Added .github/workflows/android-emulator-smoke.yml — builds the debug APK, boots a cached AVD via reactivecircus/android-emulator-runner, installs, launches the app, fails on process death or a FATAL EXCEPTION in logcat within a settle window.

How to build and run it locally

cd ~/vncmail-native
npm install
npx expo start          # Expo Go works fine UNTIL SQLCipher is added (§4) — see note below

Once SQLCipher work starts (§7): switch to a custom dev client — npx expo prebuild + npx expo run:android / npx expo run:ios, or an EAS development build. Expo Go cannot run an app with useSQLCipher enabled. Do not commit the generated ios/android directories — Continuous Native Generation regenerates them from config plugins on each build (§4).

6.2 vncmail-relay — self-hosted push relay

Forked as-is from bulwarkmail/relay. This program added:

  • deploy/k8s/{namespace,pvc,secret.example,deployment,service,ingress,kustomization}.yaml + deploy/k8s/README.md — mirrors the conventions already used to deploy vncmail-plus on microk8s (same namespace, same Recreate-strategy/PVC pattern). One deliberately unresolved item: the relay's own Dockerfile creates its runtime user via unpinned adduser -S (unlike vncmail-plus's documented uid 1001) — runAsUser/fsGroup are left unset in the manifest with instructions to verify against the real built image before first deploy, rather than guessing a UID.
  • .github/workflows/docker-publish.yml — publishes to ghcr.io/brvncde-dotcom/vncmail-relay, same multi-arch buildx/digest-merge structure as vncmail-plus's own publish workflow.
  • SETUP-VNC.md — documents a generated VAPID keypair (values are in that file only, referenced by name — not the actual secret — in secret.example.yaml's placeholders) and flags what's still human-owned before this can go live: a dedicated Firebase project + its FCM service-account JSON.

Not done, and deliberately so: no kubectl apply was run — there is no kubeconfig available in the build environment; deploying is a human-only action. The manifests and a full runbook are ready in deploy/k8s/README.md, waiting on:

  1. Create a dedicated Firebase project (not reusing vncmail-plus's or src-website's) and generate its service-account JSON.
  2. kubectl apply the manifests (with real secrets substituted for secret.example.yaml's placeholders) against the microk8s cluster.
  3. Once the relay is live and reachable (e.g. vncmail-relay.sandbox.vnc.de), repoint both vncmail-plus's DEFAULT_RELAY_BASE_URL and vncmail-native's equivalent relay base URL (check src/api/push.ts/src/lib/push-notifications.ts) at it instead of upstream's shared instance. Re-run the webmail's existing Web Push smoke path end-to-end against the new relay before treating it as the default.

7. Remaining roadmap (not yet started)

In rough order, per the VNCprodbuild skill:

  1. iOS push (vncmail-native) — blocked on the human's Apple Developer Program enrollment (§4/§5). vncmail-native already builds for iOS and ships via TestFlight; only push and client certs are missing.
  2. JMAP delta-sync engine — replace offline-sync.ts's bulk AsyncStorage download with a real Email/changes/Mailbox/changes cursor-based incremental sync. This is the highest-stakes step in the entire program — the skill calls for high/xhigh reasoning effort plus an independent, fresh-context agent adversarially reviewing the design before any implementation starts. Not yet begun.
  3. SQLCipher local store — swap AsyncStorage for expo-sqlite with useSQLCipher: true (§4), one isolated database/key per account (multi-account confirmed required, §4). Needs an explicit, security-sign-off decision on key derivation/lifecycle (from-password vs. device-random-key wrapped by biometric; wipe-on-logout) before implementation — do not let an agent default this silently.
  4. FTS5 search index — SQLite FTS5 population job tied to the sync engine above.
  5. Offline compose/outbox — queue composed messages while offline, replay via JMAP Email/set on reconnect, handle conflicts.
  6. Platform hardening — background refresh scheduling (BGTaskScheduler/WorkManager), Apple export-compliance declaration (ITSAppUsesNonExemptEncryption, triggered once SQLCipher ships in the iOS binary — an agent can draft the text, only a human can file it), Google Play Console account/signing key, final store submissions.
  7. Fix the webmail's own no-op service workerpublic/sw.js intentionally caches nothing today; adding Workbox-style precaching of the app shell is a cheap, independent improvement to the PWA's offline-shell behavior, unrelated to the native-client work above.

8. Known landmines

  • ~/vncmail-plus is a shared, actively-used checkout. Other sessions commit and switch branches there concurrently. An untracked file written directly into that checkout was lost mid-session to a concurrent branch switch — confirmed incident, 2026-08-04. Any work meant to persist must go into a dedicated worktree (like ~/worktrees/vncmail-electron) or be committed immediately — never leave meaningful uncommitted/untracked work sitting in the shared checkout.
  • ~/vncmail-native and ~/vncmail-relay are fresh clones (created 2026-08-04) with no confirmed concurrent-session activity yet — lower risk today, but don't assume that stays true as more work lands there.
  • electron-builder + node_modules — see §5's bug writeup; a general electron-builder landmine, not specific to this codebase.
  • Expo Go + SQLCipher are mutually exclusive — see §4/§6.1. The moment useSQLCipher is enabled, Expo Go can no longer run the app at all; this is a hard platform constraint, not a configuration bug to work around.
  • Electron's random localhost port breaks JMAP login against the sandbox Stalwart — deferred, not fixed, 2026-08-04. User confirmed testing the packaged Electron app directly against stalwart.sandbox.vnc.de (not localhost) hit a CORS-shaped login failure. Verified server-side: Stalwart's own CORS headers are correctly wildcarded (Access-Control-Allow-Origin: *) on every hop including the .well-known/jmap/jmap/session redirect — so this is not a Stalwart allow-list problem. Also found, separately: vncmail.sandbox.vnc.de (the documented deployed webmail domain) currently does not resolve (NXDOMAIN) — unrelated to this bug but worth knowing regardless. Leading theory, not yet confirmed against real browser devtools: electron/main.ts binds the bundled Next.js server via server.listen(0, ...) — a random OS-assigned port every launch — producing a different origin on every run; even if that origin were allow-listed once, it wouldn't stay valid. User explicitly said skip this for now — Electron packaging/building itself works, this only affects live login against the sandbox. Fix path when revisited: bind Electron's local server to a fixed port instead of 0.

9. Before merging any of this

None of the three repos' branches described here have been pushed or opened as a PR. Before that happens:

  • Run the full existing test/lint suites in each repo, not just the new smoke tests added here.
  • vncmail-plus has its own VERSION/CHANGELOG.md convention (currently 1.7.8) — a version bump belongs at actual release/merge time, not mid-feature-branch; this manual deliberately did not touch either file.
  • Cross-check the eslint-ignore fix (§5) lands even if the rest of the Electron work is split out or delayed — it's an independent, valuable fix on its own.