docs: extensive build manual for the native/desktop client program
Consolidates the repo map, architecture recap, full decision log, Phase 1/2 status, remaining roadmap, and known landmines into one canonical reference, so this doesn't live only in chat history or session memory.
This commit is contained in:
Bernd Rodler
@@ -0,0 +1,280 @@
|
||||
# VNCmail+ Native & Desktop Client — Build Manual
|
||||
|
||||
Status: living document, last updated 2026-08-04. This is the canonical reference for the
|
||||
program that takes VNCmail+ (Bulwark) beyond the hosted webmail: an Electron desktop client, a
|
||||
React Native mobile client, and a self-hosted push relay, working toward true offline mail with
|
||||
an encrypted local index. It consolidates everything decided and built so far across three
|
||||
repositories, so nothing lives only in chat history or a session's memory.
|
||||
|
||||
Companion documents:
|
||||
- `docs/OFFLINE-CLIENT-ARCHITECTURE.md` (in `~/vncmail-plus`) — the original gap analysis this
|
||||
program is based on.
|
||||
- `~/.claude/skills/VNCprodbuild/SKILL.md` — the step-by-step build plan this manual reports
|
||||
progress against. That file is the operational checklist; this file is the narrative reference.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this program exists
|
||||
|
||||
Bulwark/VNCmail+ is a Next.js JMAP webmail app. As shipped, it has zero offline capability: the
|
||||
service worker caches nothing by design, there's no local mail store, no local search index, and
|
||||
no mobile or desktop native client. The goal of this program is to change that — ship a desktop
|
||||
app, a mobile app, real push notifications, and (eventually) a true offline-first local data
|
||||
layer with an encrypted search index — without re-deriving work that already exists upstream or
|
||||
duplicating effort across repos.
|
||||
|
||||
The single most important strategic fact discovered along the way: **an upstream React Native
|
||||
mobile client already exists and already solves most of what looked like the hardest problems**
|
||||
(auth, multi-account, device pairing, Android push). Building a second mobile client from
|
||||
scratch (e.g. wrapping the webmail in Capacitor) would have thrown that away for no reason. The
|
||||
whole shape of this program reflects that discovery — see §4.
|
||||
|
||||
## 2. Repository map
|
||||
|
||||
All three repos are AGPL-3.0 forks of the upstream Bulwark project (`bulwarkmail` on GitHub),
|
||||
owned by `brvncde-dotcom`:
|
||||
|
||||
| Repo | Forked from | Purpose | Local path |
|
||||
|---|---|---|---|
|
||||
| `vncmail-plus` | `bulwarkmail/webmail` | The Next.js webmail app itself — mail, calendar, contacts, files, admin, plugins. Deploys as a container on microk8s at `vncmail.sandbox.vnc.de`. | `~/vncmail-plus` (⚠️ shared checkout — see §8) |
|
||||
| `vncmail-native` | `bulwarkmail/native` | React Native/Expo mobile client (Android + iOS). Beta/WIP upstream. | `~/vncmail-native` |
|
||||
| `vncmail-relay` | `bulwarkmail/relay` | Push notification relay — terminates JMAP `PushSubscription` pushes, forwards to FCM (mobile) or Web Push (PWA/desktop). Self-hosted per the decision in §4. | `~/vncmail-relay` |
|
||||
|
||||
The webmail's Electron desktop work happens in a **dedicated worktree**, not the shared
|
||||
checkout directly: `~/worktrees/vncmail-electron`, branch `claude/electron-desktop`, based off
|
||||
`vncmail-plus`'s `dev` branch. This will eventually become a PR into `dev`.
|
||||
|
||||
Backend: Stalwart Mail Server (`stalwartlabs/mail-server`), sandbox instance at
|
||||
`stalwart.sandbox.vnc.de`, speaking JMAP (mail/calendar/contacts), SMTP, IMAP, and ManageSieve.
|
||||
|
||||
## 3. Architecture recap
|
||||
|
||||
**The core blocker for "true offline":** `lib/jmap/client.ts` in the webmail is pure `fetch()`,
|
||||
zero Node dependencies — it's portable into any WebView, Electron renderer, or React Native
|
||||
context unchanged. But everything *around* it in the webmail — auth-cookie encryption
|
||||
(`lib/auth/crypto.ts`, uses Node's `node:crypto`), the push relay wiring, and every `app/api/**`
|
||||
route — is server-dependent. A native shell that just points a WebView at a bundled static
|
||||
export of the webmail won't work without either:
|
||||
|
||||
- **Option A — remote shell.** The native wrapper loads the *hosted* URL. Fast, gets native push
|
||||
and an installable binary, but requires connectivity for every screen — not offline.
|
||||
- **Option B — true offline-first.** The client authenticates and syncs JMAP data directly against
|
||||
Stalwart, keeps a local encrypted store, and only needs connectivity to *sync*, not to *read*.
|
||||
|
||||
For **desktop**, this fork-in-the-road barely matters: Electron can bundle the webmail's own
|
||||
standalone Next.js server (the same artifact the `Dockerfile` already produces for the Docker
|
||||
image) inside its Node runtime and point a `BrowserWindow` at `localhost`. That's Option A and B
|
||||
at the same time, practically for free — see §5.
|
||||
|
||||
For **mobile**, the fork-in-the-road is real, which is why §4's discovery mattered so much: it
|
||||
meant Option A was already mostly done upstream, letting the plan skip straight to figuring out
|
||||
what Option B (the real offline engine) needs — instead of re-building Option A from scratch in
|
||||
Capacitor first.
|
||||
|
||||
## 4. Decision log
|
||||
|
||||
Every entry here was an explicit `[DECISION]` gate in the `VNCprodbuild` skill — resolved either
|
||||
by direct research/verification or by explicit user sign-off. Dates are when each was resolved.
|
||||
|
||||
| Date | Decision | Resolution | Why |
|
||||
|---|---|---|---|
|
||||
| 2026-08-04 | Does an offline cache need to support multiple accounts per device? | **Yes** | The webmail already has an `account-registry` store; the mobile offline cache must isolate per-account, including per-account SQLCipher keys later. |
|
||||
| 2026-08-04 | Does an upstream React Native app already solve push/pairing? | **Yes — `bulwarkmail/native`** has multi-account JMAP auth, full QR cross-device pairing, and Android FCM push via `bulwarkmail/relay`. Forked to `vncmail-native`. | Duplicating working auth/pairing/push code in a fresh Capacitor wrapper has no upside. **This flipped the entire mobile strategy** — see §6. |
|
||||
| 2026-08-04 | Self-host the push relay, or depend on upstream's shared hosted instance? | **Self-host.** Forked `bulwarkmail/relay` → `vncmail-relay`. | Keeps push metadata (FCM tokens, timing) on VNC infrastructure rather than a third party's. |
|
||||
| 2026-08-04 | Which SQLite library for the eventual SQLCipher-backed local index, and what Expo workflow? | **`expo-sqlite`'s official `useSQLCipher` config-plugin option** (verified via web search — Android/iOS/macOS support, [docs](https://docs.expo.dev/versions/latest/sdk/sqlite/)), not a third-party binding. **Stay Continuous Native Generation** (don't commit `ios`/`android`, let `expo prebuild` regenerate them) rather than going fully bare. | The official plugin already covers this. SQLCipher is unusable in Expo Go — day-to-day development necessarily moves to a custom dev client, which the user explicitly accepted. Going bare would turn every future merge from upstream `bulwarkmail/native` into a native-project merge conflict, for no offsetting benefit. |
|
||||
| 2026-08-04 | Electron's background/foreground notification strategy: JMAP WebSocket push, polling, or quit-to-tray? | **JMAP WebSocket push.** Confirmed live and available: the Stalwart sandbox's JMAP session resource advertises `urn:ietf:params:jmap:websocket` with `supportsPush: true` (`wss://stalwart.sandbox.vnc.de/jmap/ws`), independently confirmed by two separate build agents. | Lower latency, no polling/backoff logic to write, and it's not hypothetical — it's live on the server this build already targets. |
|
||||
| 2026-08-04 | Code-signing: Apple Developer ID? Windows cert? | **Apple: yes** (also unblocks Phase 2's iOS push) — **human must actually enroll**, no agent can create the account or pay the ~$99/yr fee. **Windows: yes, eventually** — but ship unsigned for now during this build phase. | Signing is a purchase/account-creation action, categorically outside what an agent can do. |
|
||||
|
||||
## 5. Phase 1 — Electron desktop client
|
||||
|
||||
**Location:** `~/worktrees/vncmail-electron`, branch `claude/electron-desktop`, 7+ commits ahead
|
||||
of `dev` as of this writing. Not pushed, no PR yet — review the worktree directly first.
|
||||
|
||||
### What exists
|
||||
|
||||
- `electron/main.ts` — boots the exact standalone Next.js server artifact the `Dockerfile`
|
||||
already produces, as a child process on a random localhost port; opens a `BrowserWindow`
|
||||
pointed at it. No parallel server-bundling approach was invented.
|
||||
- `electron/preload.ts` — `contextBridge` exposing `window.vnc.isElectron` and
|
||||
`window.vnc.showNotification(title, options)`, wired to Electron's native `Notification` API in
|
||||
the main process.
|
||||
- `e2e/electron-smoke.spec.ts` + `playwright.electron.config.ts` — the regression gate, using
|
||||
Playwright's `_electron.launch()`. Run via `npm run test:electron`. Verified green, including
|
||||
against a real packaged (`--dir`) build, not just the dev skeleton.
|
||||
- `electron-builder.config.js` + `scripts/assemble-standalone.mjs` + `scripts/build-electron.mjs`
|
||||
— packaging for macOS (`dmg`/`zip`, x64+arm64), Windows (`nsis`), Linux (`AppImage`/`deb`).
|
||||
Currently unsigned.
|
||||
- `electron-updater` wired against GitHub Releases (`brvncde-dotcom/vncmail-plus`), defensively
|
||||
wrapped so a failed update check never crashes the app.
|
||||
- `.github/workflows/electron-build.yml` — CI matrix (mac/win/linux) with `npm run test:electron`
|
||||
as a required gate before packaging/upload.
|
||||
|
||||
### How to build and run it locally
|
||||
|
||||
```bash
|
||||
cd ~/worktrees/vncmail-electron
|
||||
npm install
|
||||
npm run electron:dev # dev loop against the local Next dev server
|
||||
npm run build:standalone # produces the standalone server artifact (same as Docker uses)
|
||||
npm run build:electron # packages via electron-builder (unsigned)
|
||||
npm run test:electron # the smoke-test regression gate
|
||||
```
|
||||
|
||||
### Two real bugs found and fixed while building this (worth knowing about)
|
||||
|
||||
1. **Repo-wide eslint gap.** `vnc/plugins/smime` (an independent sub-package) was missing from
|
||||
the eslint ignore list, so `npm run lint` / the husky pre-commit hook failed for *any* commit
|
||||
touching that path on `dev`, regardless of what changed. Fixed alongside `repos/**`/
|
||||
`examples/**`. **Flag this for whoever reviews the eventual PR** — it's a shared-config fix
|
||||
unrelated to Electron, worth landing on `dev` on its own merits.
|
||||
2. **electron-builder `extraResources` footgun.** electron-builder's resource-copy step
|
||||
unconditionally drops any directory literally named `node_modules` when copying
|
||||
`extraResources` — it was silently stripping the bundled standalone server's dependencies and
|
||||
crashing on launch with `Cannot find module 'next'`. Caught only because the build was
|
||||
actually launched and tested, not just configured. Worth remembering for any future
|
||||
electron-builder work generally, not just this project.
|
||||
|
||||
### Still open
|
||||
|
||||
- **JMAP WebSocket push implementation** (skill steps 6-7) — decision resolved (§4), build not
|
||||
yet done as of this manual's last update; check the skill's status log or task tracker for
|
||||
current state.
|
||||
- **Code signing** — blocked on the human actually enrolling in the Apple Developer Program
|
||||
(§4). Once done, wiring the signing identity + notarization into `electron-builder` and CI
|
||||
secrets is a config change, not a rewrite — the current config is structured for it.
|
||||
- **App icon** — using the 512×512 PWA icon as a stand-in.
|
||||
`public/branding/Bulwark_Icon_App.svg` should be rasterized at 1024×1024+ for a proper icon; no
|
||||
SVG rasterization tooling was available in-agent.
|
||||
- **Internal dogfood gate** — a human should install an unsigned build locally and sign off on
|
||||
UX before this goes any further (wider rollout, PR, etc.).
|
||||
|
||||
## 6. Phase 2 — Native mobile client + push relay
|
||||
|
||||
### 6.1 `vncmail-native` — what it already had vs. what this program added
|
||||
|
||||
Upstream `bulwarkmail/native` (forked as-is, no rewrite) already ships:
|
||||
|
||||
- Multi-account JMAP sign-in against any server.
|
||||
- Full QR-code cross-device pairing (`src/screens/LoginScreen.tsx`, `QrScanModal`,
|
||||
`redeemPairingCode`/OAuth handoff in `src/lib/oauth.ts`).
|
||||
- Android push notifications via FCM, dispatched through `bulwarkmail/relay`.
|
||||
- A basic offline mail cache (`src/lib/offline-sync.ts`, `src/stores/offline-cache-store.ts`) —
|
||||
bulk-downloads the last N days of mail via `Email/query`+`Email/get` into AsyncStorage, with a
|
||||
size cap and eviction. **Not** the delta-sync/SQLCipher/FTS engine §7 describes — a periodic
|
||||
bulk re-download, not incremental sync, plain JSON not an encrypted database.
|
||||
- Android + iOS release pipelines already working (`release-android.yml` sideloads an APK from
|
||||
GitHub Releases; `release-ios.yml` + `docs/ios-release.md` ship to TestFlight) — iOS *builds*
|
||||
already work, just without push (Android-only so far per its own README).
|
||||
|
||||
This program's first pass (2026-08-04) added, without touching any of the above:
|
||||
- Verified `npm install`, typecheck, and the existing test suite all pass cleanly (429/430 tests;
|
||||
one pre-existing, unrelated transform failure in `src/stores/__tests__/auth-store.test.ts`, not
|
||||
introduced by this work — worth a look eventually, not urgent).
|
||||
- Confirmed live reachability to `stalwart.sandbox.vnc.de` (HTTP 307 → `/jmap/session`, valid
|
||||
JMAP session JSON returned) — and incidentally re-confirmed the WebSocket push capability from
|
||||
§4/§5.
|
||||
- Added `.github/workflows/android-emulator-smoke.yml` — builds the debug APK, boots a cached
|
||||
AVD via `reactivecircus/android-emulator-runner`, installs, launches the app, fails on process
|
||||
death or a `FATAL EXCEPTION` in logcat within a settle window.
|
||||
|
||||
### How to build and run it locally
|
||||
|
||||
```bash
|
||||
cd ~/vncmail-native
|
||||
npm install
|
||||
npx expo start # Expo Go works fine UNTIL SQLCipher is added (§4) — see note below
|
||||
```
|
||||
|
||||
**Once SQLCipher work starts (§7):** switch to a custom dev client — `npx expo prebuild` +
|
||||
`npx expo run:android` / `npx expo run:ios`, or an EAS development build. Expo Go cannot run an
|
||||
app with `useSQLCipher` enabled. Do not commit the generated `ios`/`android` directories —
|
||||
Continuous Native Generation regenerates them from config plugins on each build (§4).
|
||||
|
||||
### 6.2 `vncmail-relay` — self-hosted push relay
|
||||
|
||||
Forked as-is from `bulwarkmail/relay`. This program added:
|
||||
|
||||
- `deploy/k8s/{namespace,pvc,secret.example,deployment,service,ingress,kustomization}.yaml` +
|
||||
`deploy/k8s/README.md` — mirrors the conventions already used to deploy `vncmail-plus` on
|
||||
microk8s (same namespace, same Recreate-strategy/PVC pattern). **One deliberately unresolved
|
||||
item:** the relay's own Dockerfile creates its runtime user via unpinned `adduser -S` (unlike
|
||||
`vncmail-plus`'s documented uid 1001) — `runAsUser`/`fsGroup` are left unset in the manifest
|
||||
with instructions to verify against the real built image before first deploy, rather than
|
||||
guessing a UID.
|
||||
- `.github/workflows/docker-publish.yml` — publishes to `ghcr.io/brvncde-dotcom/vncmail-relay`,
|
||||
same multi-arch buildx/digest-merge structure as `vncmail-plus`'s own publish workflow.
|
||||
- `SETUP-VNC.md` — documents a generated VAPID keypair (values are in that file only, referenced
|
||||
by name — not the actual secret — in `secret.example.yaml`'s placeholders) and flags what's
|
||||
still human-owned before this can go live: a dedicated Firebase project + its FCM
|
||||
service-account JSON.
|
||||
|
||||
**Not done, and deliberately so:** no `kubectl apply` was run — there is no kubeconfig available
|
||||
in the build environment; deploying is a human-only action. The manifests and a full runbook are
|
||||
ready in `deploy/k8s/README.md`, waiting on:
|
||||
|
||||
1. Create a dedicated Firebase project (not reusing `vncmail-plus`'s or `src-website`'s) and
|
||||
generate its service-account JSON.
|
||||
2. `kubectl apply` the manifests (with real secrets substituted for `secret.example.yaml`'s
|
||||
placeholders) against the microk8s cluster.
|
||||
3. Once the relay is live and reachable (e.g. `vncmail-relay.sandbox.vnc.de`), repoint both
|
||||
`vncmail-plus`'s `DEFAULT_RELAY_BASE_URL` and `vncmail-native`'s equivalent relay base URL
|
||||
(check `src/api/push.ts`/`src/lib/push-notifications.ts`) at it instead of upstream's shared
|
||||
instance. Re-run the webmail's existing Web Push smoke path end-to-end against the new relay
|
||||
before treating it as the default.
|
||||
|
||||
## 7. Remaining roadmap (not yet started)
|
||||
|
||||
In rough order, per the `VNCprodbuild` skill:
|
||||
|
||||
1. **iOS push (`vncmail-native`)** — blocked on the human's Apple Developer Program enrollment
|
||||
(§4/§5). `vncmail-native` already builds for iOS and ships via TestFlight; only push and
|
||||
client certs are missing.
|
||||
2. **JMAP delta-sync engine** — replace `offline-sync.ts`'s bulk AsyncStorage download with a
|
||||
real `Email/changes`/`Mailbox/changes` cursor-based incremental sync. **This is the
|
||||
highest-stakes step in the entire program** — the skill calls for high/xhigh reasoning effort
|
||||
plus an independent, fresh-context agent adversarially reviewing the design before any
|
||||
implementation starts. Not yet begun.
|
||||
3. **SQLCipher local store** — swap AsyncStorage for `expo-sqlite` with `useSQLCipher: true`
|
||||
(§4), one isolated database/key per account (multi-account confirmed required, §4). Needs an
|
||||
explicit, security-sign-off decision on key derivation/lifecycle (from-password vs.
|
||||
device-random-key wrapped by biometric; wipe-on-logout) before implementation — do not let an
|
||||
agent default this silently.
|
||||
4. **FTS5 search index** — SQLite FTS5 population job tied to the sync engine above.
|
||||
5. **Offline compose/outbox** — queue composed messages while offline, replay via JMAP
|
||||
`Email/set` on reconnect, handle conflicts.
|
||||
6. **Platform hardening** — background refresh scheduling (`BGTaskScheduler`/`WorkManager`),
|
||||
Apple export-compliance declaration (`ITSAppUsesNonExemptEncryption`, triggered once SQLCipher
|
||||
ships in the iOS binary — an agent can draft the text, only a human can file it), Google Play
|
||||
Console account/signing key, final store submissions.
|
||||
7. **Fix the webmail's own no-op service worker** — `public/sw.js` intentionally caches nothing
|
||||
today; adding Workbox-style precaching of the app shell is a cheap, independent improvement to
|
||||
the PWA's offline-shell behavior, unrelated to the native-client work above.
|
||||
|
||||
## 8. Known landmines
|
||||
|
||||
- **`~/vncmail-plus` is a shared, actively-used checkout.** Other sessions commit and switch
|
||||
branches there concurrently. An untracked file written directly into that checkout was lost
|
||||
mid-session to a concurrent branch switch — confirmed incident, 2026-08-04. **Any work meant
|
||||
to persist must go into a dedicated worktree (like `~/worktrees/vncmail-electron`) or be
|
||||
committed immediately** — never leave meaningful uncommitted/untracked work sitting in the
|
||||
shared checkout.
|
||||
- **`~/vncmail-native` and `~/vncmail-relay` are fresh clones** (created 2026-08-04) with no
|
||||
confirmed concurrent-session activity yet — lower risk today, but don't assume that stays true
|
||||
as more work lands there.
|
||||
- **electron-builder + `node_modules`** — see §5's bug writeup; a general electron-builder
|
||||
landmine, not specific to this codebase.
|
||||
- **Expo Go + SQLCipher are mutually exclusive** — see §4/§6.1. The moment `useSQLCipher` is
|
||||
enabled, Expo Go can no longer run the app at all; this is a hard platform constraint, not a
|
||||
configuration bug to work around.
|
||||
|
||||
## 9. Before merging any of this
|
||||
|
||||
None of the three repos' branches described here have been pushed or opened as a PR. Before
|
||||
that happens:
|
||||
|
||||
- Run the full existing test/lint suites in each repo, not just the new smoke tests added here.
|
||||
- `vncmail-plus` has its own `VERSION`/`CHANGELOG.md` convention (currently `1.7.8`) — a version
|
||||
bump belongs at actual release/merge time, not mid-feature-branch; this manual deliberately
|
||||
did not touch either file.
|
||||
- Cross-check the eslint-ignore fix (§5) lands even if the rest of the Electron work is split out
|
||||
or delayed — it's an independent, valuable fix on its own.
|
||||
Reference in New Issue
Block a user