- JMAP_SERVER_URL: stalwart.sandbox.vnc.de → emailcore.src-advisory.com
- Updated Electron defaults, deploy secrets example, and e2e tests
- SMTP server (emailcore-svc.src-advisory.com) is handled by Stalwart
internally via JMAP EmailSubmission — no frontend changes needed
Root cause of "No local mail index available in this session" on a real
mailbox in the packaged .app, found by probing the live packaged build:
getSessionSecret() has four sources (env, env file, wizard config,
config file) and the desktop shell provided NONE — getDesktopDefaults()
sets JMAP_SERVER_URL (which also skips the setup wizard that would have
persisted a secret) but never a SESSION_SECRET. So every login's POST
/api/auth/stalwart-context 500'd, the jmap_stalwart_ctx cookie was never
minted, and every server-side-identity feature 401'd forever: encrypted
local index, offline replica, S/MIME enrolment, AI server class. The AI
retrieval leg renders any non-OK as "no local index", so the failure was
completely silent. Every test had masked this by injecting its own
SESSION_SECRET into the child env.
Fix 1 — electron/main.ts ensureSessionSecretFile(): a 64-hex-char secret
generated once per install, persisted 0600 under userData, handed to the
server as SESSION_SECRET_FILE (value stays out of the env block; an
operator-provided SESSION_SECRET env var still wins by resolution order).
Fix 2 — page.tsx boot catch-up now RETRIES (4s/20s/60s) instead of one
silent shot: the first attempt races login's own auth-context POST, and a
401 on that race used to mean an empty index until the next app restart.
requestIndex() already separates permanent (404/503 unavailable) from
retryable failures, so the retry is cheap and self-limiting.
Fix 3 — new components/ai/ai-ask-button.tsx: the AI Assistant finally has
an entry point in the MAIN mail view (Sparkles button next to the search
filter) opening a compact Ask dialog — same askMail client, same persisted
provider settings as the Settings pane. When nothing is configured it
deep-links to Settings → AI Assistant, where local-discovery's one-click
Connect does setup.
e2e hardened to prove the whole thing honestly: SESSION_SECRET explicitly
EMPTY in the launch env (the per-install secret must carry auth), the
manual sync/reindex calls removed (the automatic boot catch-up must build
the index on its own — polled, not triggered), and the toolbar entry
point asserted. Passing: auto-built index, discovery banner, Connect, and
a grounded answer citing the one email containing the fact.
Gate: tsc clean, eslint clean, 2502/2502 unit tests, e2e passing.
Found by a new real Electron e2e test built specifically to prove the
`local` AI class genuinely works end-to-end in the packaged desktop shell:
a real local Ollama model answering a real question, grounded in the real
encrypted SQLite/FTS5 mail index — not a browser tab, not a mock.
First run surfaced a genuine bug: toFtsMatchQuery() AND-joins every token,
which is right for a deliberate search-box query but wrong for the natural-
language questions the AI retrieval surface (/api/offline/search — see its
own module header, "THE RETRIEVAL SURFACE") actually receives. "When is
check-in for the Villa sul Lago booking, and what time?" shares almost none
of its own function words with the email that answers it, so ANDing every
token — including "when"/"is"/"for"/"the"/"and"/"what" — returned 0 hits
against an index that correctly returns the right email for "Villa sul Lago
check-in".
Fix: new toFtsMatchQueryAny() (lib/mail-index/store.ts) — drops a small,
well-known English stop-word list, OR-joins what's left, and lets the
existing bm25 ranking pick the winner among partial matches. Deliberately a
NEW function, not a change to toFtsMatchQuery itself: that one's own tests
rely on "AND"/"OR"/"NOT" surviving verbatim as literal search terms
(FTS5-keyword-injection safety) — a different guarantee than this one's job
of turning a question into a good search. search() gains a `mode: 'and' |
'any'` option (default 'and', so every existing caller is unaffected); the
offline-search route passes 'any', since its one real caller is exactly
this AI-question shape.
Also added, to make the e2e test possible at all: electron/main.ts's
VNCMAIL_TEST_FIXED_PORT — a narrow, off-by-default escape hatch so
DEV_MOCK_JMAP's JMAP_SERVER_URL can point at this same standalone server's
own /api/dev-jmap. Needed because the encrypted index's key channel
(fd-3/safeStorage) only gets wired up in startStandaloneServer()'s own
random-port launch path, never when ELECTRON_LOAD_URL bypasses it for a
plain `next dev` target — so this was the only way to exercise the real
index without a full Stalwart+SMTP Docker fixture.
Verified live in the real packaged Electron shell, not just unit tests:
real dev-mode login, real multi-round /api/offline/sync + /api/offline/reindex
(39 mail/35 calendar/23 contacts indexed), real local-discovery banner
(11 real Ollama models on this machine), real "Connect", a real question
through the real Settings UI, a real direct renderer->Ollama /api/chat call
(confirmed via network log, never proxied through this app's backend), and
the model's own answer citing the exact right fact: "Saturday 28 March at
15:00" — a fact that exists nowhere except in the one indexed email.
4 new unit tests for toFtsMatchQueryAny. Full gate: tsc clean, eslint
clean, 2502/2502 tests passing, build clean, e2e/electron-ai-local-index.spec.ts
passing against the real standalone server + real Electron + real Ollama.
electron-builder's default npmRebuild pass scans the entire node_modules
tree (not just what's actually packaged) for native addons and tries to
recompile them against Electron's ABI via node-gyp. It caught
@parcel/watcher - a transitive devDependency of some dev tool, never
shipped in this app - and hard-failed packaging on any machine without a
full Xcode Command Line Tools install ("gyp: No Xcode or CLT version
detected!"). GitHub's macOS runners happen to have Xcode, which is
presumably why CI never caught this.
The packaged app is plain esbuild-bundled JS with no native modules of
its own; the one native dependency in the repo (@signalapp/sqlcipher,
used by lib/mail-index/) ships prebuilt .node binaries for every
platform and is copied in wholesale by scripts/assemble-standalone.mjs,
never rebuilt by electron-builder. Verified by execution: packaging
failed with npmRebuild at its default (true), succeeded once set false,
and the resulting .dmg launches and runs correctly.
Also adds e2e/electron-live-sandbox.spec.ts - a live-connectivity check
against the real sandbox JMAP backend (stalwart.sandbox.vnc.de), proving
the packaged/launched app reaches it with no TLS/network errors and gets
a real structured auth-rejection on a deliberately fake credential.
Deliberately NOT wired into playwright.electron.config.ts's default
testMatch (electron-smoke.spec.ts only) - this depends on a live external
service and is a manual/opt-in verification tool, not part of the regular
regression suite.
Also carries the pre-existing lib/smime-ca/ejbca.ts no-control-regex
eslint fix from MR !1's branch (not yet merged to dev) so this commit's
own pre-commit hook passes - unrelated to electron work otherwise.
Phase 1 step 3 of VNCprodbuild. electron/preload.ts's contextBridge now
exposes window.vnc.showNotification(title, options), routed via
ipcRenderer.invoke("vnc:show-notification") to a new ipcMain.handle in
electron/main.ts that calls Electron's own Notification API. This is the
desktop shell's native notification path - it sits alongside, not in place
of, the browser/PWA's service-worker push path (public/sw.js's push/
notificationclick handlers + lib/web-push.ts), which is untouched.
lib/electron-bridge.ts gives the renderer a `isElectronShell()` +
`showElectronNotification()` wrapper so app code can detect the shell and
use the native path instead of/alongside SW push - not wired to any real
mail-delivery trigger yet, that's Phase 1 steps 4-6 (JMAP realtime
capability investigation, the background/foreground strategy decision, and
implementing it).
Extended e2e/electron-smoke.spec.ts to prove the IPC plumbing actually
fires end-to-end: calls window.vnc.showNotification from the renderer and
asserts the round-trip resolves (not that a real OS toast appears - not
observable in CI). Verified locally: the call resolves {"shown":true} on
this machine, confirming it genuinely reaches Electron's Notification API
and back, not just that window.vnc exists.
Also fixes a real bug caught by this step's typecheck: the smoke test's
Playwright Page variable was named `window`, shadowing the DOM global
inside every evaluate() callback and silently breaking their types. Renamed
to `appWindow`.
All 4 smoke-test assertions green: npm run build:electron && npm run
test:electron.
Phase 1 step 2 of VNCprodbuild. e2e/electron-smoke.spec.ts uses Playwright's
_electron.launch() to boot the real skeleton (dist-electron/main.js from
step 1) and asserts:
- the login screen renders (same input[type="text"]/[type="password"]
selectors as e2e/login.spec.ts's browser-based check)
- zero uncaught page errors fire during load
Sets JMAP_SERVER_URL (any non-empty value) so the app reaches
lib/setup/state.ts's "env-managed" state and serves the normal login screen
instead of 302ing to the first-run /setup wizard - no live mail server or
mock JMAP build flag needed just to prove the shell renders.
playwright.electron.config.ts is deliberately separate from
playwright.config.ts: it has no `webServer` block, since this suite's app
boots its own server and would otherwise race pointlessly with `npm run dev`
starting on :3000 for the browser-based e2e/*.spec.ts suite.
Wired as `npm run test:electron`. Verified green locally (2 passed) after
`npm run build:standalone && npm run build:electron`; every later step in
the Electron rollout must keep this passing before moving on.
Bernd Rodler