Files
SRCmail/electron/main.ts
T
Bernd Rodler 7b2047681e fix(mail-index): local AI retrieval returned 0 hits for real questions — AND-every-token FTS matching killed on stop words
Found by a new real Electron e2e test built specifically to prove the
`local` AI class genuinely works end-to-end in the packaged desktop shell:
a real local Ollama model answering a real question, grounded in the real
encrypted SQLite/FTS5 mail index — not a browser tab, not a mock.

First run surfaced a genuine bug: toFtsMatchQuery() AND-joins every token,
which is right for a deliberate search-box query but wrong for the natural-
language questions the AI retrieval surface (/api/offline/search — see its
own module header, "THE RETRIEVAL SURFACE") actually receives. "When is
check-in for the Villa sul Lago booking, and what time?" shares almost none
of its own function words with the email that answers it, so ANDing every
token — including "when"/"is"/"for"/"the"/"and"/"what" — returned 0 hits
against an index that correctly returns the right email for "Villa sul Lago
check-in".

Fix: new toFtsMatchQueryAny() (lib/mail-index/store.ts) — drops a small,
well-known English stop-word list, OR-joins what's left, and lets the
existing bm25 ranking pick the winner among partial matches. Deliberately a
NEW function, not a change to toFtsMatchQuery itself: that one's own tests
rely on "AND"/"OR"/"NOT" surviving verbatim as literal search terms
(FTS5-keyword-injection safety) — a different guarantee than this one's job
of turning a question into a good search. search() gains a `mode: 'and' |
'any'` option (default 'and', so every existing caller is unaffected); the
offline-search route passes 'any', since its one real caller is exactly
this AI-question shape.

Also added, to make the e2e test possible at all: electron/main.ts's
VNCMAIL_TEST_FIXED_PORT — a narrow, off-by-default escape hatch so
DEV_MOCK_JMAP's JMAP_SERVER_URL can point at this same standalone server's
own /api/dev-jmap. Needed because the encrypted index's key channel
(fd-3/safeStorage) only gets wired up in startStandaloneServer()'s own
random-port launch path, never when ELECTRON_LOAD_URL bypasses it for a
plain `next dev` target — so this was the only way to exercise the real
index without a full Stalwart+SMTP Docker fixture.

Verified live in the real packaged Electron shell, not just unit tests:
real dev-mode login, real multi-round /api/offline/sync + /api/offline/reindex
(39 mail/35 calendar/23 contacts indexed), real local-discovery banner
(11 real Ollama models on this machine), real "Connect", a real question
through the real Settings UI, a real direct renderer->Ollama /api/chat call
(confirmed via network log, never proxied through this app's backend), and
the model's own answer citing the exact right fact: "Saturday 28 March at
15:00" — a fact that exists nowhere except in the one indexed email.

4 new unit tests for toFtsMatchQueryAny. Full gate: tsc clean, eslint
clean, 2502/2502 tests passing, build clean, e2e/electron-ai-local-index.spec.ts
passing against the real standalone server + real Electron + real Ollama.
2026-08-06 18:09:14 +02:00

373 lines
16 KiB
TypeScript

// Electron main process for the VNCmail+ (Bulwark) desktop shell.
//
// Boots the exact same Next.js "standalone" server artifact the Dockerfile
// already produces for production (see next.config.ts's `output:
// "standalone"` and the Dockerfile's builder stage) as a child process on a
// random localhost port, then opens a BrowserWindow pointed at it. This is
// deliberately the same server, not a reimplementation - lib/jmap/client.ts
// and every app/api/** route behave identically to the web deployment.
import { app, BrowserWindow, ipcMain, Notification } from "electron";
import { autoUpdater } from "electron-updater";
import { spawn, type ChildProcess } from "node:child_process";
import { createServer } from "node:net";
import { get as httpGet } from "node:http";
import path from "node:path";
import fs from "node:fs";
import type { Duplex } from "node:stream";
import { attachKeyService, checkEncryptionAvailable } from "./key-service";
let serverProcess: ChildProcess | null = null;
let mainWindow: BrowserWindow | null = null;
/**
* Root for the encrypted local search index (lib/mail-index/**). Under
* `userData`, so it is per-OS-user and removed with the app's data.
*
* Passing this to the server child process is what ACTIVATES the index: the
* routes 404 without it. That matters because the standalone server is the same
* artifact the production Dockerfile ships to multi-tenant deployments, where a
* server-side index of every user's mail would be badly wrong. One variable
* both enables the feature and supplies its path, so the two cannot drift apart.
*/
function getIndexStoreDir(): string {
return path.join(app.getPath("userData"), "offline");
}
/**
* Every writable data dir the standalone server uses, redirected under
* `userData`.
*
* WITHOUT this, all four default to `<cwd>/data/*` (see lib/admin/paths.ts,
* lib/settings-sync.ts, lib/telemetry/state.ts, lib/version-check/state.ts),
* and in a packaged build cwd is `.../VNCmail+.app/Contents/Resources/standalone`
* - i.e. the app writes its own runtime state INSIDE its own bundle. Three
* separate failure modes, all observed rather than theorised:
*
* 1. It INVALIDATES THE CODE SIGNATURE. A signed .app seals its Resources;
* writing there breaks the seal, so `codesign --verify` starts failing
* ("code has no resources but signature indicates they must be present")
* and macOS reports the app as *damaged* on a later launch. Verified on
* an installed copy in /Applications: signature valid at install time,
* exit 1 after the app had run once and written data/admin + data/telemetry.
* Deep-signing the bundle at build time (scripts/after-sign.cjs) is
* necessary but NOT sufficient on its own - the app immediately breaks
* its own signature at runtime unless the writes go elsewhere.
* 2. An app update replaces the bundle, silently destroying the user's admin
* config, settings and setup state.
* 3. It fails outright wherever the bundle isn't user-writable.
*
* `userData` is the correct home for per-user mutable state on every platform
* and is where the search index already lives, so this keeps one convention.
*/
function getServerDataDirs(): Record<string, string> {
const root = app.getPath("userData");
return {
ADMIN_CONFIG_DIR: path.join(root, "admin"),
ADMIN_STATE_DIR: path.join(root, "admin-state"),
SETTINGS_DATA_DIR: path.join(root, "settings"),
TELEMETRY_DATA_DIR: path.join(root, "telemetry"),
VERSION_CHECK_DATA_DIR: path.join(root, "version-check"),
};
}
/**
* Desktop-shell defaults for a fresh, un-configured install.
*
* Setting JMAP_SERVER_URL puts the standalone server into "env-managed"
* mode (see lib/setup/state.ts's detectSetupState()) - the ONLY thing that
* disables the setup wizard short of an operator finishing it by hand. Every
* distributable build of this desktop shell up to 2026-08-05 skipped this,
* so handing someone the packaged app landed them on "Bulwark Webmail
* Setup" asking for a token out of container logs they have no access to -
* caught only by actually launching the packaged .app and looking, not by
* reading the build log.
*
* The rest are CONFIG_ENV_MAP entries (lib/admin/types.ts) that only matter
* while env-managed - once an admin completes the wizard, config.json wins
* for everything except jmapServerUrl itself. allowCustomJmapEndpoint keeps
* the server field on the login screen editable, so this is a starting
* point for the sandbox, not a hard lock to it.
*
* `...process.env` in startStandaloneServer() below is spread AFTER this
* object, so a real deployment env (the Dockerfile path, or a future
* per-install override) still wins over these defaults.
*/
function getDesktopDefaults(): Record<string, string> {
return {
JMAP_SERVER_URL: "https://stalwart.sandbox.vnc.de",
APP_NAME: "VNCmail+",
APP_SHORT_NAME: "VNCmail+",
LOGIN_LOGO_LIGHT_URL: "/branding/SRC_Symbol.png",
LOGIN_LOGO_DARK_URL: "/branding/SRC_Symbol.png",
LOGIN_COMPANY_NAME: "VNC AG",
FAVICON_URL: "/branding/SRC_Symbol.png",
ALLOW_CUSTOM_JMAP_ENDPOINT: "true",
// The login page's subtitle falls back to the login.title i18n string
// whenever it differs from appName (app/(main)/[locale]/login/page.tsx)
// - a check clearly written for the original Bulwark/"Webmail" pairing,
// where they matched. With APP_NAME overridden to "VNCmail+" they no
// longer match, so the raw translation ("Webmail") surfaces instead of
// anything brand-appropriate. Hiding the subtitle avoids editing a
// shared i18n string that every other deployment (incl. Bulwark
// default) still uses - the SRC logo + "VNCmail+" heading is enough
// context on its own.
LOGIN_SHOW_SUBTITLE: "false",
};
}
/**
* Locates the standalone server's entrypoint. Packaged builds ship it as an
* extraResource (see electron-builder.config.js) because .next/standalone
* isn't inside the app.asar; dev runs read it straight out of the repo via
* `npm run build:standalone`.
*/
function getStandaloneServerEntry(): string {
if (app.isPackaged) {
return path.join(process.resourcesPath, "standalone", "server.js");
}
return path.join(app.getAppPath(), ".next", "standalone", "server.js");
}
function getFreePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (address && typeof address === "object") {
const { port } = address;
server.close(() => resolve(port));
} else {
server.close(() => reject(new Error("Could not allocate a free localhost port")));
}
});
});
}
function waitForServerReady(url: string, timeoutMs = 20000): Promise<void> {
const deadline = Date.now() + timeoutMs;
return new Promise((resolve, reject) => {
const attempt = () => {
const req = httpGet(url, (res) => {
res.resume();
resolve();
});
req.on("error", () => {
if (Date.now() > deadline) {
reject(new Error(`Standalone server never became reachable at ${url}`));
return;
}
setTimeout(attempt, 200);
});
};
attempt();
});
}
async function startStandaloneServer(): Promise<string> {
const serverEntry = getStandaloneServerEntry();
if (!fs.existsSync(serverEntry)) {
throw new Error(
`Standalone Next.js server not found at ${serverEntry}. Run "npm run build:standalone" first.`,
);
}
// Normally a random free port, chosen fresh every launch - JMAP_SERVER_URL
// never needs to reference it back (a real deployment's Stalwart lives at
// its own fixed address). VNCMAIL_TEST_FIXED_PORT is a narrow escape hatch
// for e2e tests that DO need to know the port ahead of time - specifically
// to point DEV_MOCK_JMAP's JMAP_SERVER_URL at this same standalone server's
// own /api/dev-jmap route, which is the only way to exercise the real
// encrypted offline index (lib/mail-index/**) without a real Stalwart
// fixture: that index's key channel only gets wired up in this function,
// never when ELECTRON_LOAD_URL bypasses it for a plain `next dev` target.
// Unset in every normal launch, so this changes nothing outside a test run.
const fixedPort = process.env.VNCMAIL_TEST_FIXED_PORT ? Number(process.env.VNCMAIL_TEST_FIXED_PORT) : null;
const port = fixedPort && Number.isInteger(fixedPort) ? fixedPort : await getFreePort();
const url = `http://127.0.0.1:${port}`;
const storeDir = getIndexStoreDir();
const encryption = checkEncryptionAvailable();
if (!encryption.ok) {
// Refuse rather than degrade. On Linux with no keyring, safeStorage
// "succeeds" using a hardcoded public password, which would look like an
// encrypted mailbox index while providing no protection. Leaving the env
// vars unset makes every index route 404, so the app runs normally without
// the feature.
console.error(`[electron] local search index disabled: ${encryption.reason}`);
}
// Spawn the Electron binary itself as a plain Node process
// (ELECTRON_RUN_AS_NODE) instead of depending on a system Node install -
// the packaged app can't assume Node exists on the target machine, and
// this keeps dev/packaged behavior identical.
//
// stdio gains a 4th entry: fd 3 is the key channel for the local index (see
// electron/key-service.ts). libuv creates extra stdio "pipe" entries as
// socketpairs, so it is duplex in both directions - verified by execution
// before this was built on. Deliberately NOT an environment variable: env is
// readable by any process running as the same OS user, which would defeat
// using the OS keychain at all. The fd NUMBER below is not a secret; only
// what travels over it is.
serverProcess = spawn(process.execPath, [serverEntry], {
env: {
// First, so any real deployment env (a future per-install override,
// or this same binary run somewhere JMAP_SERVER_URL is already set)
// wins over these desktop-shell defaults - see getDesktopDefaults().
...getDesktopDefaults(),
...process.env,
ELECTRON_RUN_AS_NODE: "1",
PORT: String(port),
HOSTNAME: "127.0.0.1",
NODE_ENV: process.env.NODE_ENV || "production",
// Keep all mutable state out of the .app bundle - see
// getServerDataDirs() for why that matters. Placed after
// ...process.env so the desktop shell's paths win over any inherited
// value; the same standalone server run outside Electron (the Docker
// image) never executes this and keeps its documented env behaviour.
...getServerDataDirs(),
...(encryption.ok
? { VNCMAIL_DESKTOP_STORE_DIR: storeDir, VNCMAIL_DESKTOP_KEY_FD: "3" }
: {}),
},
stdio: encryption.ok
? ["inherit", "inherit", "inherit", "pipe"]
: "inherit",
});
if (encryption.ok) {
attachKeyService(serverProcess.stdio[3] as Duplex | null, storeDir);
}
serverProcess.on("exit", (code, signal) => {
if (code !== 0 && code !== null) {
console.error(`[electron] standalone server exited early (code=${code}, signal=${signal})`);
}
serverProcess = null;
});
await waitForServerReady(url);
return url;
}
function stopStandaloneServer(): void {
if (serverProcess && !serverProcess.killed) {
serverProcess.kill();
}
serverProcess = null;
}
async function createMainWindow(): Promise<void> {
// Test-only escape hatch: when set, skip spawning the standalone server
// entirely and load this URL instead. Used by
// integration/tests/11-electron-notification.spec.ts, which needs a
// dev-mode Next.js server (proxy.ts's CSP only widens connect-src to
// allow plain-HTTP/ws JMAP in dev - see that file's comments) to reach
// the integration fixture's deliberately-plaintext local Stalwart,
// exactly the same trade-off integration/webmail.Dockerfile already makes
// for the browser-based integration suite. Never set by real users or by
// any of the packaging/CI paths - those always go through
// startStandaloneServer() below.
const url = process.env.ELECTRON_LOAD_URL || (await startStandaloneServer());
mainWindow = new BrowserWindow({
width: 1280,
height: 860,
webPreferences: {
preload: path.join(__dirname, "preload.js"),
contextIsolation: true,
nodeIntegration: false,
sandbox: true,
},
});
mainWindow.on("closed", () => {
mainWindow = null;
});
await mainWindow.loadURL(url);
}
// --- Native notification bridge --------------------------------------------
// Called from the preload's `window.vnc.showNotification` (electron/preload.ts),
// itself called from lib/electron-bridge.ts's showElectronNotification(),
// itself called from app/(main)/[locale]/page.tsx's "new mail arrived"
// effect whenever lib/jmap/client.ts's push pipeline (WebSocket, or its SSE/
// polling fallback - see that file's circuit breaker) reports a genuine new
// message. Electron's own Notification API is the desktop shell's
// notification path - it sits alongside, not in place of, the browser/PWA's
// service-worker push path (public/sw.js's `push`/`notificationclick`
// handlers + lib/web-push.ts).
ipcMain.handle(
"vnc:show-notification",
(_event, title: string, options?: { body?: string; tag?: string }) => {
// Test-only observability hook, read via Playwright's
// electronApp.evaluate(({ app }) => ...) - see
// integration/tests/11-electron-notification.spec.ts. Not gated behind
// NODE_ENV: it's an inert counter with no behavioral effect, cheaper
// than maintaining a second code path just for tests.
const counters = app as unknown as { __notificationCallCount?: number };
counters.__notificationCallCount = (counters.__notificationCallCount ?? 0) + 1;
if (!Notification.isSupported()) {
return { shown: false };
}
const notification = new Notification({
title,
body: options?.body ?? "",
});
notification.show();
return { shown: true };
},
);
// --- Auto-update -------------------------------------------------------
// GitHub Releases as the update feed (electron-builder.config.js's
// `publish` block) - the skill's recommendation over standing up a new
// distribution channel, since the repo is already private. "Light
// decision" per VNCprodbuild step 7, not re-litigated here.
//
// Deliberately best-effort: there's no code signing yet (step 9), so on
// macOS in particular an update download/install can fail signature
// verification. A failed check must never take the app down - it's
// background maintenance, not something the user is blocked on.
function setupAutoUpdater(): void {
if (!app.isPackaged) {
// Unpacked dev/test runs (npm run electron:dev, the Playwright smoke
// test) have no latest.yml alongside them - checking would just log a
// noisy 404 against GitHub Releases for every dev run.
return;
}
autoUpdater.autoDownload = true;
autoUpdater.autoInstallOnAppQuit = true;
autoUpdater.on("error", (error) => {
console.error("[electron] auto-update error:", error);
});
autoUpdater.checkForUpdatesAndNotify().catch((error) => {
console.error("[electron] checkForUpdatesAndNotify failed:", error);
});
}
app.whenReady().then(() => {
void createMainWindow();
setupAutoUpdater();
});
app.on("window-all-closed", () => {
stopStandaloneServer();
if (process.platform !== "darwin") {
app.quit();
}
});
app.on("before-quit", () => {
stopStandaloneServer();
});
app.on("activate", () => {
if (BrowserWindow.getAllWindows().length === 0) {
void createMainWindow();
}
});