An on-device, SQLCipher-encrypted full-text index the app can retrieve from to
feed an LLM ("prompt against"), for the Electron desktop shell only.
Shape: no persistent background worker and no resident credential. Indexing is
a normal request-scoped API route, triggered by the renderer's EXISTING live
JMAP push connection - so it reacts to each delivery/change rather than polling.
- lib/mail-index/binding.ts guarded require of the optional native binding
- lib/mail-index/paths.ts the VNCMAIL_DESKTOP_STORE_DIR gate + hashed paths
- lib/mail-index/store.ts schema, upsert, FTS5 search, encryption assertion
- lib/mail-index/extract.ts PURE JMAP-object -> document extractors
- lib/mail-index/jmap.ts minimal stateless server-side JMAP client
- lib/mail-index/key.ts per-job key fetch over the inherited fd
- lib/mail-index/reindex.ts the job + slot->account resolution
- electron/key-service.ts safeStorage wrap/unwrap, served over fd 3
- app/api/offline/reindex POST, event-driven + catch-up
- app/api/offline/search GET, the retrieval surface (hits + contextBlock)
- lib/mail-index-client.ts renderer client; StateChange -> index call
- components/settings/local-index-settings.tsx status + manual catch-up
Decisions worth knowing:
* `@signalapp/sqlcipher` is an OPTIONAL dependency with a guarded runtime
require. It publishes six N-API prebuilds and NO build sources, and both
Dockerfiles are node:24-alpine (musl, no matching prebuild) - as a hard
dependency it would break the production image and the integration fixture's
webmail container, neither of which wants this feature.
* Credentials come from the existing per-slot encrypted `jmap_stalwart_ctx`
cookie via lib/stalwart/credentials.ts - the same helper /api/settings and
/api/push/preview already use. It carries a ready-made header for basic AND
bearer accounts, so the indexer never touches the OAuth refresh-token cookie;
a server-side refresh would rotate a token into a response nobody reads and
silently log the user out.
* The encryption key crosses main -> server over an INHERITED FILE DESCRIPTOR,
never an environment variable: env is readable by any process running as the
same OS user, which would defeat using the OS keychain at all. Fetched per
job and zeroed after, so there is no long-lived key copy.
* safeStorage's Linux `basic_text` backend (no keyring) is treated as refusal,
not degradation - it "encrypts" with a hardcoded public password, which would
look like an encrypted mailbox while providing nothing.
getSelectedStorageBackend() is Linux-only and platform-guarded.
* Every store open asserts `PRAGMA cipher_version` returns a non-empty STRING,
not merely a row: a non-cipher binding returns ZERO ROWS, so a row-count check
would pass vacuously while writing the mailbox to disk in cleartext.
* Files are indexed by name/path/date/size only - NOT by extracted content.
Text extraction from arbitrary PDFs/office documents is a separate problem.
* Account-scoped composite keys `(jmap_account_id, content_type, id)` are kept
even though there is one file per account: one login exposes delegated/shared
JMAP accounts too, and JMAP ids are unique only within an account.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
84 lines
3.7 KiB
TypeScript
84 lines
3.7 KiB
TypeScript
// Guarded loader for the SQLCipher native binding.
|
|
//
|
|
// WHY THIS FILE EXISTS AT ALL: `@signalapp/sqlcipher` is declared in
|
|
// package.json's `optionalDependencies`, not `dependencies`, and it MUST stay
|
|
// there. It publishes six N-API prebuilds (darwin/linux/win32 x arm64/x64) and
|
|
// **no build sources at all** - the published tarball has no `binding.gyp`, no
|
|
// `src/`, no `deps/`. Its install script is `node-gyp-build`, which falls back
|
|
// to `node-gyp rebuild` when no prebuild matches, and that fallback cannot
|
|
// succeed without sources. So on a platform with no matching prebuild the
|
|
// install FAILS.
|
|
//
|
|
// Both Dockerfiles in this repo are `FROM node:24-alpine` + `npm ci`
|
|
// (`Dockerfile:1-4`, `integration/webmail.Dockerfile:15-19`). Alpine is musl;
|
|
// there is no `linuxmusl-*` prebuild (and the glibc prebuild could not load
|
|
// there anyway). As a hard `dependencies` entry this would break the
|
|
// production image build and the integration fixture's webmail container -
|
|
// neither of which wants this feature, they just need `npm ci` to exit 0.
|
|
// `optionalDependencies` makes npm treat that install failure as non-fatal and
|
|
// simply omit the package.
|
|
//
|
|
// The cost of that choice is exactly this module: the require must be guarded
|
|
// at runtime, because "installed" is no longer guaranteed. Callers get
|
|
// `null` and the feature turns itself off, which is the correct behaviour for
|
|
// a desktop-only search index in a server that may not be a desktop.
|
|
|
|
/**
|
|
* Minimal structural type for the bits of `@signalapp/sqlcipher` we use.
|
|
*
|
|
* Deliberately hand-written rather than `typeof import('@signalapp/sqlcipher')`:
|
|
* the package is optional, so a type-only import would make `tsc` fail on any
|
|
* machine where the install was skipped - which is every Alpine CI container.
|
|
*
|
|
* NOTE the parameter shape. `@signalapp/sqlcipher` is NOT drop-in compatible
|
|
* with better-sqlite3 here: its `#checkParams` throws
|
|
* `TypeError: Params must be either object or array`, so `stmt.run(a, b, c)`
|
|
* (varargs, which better-sqlite3 accepts) is a runtime error. Always pass a
|
|
* single array or object. Found by executing it, not by reading the types.
|
|
*/
|
|
export interface SqlcipherStatement {
|
|
run(params?: readonly unknown[] | Record<string, unknown>): { changes: number; lastInsertRowid: number };
|
|
get(params?: readonly unknown[] | Record<string, unknown>): Record<string, unknown> | undefined;
|
|
all(params?: readonly unknown[] | Record<string, unknown>): Array<Record<string, unknown>>;
|
|
}
|
|
|
|
export interface SqlcipherDatabase {
|
|
exec(sql: string): void;
|
|
prepare(sql: string): SqlcipherStatement;
|
|
pragma(source: string): unknown;
|
|
close(): void;
|
|
}
|
|
|
|
export interface SqlcipherConstructor {
|
|
new (path?: string): SqlcipherDatabase;
|
|
}
|
|
|
|
let cached: SqlcipherConstructor | null | undefined;
|
|
|
|
/**
|
|
* Returns the Database constructor, or `null` when the optional native binding
|
|
* is not installed / cannot load on this platform. Never throws.
|
|
*
|
|
* Memoised on both outcomes so a missing binding costs one failed require per
|
|
* process rather than one per request.
|
|
*/
|
|
export function loadSqlcipher(): SqlcipherConstructor | null {
|
|
if (cached !== undefined) return cached;
|
|
try {
|
|
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
|
const mod = require('@signalapp/sqlcipher') as
|
|
| { default?: SqlcipherConstructor }
|
|
| SqlcipherConstructor;
|
|
const ctor = (mod as { default?: SqlcipherConstructor }).default ?? (mod as SqlcipherConstructor);
|
|
cached = typeof ctor === 'function' ? ctor : null;
|
|
} catch {
|
|
cached = null;
|
|
}
|
|
return cached;
|
|
}
|
|
|
|
/** True when the local index can work at all in this process. */
|
|
export function isSqlcipherAvailable(): boolean {
|
|
return loadSqlcipher() !== null;
|
|
}
|