Files
SRCmail/electron/key-service.ts
Bernd RodlerandClaude Sonnet 5 b966d285a9 feat(mail-index): encrypted SQLite/FTS5 index over mail, calendar, contacts, files
An on-device, SQLCipher-encrypted full-text index the app can retrieve from to
feed an LLM ("prompt against"), for the Electron desktop shell only.

Shape: no persistent background worker and no resident credential. Indexing is
a normal request-scoped API route, triggered by the renderer's EXISTING live
JMAP push connection - so it reacts to each delivery/change rather than polling.

- lib/mail-index/binding.ts   guarded require of the optional native binding
- lib/mail-index/paths.ts     the VNCMAIL_DESKTOP_STORE_DIR gate + hashed paths
- lib/mail-index/store.ts     schema, upsert, FTS5 search, encryption assertion
- lib/mail-index/extract.ts   PURE JMAP-object -> document extractors
- lib/mail-index/jmap.ts      minimal stateless server-side JMAP client
- lib/mail-index/key.ts       per-job key fetch over the inherited fd
- lib/mail-index/reindex.ts   the job + slot->account resolution
- electron/key-service.ts     safeStorage wrap/unwrap, served over fd 3
- app/api/offline/reindex     POST, event-driven + catch-up
- app/api/offline/search      GET, the retrieval surface (hits + contextBlock)
- lib/mail-index-client.ts    renderer client; StateChange -> index call
- components/settings/local-index-settings.tsx  status + manual catch-up

Decisions worth knowing:

* `@signalapp/sqlcipher` is an OPTIONAL dependency with a guarded runtime
  require. It publishes six N-API prebuilds and NO build sources, and both
  Dockerfiles are node:24-alpine (musl, no matching prebuild) - as a hard
  dependency it would break the production image and the integration fixture's
  webmail container, neither of which wants this feature.

* Credentials come from the existing per-slot encrypted `jmap_stalwart_ctx`
  cookie via lib/stalwart/credentials.ts - the same helper /api/settings and
  /api/push/preview already use. It carries a ready-made header for basic AND
  bearer accounts, so the indexer never touches the OAuth refresh-token cookie;
  a server-side refresh would rotate a token into a response nobody reads and
  silently log the user out.

* The encryption key crosses main -> server over an INHERITED FILE DESCRIPTOR,
  never an environment variable: env is readable by any process running as the
  same OS user, which would defeat using the OS keychain at all. Fetched per
  job and zeroed after, so there is no long-lived key copy.

* safeStorage's Linux `basic_text` backend (no keyring) is treated as refusal,
  not degradation - it "encrypts" with a hardcoded public password, which would
  look like an encrypted mailbox while providing nothing.
  getSelectedStorageBackend() is Linux-only and platform-guarded.

* Every store open asserts `PRAGMA cipher_version` returns a non-empty STRING,
  not merely a row: a non-cipher binding returns ZERO ROWS, so a row-count check
  would pass vacuously while writing the mailbox to disk in cleartext.

* Files are indexed by name/path/date/size only - NOT by extracted content.
  Text extraction from arbitrary PDFs/office documents is a separate problem.

* Account-scoped composite keys `(jmap_account_id, content_type, id)` are kept
  even though there is one file per account: one login exposes delegated/shared
  JMAP accounts too, and JMAP ids are unique only within an account.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-04 23:16:22 +02:00

232 lines
8.6 KiB
TypeScript

// Main-process key service for the local search index.
//
// The index database is SQLCipher-encrypted with a random per-account 32-byte
// key. That key is wrapped with Electron's `safeStorage` (OS keychain / DPAPI /
// libsecret-or-kwallet) and stored under the store directory. Only the main
// process can call `safeStorage`, but the index itself lives in the standalone
// Next.js server child process - so the unwrapped key has to cross one process
// boundary.
//
// TRANSPORT: an inherited file descriptor (fd 3), NOT an environment variable.
// A nonce or key passed through the spawned process's environment is readable
// by any other process running as the same OS user (`ps eww`, /proc/<pid>/environ),
// which would defeat the entire point of using the OS keychain. An inherited fd
// is not exposed to process listing. libuv creates extra stdio "pipe" entries
// as socketpairs, so fd 3 is duplex - verified by execution through Electron's
// own spawn before this was built on.
//
// The server side asks for a key only when a reindex job actually runs and drops
// it when the job finishes (see lib/mail-index/key.ts) - there is no long-lived
// resident copy anywhere.
import { safeStorage } from "electron";
import { createHash, randomBytes } from "node:crypto";
import fs from "node:fs";
import path from "node:path";
import type { Readable, Writable } from "node:stream";
/** Must match lib/mail-index/paths.ts's accountFileToken(). */
function accountFileToken(accountId: string): string {
return createHash("sha256").update(accountId, "utf8").digest("hex").slice(0, 32);
}
function keyFilePath(storeDir: string, accountId: string): string {
return path.join(storeDir, "keys", `${accountFileToken(accountId)}.bin`);
}
export type KeyServiceFailure =
/** No OS keyring at all - see the Linux note in checkEncryptionAvailable(). */
| "no-secure-storage"
/** Reading/writing the wrapped key file failed. */
| "key-io-failed"
/** The wrapped key exists but safeStorage could not decrypt it. */
| "key-unreadable";
export class KeyServiceError extends Error {
code: KeyServiceFailure;
constructor(code: KeyServiceFailure, message: string) {
super(message);
this.name = "KeyServiceError";
this.code = code;
}
}
/**
* Decides whether we are willing to store an encryption key on this system.
*
* The Linux caveat is the reason this is a function and not a one-liner:
* `safeStorage.isEncryptionAvailable()` can return **true** on Linux while the
* data is protected by a hardcoded, publicly-known password, with
* `getSelectedStorageBackend()` reporting `basic_text`. That is worse than an
* honest failure, because it looks like it worked. So a `basic_text` backend is
* treated as "no secure storage" and the feature refuses to materialise
* anything - the index is a convenience, and silently pretending a mailbox is
* encrypted when it is not is not a trade worth making.
*
* `getSelectedStorageBackend()` is **Linux-only** and throws elsewhere, hence
* the platform guard. Both calls also require `app.whenReady()`.
*/
export function checkEncryptionAvailable(): { ok: true } | { ok: false; reason: string } {
if (!safeStorage.isEncryptionAvailable()) {
return { ok: false, reason: "The OS reports no secure storage available for encryption keys." };
}
if (process.platform === "linux") {
let backend: string;
try {
backend = safeStorage.getSelectedStorageBackend();
} catch {
// Older/newer Electron, or called too early. Be conservative.
return { ok: false, reason: "Could not determine the Linux secret-storage backend." };
}
if (backend === "basic_text" || backend === "unknown") {
return {
ok: false,
reason:
`No OS keyring is available (backend: ${backend}). Electron would "encrypt" the key ` +
`with a hardcoded password, which provides no real protection, so the encrypted ` +
`local index is disabled on this system.`,
};
}
}
return { ok: true };
}
/** Fetches the account's raw index key, creating and wrapping one on first use. */
function getOrCreateKey(storeDir: string, accountId: string): Buffer {
const availability = checkEncryptionAvailable();
if (!availability.ok) {
throw new KeyServiceError("no-secure-storage", availability.reason);
}
const file = keyFilePath(storeDir, accountId);
if (fs.existsSync(file)) {
let wrapped: Buffer;
try {
wrapped = fs.readFileSync(file);
} catch (error) {
throw new KeyServiceError("key-io-failed", `Could not read the key file: ${String(error)}`);
}
let hex: string;
try {
hex = safeStorage.decryptString(wrapped);
} catch (error) {
// Most likely cause on macOS: the app's code identity changed (unsigned
// builds get a fresh ad-hoc signature per build), so the Keychain ACL no
// longer matches. Not recoverable and not a user secret - the caller
// deletes the database and re-indexes.
throw new KeyServiceError(
"key-unreadable",
`The stored key could not be decrypted (${String(error)}). It must be recreated.`,
);
}
const key = Buffer.from(hex.trim(), "hex");
if (key.length === 32) return key;
// Corrupt payload: fall through and mint a new one.
}
const key = randomBytes(32);
let wrapped: Buffer;
try {
wrapped = safeStorage.encryptString(key.toString("hex"));
} catch (error) {
throw new KeyServiceError("no-secure-storage", `Could not wrap the key: ${String(error)}`);
}
try {
fs.mkdirSync(path.dirname(file), { recursive: true, mode: 0o700 });
// Write-then-rename so a crash mid-write cannot leave a truncated wrapped
// key that would look like "key-unreadable" forever.
const tmp = `${file}.tmp-${process.pid}`;
fs.writeFileSync(tmp, wrapped, { mode: 0o600 });
fs.renameSync(tmp, file);
} catch (error) {
throw new KeyServiceError("key-io-failed", `Could not persist the key: ${String(error)}`);
}
return key;
}
function deleteKey(storeDir: string, accountId: string): void {
try {
fs.rmSync(keyFilePath(storeDir, accountId), { force: true });
} catch {
/* best effort - the caller is purging anyway */
}
}
interface Request {
id?: unknown;
op?: unknown;
accountId?: unknown;
}
/**
* Serves newline-delimited JSON requests from the standalone server over the
* inherited fd. One line in, one line out, no streaming and no state.
*/
export function attachKeyService(
channel: (Readable & Writable) | null | undefined,
storeDir: string,
): void {
if (!channel) {
console.error("[electron] key service: no channel on fd 3; the local index will be disabled");
return;
}
let buffer = "";
const respond = (payload: Record<string, unknown>) => {
try {
channel.write(`${JSON.stringify(payload)}\n`);
} catch (error) {
console.error("[electron] key service: failed to write response:", error);
}
};
channel.on("data", (chunk: Buffer | string) => {
buffer += typeof chunk === "string" ? chunk : chunk.toString("utf8");
// Guard against a peer that never sends a newline.
if (buffer.length > 64 * 1024) buffer = "";
let newline: number;
while ((newline = buffer.indexOf("\n")) >= 0) {
const line = buffer.slice(0, newline);
buffer = buffer.slice(newline + 1);
if (!line.trim()) continue;
let req: Request;
try {
req = JSON.parse(line) as Request;
} catch {
respond({ id: null, ok: false, code: "bad-request", error: "Malformed request" });
continue;
}
const id = typeof req.id === "number" ? req.id : null;
const accountId = typeof req.accountId === "string" ? req.accountId : "";
if (!accountId) {
respond({ id, ok: false, code: "bad-request", error: "Missing accountId" });
continue;
}
try {
if (req.op === "getIndexKey") {
const key = getOrCreateKey(storeDir, accountId);
respond({ id, ok: true, key: key.toString("hex") });
key.fill(0);
} else if (req.op === "deleteIndexKey") {
deleteKey(storeDir, accountId);
respond({ id, ok: true });
} else {
respond({ id, ok: false, code: "bad-request", error: `Unknown op: ${String(req.op)}` });
}
} catch (error) {
const code = error instanceof KeyServiceError ? error.code : "key-io-failed";
const message = error instanceof Error ? error.message : String(error);
respond({ id, ok: false, code, error: message });
}
}
});
channel.on("error", (error: unknown) => {
console.error("[electron] key service channel error:", error);
});
}