2a8778c9052b0e4e1df4ea29d2c1e713fdeffa08
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bernd RodlerandClaude Sonnet 5
|
bde8455df5 |
fix(mail-index): close handle on every pragma failure, reconcile contact/file deletes
QA pass on the encrypted mail index found two real gaps beyond what the
prior end-to-end fix pass caught:
1. store.ts's MailIndex.open() only wrapped SOME of the post-key pragma
calls in a try/catch before this: the first attempt's `key` pragma and
assertEncrypted() ran outside any try at all, and the wrong-key retry
repeated the same gap. Any pragma throwing there (SQLITE_BUSY, a full
disk on the first WAL write) leaked the native SQLite handle instead of
closing it. Factored the open+key+verify+pragma sequence into openKeyed(),
which guarantees a close before rethrowing on any failure, and reused it
for both the first attempt and the retry.
2. reindex.ts never removed a deleted contact or file from the index. The
`removed` field exists in the API and is fully tested at the store layer,
but nothing in the renderer populates it, so a deleted contact/file stayed
searchable - and retrievable by the AI feature - indefinitely. Mail and
calendar can't use the same fix (their queries are date-windowed, so an id
missing from one fetch may just be outside the window), but contacts/files
have no date filter - a catch-up fetch that comes back under its cap IS
the complete set, so anything locally indexed but absent from it is safely
known to be deleted. Added strayIdsAfterCatchUp() and wired it into the
catch-up path for those two types only.
Also read binding.ts, key.ts, paths.ts, jmap.ts, extract.ts, the FTS5
query builder, and both /api/offline/{search,reindex} routes end to end;
no other concrete bugs found there. Full findings reported separately.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Bernd RodlerandClaude Sonnet 5
|
b966d285a9 |
feat(mail-index): encrypted SQLite/FTS5 index over mail, calendar, contacts, files
An on-device, SQLCipher-encrypted full-text index the app can retrieve from to
feed an LLM ("prompt against"), for the Electron desktop shell only.
Shape: no persistent background worker and no resident credential. Indexing is
a normal request-scoped API route, triggered by the renderer's EXISTING live
JMAP push connection - so it reacts to each delivery/change rather than polling.
- lib/mail-index/binding.ts guarded require of the optional native binding
- lib/mail-index/paths.ts the VNCMAIL_DESKTOP_STORE_DIR gate + hashed paths
- lib/mail-index/store.ts schema, upsert, FTS5 search, encryption assertion
- lib/mail-index/extract.ts PURE JMAP-object -> document extractors
- lib/mail-index/jmap.ts minimal stateless server-side JMAP client
- lib/mail-index/key.ts per-job key fetch over the inherited fd
- lib/mail-index/reindex.ts the job + slot->account resolution
- electron/key-service.ts safeStorage wrap/unwrap, served over fd 3
- app/api/offline/reindex POST, event-driven + catch-up
- app/api/offline/search GET, the retrieval surface (hits + contextBlock)
- lib/mail-index-client.ts renderer client; StateChange -> index call
- components/settings/local-index-settings.tsx status + manual catch-up
Decisions worth knowing:
* `@signalapp/sqlcipher` is an OPTIONAL dependency with a guarded runtime
require. It publishes six N-API prebuilds and NO build sources, and both
Dockerfiles are node:24-alpine (musl, no matching prebuild) - as a hard
dependency it would break the production image and the integration fixture's
webmail container, neither of which wants this feature.
* Credentials come from the existing per-slot encrypted `jmap_stalwart_ctx`
cookie via lib/stalwart/credentials.ts - the same helper /api/settings and
/api/push/preview already use. It carries a ready-made header for basic AND
bearer accounts, so the indexer never touches the OAuth refresh-token cookie;
a server-side refresh would rotate a token into a response nobody reads and
silently log the user out.
* The encryption key crosses main -> server over an INHERITED FILE DESCRIPTOR,
never an environment variable: env is readable by any process running as the
same OS user, which would defeat using the OS keychain at all. Fetched per
job and zeroed after, so there is no long-lived key copy.
* safeStorage's Linux `basic_text` backend (no keyring) is treated as refusal,
not degradation - it "encrypts" with a hardcoded public password, which would
look like an encrypted mailbox while providing nothing.
getSelectedStorageBackend() is Linux-only and platform-guarded.
* Every store open asserts `PRAGMA cipher_version` returns a non-empty STRING,
not merely a row: a non-cipher binding returns ZERO ROWS, so a row-count check
would pass vacuously while writing the mailbox to disk in cleartext.
* Files are indexed by name/path/date/size only - NOT by extracted content.
Text extraction from arbitrary PDFs/office documents is a separate problem.
* Account-scoped composite keys `(jmap_account_id, content_type, id)` are kept
even though there is one file per account: one login exposes delegated/shared
JMAP accounts too, and JMAP ids are unique only within an account.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|