output: "standalone" + next build --webpack drops the whole metadata
directory despite a plain top-level require in router-utils/filesystem.js
("../../../lib/metadata/get-metadata-route") - every packaged build
(Electron dmg and the Docker image) crashed on its very first line with
Cannot find module. Verified: a fresh dist:mac build failed to boot at
all; copying the directory by hand (same pattern already used for the
sqlcipher prebuilds and plugin bundles) fixes it, confirmed by booting
.next/standalone directly and getting a real HTTP response.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Before this, the OpenCode class could only use providers already authenticated
via its own CLI (opencode auth login) — this app could pick a MODEL, never add
a PROVIDER. That is the one thing standing between "OpenCode integration" and
the actual ask: any LLM it supports, added from here.
New GET/PUT/DELETE /api/ai/opencode/providers, backed by GET /provider (every
provider OpenCode knows — 180 on a real run) and GET /provider/auth (which
auth method each accepts). New "Manage providers" panel in the OpenCode
settings section: search, add a key, remove one.
Scoped to API-key auth only, deliberately — recorded in lib/ai/opencode.ts's
module comment. `PUT /auth/{id}` with `{type:'api', key}` is one HTTP call
with a schema-verified shape. OAuth entries in /provider/auth need a browser
redirect + callback this app has no page for, and some carry interactive
prompts beyond a single form (GitHub Copilot's deployment-type picker) — real
scope for later, not something to half-build. OAuth-only providers are still
LISTED, just marked "Browser sign-in only" rather than hidden, so the picker
stays honest about what it can't do here.
A real finding from testing this against opencode's actual behaviour rather
than trusting a 200: NOT EVERY PROVIDER BECOMES CONNECTED FROM A BARE API KEY.
Snowflake Cortex needs SNOWFLAKE_ACCOUNT alongside its token; a single key
field silently leaves it stored-but-unconnected with no error from the PUT
itself. Worse, the provider's own `env` array length does not predict this —
Azure also needs two env vars and DOES connect from one key. There is no
reliable way to know in advance, so the route now VERIFIES by re-listing
providers after the write and reports plainly when a key was accepted but the
provider still isn't connected, rather than reporting the PUT's own success.
Verified live end-to-end, twice: once confirming a simple single-field
provider connects and can be removed cleanly, once confirming the honest
"stored but not connected" case is real and detected, not theoretical.
Cleaned up every throwaway credential from this machine's real opencode
config afterwards (checked auth.json directly, not just this app's view of it).
17 new/updated unit tests. Gate: tsc clean, eslint clean, 2527/2527 tests, build clean.
B1 — LIFECYCLE. The OpenCode class previously required the user to remember
to run `opencode serve` in a terminal before opening their mail app, and again
after every reboot; in practice that means the feature quietly stops existing.
The desktop shell now owns it: finds the binary (OPENCODE_BIN, then
~/.opencode/bin — its installer's default, which is NOT on the PATH a macOS
GUI app inherits, so PATH alone finds nothing for most users), starts it on a
free port, restarts up to 3 times if it dies, and kills it on quit. Absent
binary = the class simply stays unavailable, no error.
B3 — SECURITY. opencode's own startup warns "OPENCODE_SERVER_PASSWORD is not
set; server is unsecured" — without one, any local process can drive the
agent. A per-launch password is now always generated (never persisted: the
server dies with the app, so a durable secret would be pure liability) and
handed to the standalone server alongside the base URL.
The auth scheme is worth recording because it is NOT in opencode's own
OpenAPI spec, which declares no securitySchemes at all: HTTP Basic with the
username EXACTLY `opencode`. Verified against 1.18.14 by trying them — an
empty username, an arbitrary one, Bearer, and every plausible custom header
all 401 with the correct password. Pinned by a unit test that decodes the
header, so a future refactor can't silently drop it.
Verified live against a real password-protected server on 4097: authenticated
discovery + prompt round-tripped, AND the same call with no password was
rejected — proving the auth is real rather than decorative.
Also removed now-stale guidance: the 503 no longer says "start one with
opencode serve", because the app does that; it says to install the CLI.
Gate: tsc clean, eslint clean, build clean, 2521/2522 tests. The one failure
is lib/__tests__/jmap-client-resilience.test.ts's onConnectionChange timing
flake — byte-identical to what is already running in prod (git diff vs
origin/main for that file and lib/jmap/ is empty), pre-existing, and
unrelated to anything here.
The two things that made real questions fail against a correctly-populated
index, both fixed at the root.
RETENTION (A1). `INDEX_WINDOW_DAYS = 30` was not merely a fetch bound — catch-up
also PRUNED mail older than it, so "summarise everything from July" was
unanswerable in August because the rows had been deleted, while the UI said
only that nothing matched. Now a user-visible setting (Settings → About &
Data): 30 days / 3 months / 1 year / everything, defaulting to 1 YEAR per the
product owner. The window bounds the fetch AND the prune from one value so the
two can never disagree and delete what was just written; "everything" skips
pruning entirely rather than falling back to some default bound. The per-pass
ceiling scales with the window (500/30d, hard cap 20k) because 500 messages is
right for a month and nonsense for "everything". Email/query now omits the
`after` filter entirely when unbounded — Stalwart rejects a malformed filter
rather than treating `undefined` as unset.
RECENCY (A2). Keyword search structurally cannot answer a question about WHEN:
bm25 ranks by term overlap, so "who sent the last email" matches documents
containing the word "last", and "all mails in July" matches documents
containing "July" — not documents dated in July. Both were asked by a real
user and both failed. New lib/mail-index/recency.ts detects time intent
(English + German, since the UI ships German) and turns it into a date RANGE;
new MailIndex.recent() answers it with an ordered scan over the already-indexed
`occurred_at`. The route ADDS these hits to the keyword hits rather than
replacing them — "what did the last mail from Anna say" is both kinds of
question at once.
Timezone subtlety worth knowing: bounds are built from LOCAL calendar
boundaries and serialised as UTC instants, so "July" covers the user's July.
A mail at 00:30 local on 1 July belongs to it even though its stored UTC
timestamp reads 30 June. My first test asserted the ISO string prefix, which
would have enshrined the opposite and passed only in UTC — the tests now
assert the local-time property instead.
SCOPE, stated by the product owner and now enforced structurally: the
assistant only ever sees the mailbox the user is signed in to. Both retrieval
legs resolve the active account (local leg by cookie slot, server leg by the
session's own JMAP account); there is deliberately no fan-out across connected
or shared mailboxes, and adding one would be a policy change, not a feature.
Gate: tsc clean, eslint clean, 2520/2520 tests (8 new for recency intent), build clean.
Three things, all from running the real thing rather than trusting a status code.
1. OpenCode as a 4th AI class (lib/ai/opencode.ts + app/api/ai/opencode/*).
A locally-running `opencode serve` — the same runtime Paperclip drives as
an adapter. Its appeal over a BYOK profile is precisely what was broken
before: opencode owns provider auth itself, so there is NO api key for
this app to hold, and it reports a REAL model list (25 on this machine)
instead of asking the user to type an exact provider-specific model id
from memory. Typing "Sonnet 5" into a free-text box and getting a bare
"Provider returned 401" is the failure this removes.
IMPORTANT trap, documented in the module header and pinned by a test:
opencode is NOT OpenAI-compatible. `/v1/models` and `/v1/chat/completions`
both answer 200 — because a web-UI catch-all serves index.html for ANY
unknown path. I built the first version against that assumed compatibility
on the strength of two 200s and had to throw it away once I read a body.
Every probe now validates the parsed shape and content-type, never the
status alone. The real API is GET /api/model + POST /session +
POST /session/{id}/message, and the reply's `reasoning` parts are stripped
so a model's private chain of thought can never surface as the answer.
Proxied through our own backend (like the `server` class) because the
desktop renderer's origin is a random port that changes every launch;
same-origin sidesteps opencode's CORS allowlist entirely. Loopback-only by
construction: a non-loopback OPENCODE_BASE_URL is refused, since "local,
no keys, nothing leaves the device" is the whole point of this class.
2. Retrieval read the WRONG ACCOUNT'S index. The indexer writes under the
active account's cookie slot (catchUpIndex passes it) but fetchLocalLeg
omitted `?slot=`, so search resolved to whichever account the multi-slot
resolver found first. Single-account installs never noticed; a real
multi-account/shared-mailbox setup reads an empty store every time. Both
call sites now pass the active slot.
3. "No local mail index available in this session" was shown even when the
index existed and simply matched nothing — actively misleading, and it
masked the missing-SESSION_SECRET bug for hours. AskResult now carries
retrievalState ('augmented' | 'no-match' | 'no-index') and the two cases
get different words: build the index, versus rephrase (with the honest
caveat that keyword search answers content questions better than recency
ones like "the last mail").
Verified live against real opencode 1.18.14: discovery found 25 models and a
real prompt round-tripped the exact expected answer through the real helper
code, not curl. Gate: tsc clean, eslint clean, 2512/2512 unit tests (10 new,
incl. one that fails if the HTML catch-all is ever accepted as an API), build clean.
A previous packaged .app under dist-electron-builds/ carries its own
data/ tree, and output tracing tried to copy pieces of the OLD app into
the NEW standalone output during dist:mac ("Failed to copy traced
files..." warnings). Zero bytes actually leaked (the copies ENOENT'd),
but the failure mode — yesterday's build inside today's artifact — is
bad enough to fence off explicitly, same as ./repos already was.
Root cause of "No local mail index available in this session" on a real
mailbox in the packaged .app, found by probing the live packaged build:
getSessionSecret() has four sources (env, env file, wizard config,
config file) and the desktop shell provided NONE — getDesktopDefaults()
sets JMAP_SERVER_URL (which also skips the setup wizard that would have
persisted a secret) but never a SESSION_SECRET. So every login's POST
/api/auth/stalwart-context 500'd, the jmap_stalwart_ctx cookie was never
minted, and every server-side-identity feature 401'd forever: encrypted
local index, offline replica, S/MIME enrolment, AI server class. The AI
retrieval leg renders any non-OK as "no local index", so the failure was
completely silent. Every test had masked this by injecting its own
SESSION_SECRET into the child env.
Fix 1 — electron/main.ts ensureSessionSecretFile(): a 64-hex-char secret
generated once per install, persisted 0600 under userData, handed to the
server as SESSION_SECRET_FILE (value stays out of the env block; an
operator-provided SESSION_SECRET env var still wins by resolution order).
Fix 2 — page.tsx boot catch-up now RETRIES (4s/20s/60s) instead of one
silent shot: the first attempt races login's own auth-context POST, and a
401 on that race used to mean an empty index until the next app restart.
requestIndex() already separates permanent (404/503 unavailable) from
retryable failures, so the retry is cheap and self-limiting.
Fix 3 — new components/ai/ai-ask-button.tsx: the AI Assistant finally has
an entry point in the MAIN mail view (Sparkles button next to the search
filter) opening a compact Ask dialog — same askMail client, same persisted
provider settings as the Settings pane. When nothing is configured it
deep-links to Settings → AI Assistant, where local-discovery's one-click
Connect does setup.
e2e hardened to prove the whole thing honestly: SESSION_SECRET explicitly
EMPTY in the launch env (the per-install secret must carry auth), the
manual sync/reindex calls removed (the automatic boot catch-up must build
the index on its own — polled, not triggered), and the toolbar entry
point asserted. Passing: auto-built index, discovery banner, Connect, and
a grounded answer citing the one email containing the fact.
Gate: tsc clean, eslint clean, 2502/2502 unit tests, e2e passing.
Found by a new real Electron e2e test built specifically to prove the
`local` AI class genuinely works end-to-end in the packaged desktop shell:
a real local Ollama model answering a real question, grounded in the real
encrypted SQLite/FTS5 mail index — not a browser tab, not a mock.
First run surfaced a genuine bug: toFtsMatchQuery() AND-joins every token,
which is right for a deliberate search-box query but wrong for the natural-
language questions the AI retrieval surface (/api/offline/search — see its
own module header, "THE RETRIEVAL SURFACE") actually receives. "When is
check-in for the Villa sul Lago booking, and what time?" shares almost none
of its own function words with the email that answers it, so ANDing every
token — including "when"/"is"/"for"/"the"/"and"/"what" — returned 0 hits
against an index that correctly returns the right email for "Villa sul Lago
check-in".
Fix: new toFtsMatchQueryAny() (lib/mail-index/store.ts) — drops a small,
well-known English stop-word list, OR-joins what's left, and lets the
existing bm25 ranking pick the winner among partial matches. Deliberately a
NEW function, not a change to toFtsMatchQuery itself: that one's own tests
rely on "AND"/"OR"/"NOT" surviving verbatim as literal search terms
(FTS5-keyword-injection safety) — a different guarantee than this one's job
of turning a question into a good search. search() gains a `mode: 'and' |
'any'` option (default 'and', so every existing caller is unaffected); the
offline-search route passes 'any', since its one real caller is exactly
this AI-question shape.
Also added, to make the e2e test possible at all: electron/main.ts's
VNCMAIL_TEST_FIXED_PORT — a narrow, off-by-default escape hatch so
DEV_MOCK_JMAP's JMAP_SERVER_URL can point at this same standalone server's
own /api/dev-jmap. Needed because the encrypted index's key channel
(fd-3/safeStorage) only gets wired up in startStandaloneServer()'s own
random-port launch path, never when ELECTRON_LOAD_URL bypasses it for a
plain `next dev` target — so this was the only way to exercise the real
index without a full Stalwart+SMTP Docker fixture.
Verified live in the real packaged Electron shell, not just unit tests:
real dev-mode login, real multi-round /api/offline/sync + /api/offline/reindex
(39 mail/35 calendar/23 contacts indexed), real local-discovery banner
(11 real Ollama models on this machine), real "Connect", a real question
through the real Settings UI, a real direct renderer->Ollama /api/chat call
(confirmed via network log, never proxied through this app's backend), and
the model's own answer citing the exact right fact: "Saturday 28 March at
15:00" — a fact that exists nowhere except in the one indexed email.
4 new unit tests for toFtsMatchQueryAny. Full gate: tsc clean, eslint
clean, 2502/2502 tests passing, build clean, e2e/electron-ai-local-index.spec.ts
passing against the real standalone server + real Electron + real Ollama.
New lib/ai/local-discovery.ts: one /api/tags query against the loopback
addresses Ollama binds to (127.0.0.1/localhost), no follow-up /api/show
round trips needed — the tags response already carries capabilities, size,
and parameter_size, enough to recommend a default model. Picks the
smallest non-"thinking" chat-capable model for the fastest first response
("Connect" pre-fills provider+baseUrl+model in one click), and separately
surfaces the largest as a "higher quality" alternative.
New banner in ai-assistant-settings.tsx: fires when Local isn't yet
configured, offers one-click Connect or a persisted "Not now" dismissal.
13 new unit tests using this machine's actual Ollama /api/tags response
(11 real installed models — qwen2.5:32b, llama3.2, deepseek-r1 x2,
gemma4 x3, hermes3, qwen3, qwen3.5, nomic-embed-text) as literal fixtures,
per the explicit instruction to use this machine as the test case:
confirms exactly one query is required, the heuristic recommends
llama3.2:latest (fastest) / qwen2.5:32b (largest) on this real fleet,
never recommends an embedding-only model, and degrades correctly when a
candidate base URL is unreachable.
Full QA gate: tsc clean, eslint clean, 2498/2498 tests passing, build clean.
Also live-verified in a real browser session against this machine's real
Ollama — the banner rendered with exactly these two model names.
The bump-dev (and identically-configured bump-prod) job failed during
prepare_script with:
ERROR: Job failed: prepare environment: waiting for pod running:
pulling image "alpine/git:2.47.0": image pull failed: ... not found
Root cause: alpine/git:2.47.0 does not exist on Docker Hub. The alpine/git
2.47.x line starts at 2.47.1 — there is no 2.47.0 build. The runner's
image pull correctly fails with 'not found', and GitLab's Kubernetes
executor treats an image-pull failure during prepare_script as fatal, so
the job never reaches its script block.
Fix: pin both bump-dev and bump-prod to alpine/git:2.47.2 (latest 2.47.x).
Pinned rather than 'latest' so the job stays reproducible. bump-prod had
the same nonexistent tag and would have hit the identical failure on its
next run (whenever main advances), so both are fixed together.
The build job failed with:
Cannot connect to the Docker daemon at tcp://localhost:2375.
Is the docker daemon running?
Three things were wrong:
1. DOCKER_HOST was set to tcp://localhost:2375. The DinD daemon runs in
the service sidecar container, not in the build container, so
localhost was always going to refuse the connection. The correct host
is the service alias 'docker'.
2. The docker:28.4.0-dind service was declared without an explicit
alias. Without alias: docker, GitLab derives the hostname from the
image string 'docker:28.4.0-dind', and since ':' is invalid in DNS,
the 'docker' hostname never resolves. The explicit alias is required
for tcp://docker:2375 to work at all.
3. docker:28.4.0-dind enables TLS by default and listens on 2376, but
DOCKER_HOST points at 2375. Setting DOCKER_TLS_CERTDIR="" disables
TLS so the daemon listens on plaintext 2375, matching DOCKER_HOST.
This mirrors the known-working pattern in the vnc-localidp pipeline
(docker:20.10.17-dind + alias: docker + DOCKER_HOST=tcp://docker:2375
+ DOCKER_TLS_CERTDIR=""). The TLS-defaults behavior has been unchanged
since docker 19.03, so the same pattern applies on 28.4.0.
DOCKER_TLS_CERTDIR forced the docker:28.4.0-dind service to listen on
port 2376 with TLS, but DOCKER_HOST pointed to the non-TLS port 2375.
This caused the docker client to loop forever with:
Cannot connect to the Docker daemon at tcp://localhost:2375.
Removing DOCKER_TLS_CERTDIR lets the daemon listen on 2375 again,
matching DOCKER_HOST, restoring docker-in-docker connectivity.
Both were "known gaps" in the original doc; rewrite those sections to
reflect the actual shipped, live-verified state and note what's newly
open instead (BYOK allow-list is advisory-only, consent text has no
client-side reader yet).
New enroll.js: generates an RSA-2048 keypair with WebCrypto (extractable
only long enough to export to PKCS#8), builds and signs a real CSR with
pkijs (same per-call-engine convention as smime-sign.js/smime-verify.js —
nativeEngine() passed explicitly, no global pkijs.setEngine call), POSTs
it to the already-existing /api/smime/enroll (same-origin fetch — the
plugin's privileged tier gets allow-same-origin, cookies included by
default), and packages the result into a key record using the EXACT same
encrypted-at-rest convention as a PKCS#12 import (AES-GCM/PBKDF2 600k,
exported from pkcs12.js) so every downstream sign/encrypt/decrypt/verify
path is identical regardless of how the key arrived.
New "Get a certificate" button in the settings-section UI, next to
"Import key" — prompts for a storage passphrase, calls enroll(), saves
the key record, and refreshes the list. No changes needed to the CA route
or the CA provider — both were already real and already tested.
Live end-to-end verified (not just unit-level): logged in via the real
dev-mode session flow, clicked through the actual plugin UI, got back a
real certificate (RSA-2048, correct validity window, real fingerprint) for
dev@localhost, then unlocked it with the same passphrase — the encrypted
private key round-trips correctly through the identical code path a
PKCS#12 import would use.
Also fixes a real bug hit during that verification: SESSION_SECRET must be
>= 32 chars (lib/auth/crypto.ts), but .env.dev.example's own documented
placeholder was 29 - failing "Failed to store Stalwart auth context" on
every feature needing the real session-cookie flow (this enrolment route,
offline sync, AI server class). Anyone following the setup doc verbatim
would have hit this. Padded the placeholder to 37 chars.
New admin tab "AI" (app/(main)/admin/_tabs/ai-policy.tsx): provider-class
toggles, server model allow-list, BYOK provider allow-list, seats/usage
(front-end for the already-real lib/ai/entitlement.ts), retrieval on/off,
consent text + version bump.
Real backend, not cosmetic: AiConsoleConfig persisted via config-manager
(lib/ai/types.ts, ai-policy.json in the CONFIG dir). New GET/PUT
/api/admin/ai/policy. Enforcement wired at every real chokepoint, not just
the picker: /api/ai/server/chat checks classesEnabled.server and the model
allow-list, /api/ai/retrieve checks retrievalEnabled, /api/ai/server/models
filters by allow-list. GET /api/ai/policy folds classesEnabled into the
classes list clients see.
Resolved the spec's 3 open questions as recommended: BYOK allow-list stays
client-side/advisory (wired into ai-assistant-settings.tsx's addProfile),
tier picker stays cosmetic, master aiAssistantEnabled toggle stays in the
existing Policy tab (this tab links to it instead of duplicating it).
Defaults preserve today's behavior exactly (classesEnabled/allowlists all
start empty/null) — turning this on changes nothing until an admin touches it.
Concrete, runnable test steps per area with explicit known-gaps sections
so nothing reads as more finished than it is. Also documents the mobile
S/MIME merge (vncmail-native main 7b89839) done this session.
.env.dev.example: the documented relative JMAP_SERVER_URL 400s
/api/auth/stalwart-context (resolveTrustedJmapUrl rejects relative URLs),
silently breaking the real session-cookie flow that S/MIME enrollment,
offline sync, and the AI server/retrieval routes all depend on. Switched
the example to an absolute URL with an explanatory comment.
Documents the 7 real gaps (per-class enable, model/provider allow-lists,
seats/usage UI, retrieval off-switch, consent) against the existing
entitlement.ts/policy.tsx backend, proposed AiConsoleConfig schema, new
endpoints, and a 6-section UI layout. Companion visual mockup presented
separately. No application code changed — spec + mockup only, as instructed.
The production EJBCA needs a client mTLS certificate + password this
session doesn't have, and lives on the private dev-k8s network - genuinely
unreachable from here tonight (confirmed, not assumed - see
lib/smime-ca/index.ts's build() and the memory on the EJBCA CA project).
lib/smime-ca/local-dev-provider.ts implements the same CaProvider seam
(lib/smime-ca/types.ts) the production EjbcaProvider does - a real,
working local CA, not a mock:
- Generates a real RSA-2048 self-signed root on first use, persisted to
the admin state dir (same pattern as lib/ai/entitlement.ts).
- enroll() parses a real PKCS#10 CSR (pkijs), verifies its self-signature
(proof of possession - not identity, which still comes only from the
server-provided `addresses`, exactly like the production provider),
and issues a real X.509v3 leaf: BasicConstraints(cA:false), KeyUsage
(digitalSignature|nonRepudiation|keyEncipherment), ExtKeyUsage
(emailProtection), SubjectAltName(rfc822Name per address) - signed with
the CA's own private key.
- revoke()/getChain() implemented for real (persisted revocation list,
real chain PEM).
- Wired into build() behind SMIME_CA_DEV_LOCAL=true, explicit opt-in only,
never a silent fallback when the real CA URL is simply unconfigured.
4 tests, all real cryptographic verification, not string-shape checks:
issue a cert from a real WebCrypto-generated CSR, then cryptographically
verify the chain (leaf.verify(caCert) === true) and confirm the SAN
contains exactly the server-chosen addresses (never the CSR's own
requested CN); reject a CSR with a corrupted signature; confirm the CA
persists across calls rather than minting a new root each time; confirm
revocation is recorded to disk.
Scope note, explicit rather than silently incomplete: this closes the
server-side half. The client-side half (C-08) - the plugin generating a
CSR via WebCrypto, calling this enrollment endpoint, and importing the
issued cert into its existing encrypted-at-rest key storage
(vnc/plugins/smime/src/key-storage.js, matching the AES-GCM+PBKDF2(600k)
wrapping pkcs12.js already uses for imports) - was NOT built tonight.
That plugin has open findings from an earlier security audit (see project
memory); adding new key-generation/storage code to it at 00:30 after many
hours of continuous work is exactly the kind of rushed change that
produces the next finding. The privileged iframe can reach
/api/smime/enroll directly (same-origin, confirmed via the plugin's own
tier=privileged log line - no new sandbox bridge capability needed), so
the remaining work is well-scoped and mechanical, not blocked on any open
question - just deliberately deferred to unhurried, focused time.
Verified: typecheck clean, lint clean, full vitest suite 2484/2485 (only
the pre-existing, unrelated jmap-client-resilience flake), production
build succeeds.
Full retrieval pipeline per docs/AI-ASSISTANT-CONCEPT.md §7/§8.1, real end
to end, not mocked:
- lib/ai/retrieval/types.ts: SourceRef + RetrieverAdapter schema. Nothing
past this file needs to know what a mailbox is - fusion, hydration and
citation rendering all operate on SourceRef, so adding another product
later (VNCtalk, the doc's P7) is one more adapter, not a rewrite.
- lib/ai/retrieval/fusion.ts: Reciprocal Rank Fusion, score = Σ1/(k+rank),
k=60. Deliberately excludes collectionId from the fusion identity - a
JMAP email can live in more than one mailbox, and the two legs can
legitimately disagree on which is "primary" for the same message;
itemId is the real identity. 5 unit tests, including that exact
double-count case.
- lib/ai/retrieval/mail-embeddings.ts: the server embedding leg. Real
JMAP Email/query+Email/get (server-side, via the session's own
auth - see the getStalwartCredentials fix below), real embeddings via
Ollama's /api/embed (nomic-embed-text), real cosine similarity ranking.
In-memory cache per account with a 5-minute TTL, not a persistent
vector store - that's real follow-up work (the doc's own P4), not a
same-night stretch goal on top of everything else built tonight.
- app/api/ai/retrieve/route.ts: wires it together. ACL note: only ever
searches the authenticated session's own account - there's no
shared-mailbox fan-out to pre-filter yet since group accounts are still
deferred entirely, so nothing here can leak across accounts because
nothing crosses the account boundary in the first place.
- lib/ai/local-client.ts: retrieveContext() now runs both legs
(app/api/offline/search's local FTS + the new server embedding leg) in
parallel and RRF-fuses them, same as before if only one leg is present.
Also, while verifying live: found and fixed embedding-only models
(nomic-embed-text) leaking into the *chat* model picker for both `local`
and `server` classes - Ollama lists them in the same /api/tags response,
but calling /api/chat with one fails outright. Filtered by `capabilities`
(fails open if absent, for older Ollama).
Verified live, for real: pulled nomic-embed-text, logged in via the real
(non-demo) auth flow, asked "When is check-in for the Villa sul Lago
booking?" against the seeded mock inbox - got back "Check-in ... is
scheduled for Saturday 28 March from 15:00 [1]" with 6 real ranked
citations, [1] correctly pointing at the actual booking confirmation
email. Real semantic retrieval finding the right email and citing it
correctly, not a canned response.
Also fixed two pre-existing, unrelated test failures found while running
the full suite for the first time in a while (confirmed via diff against
origin/main - neither touched by anything built tonight; neither pipeline's
CI runs the full vitest suite, only test:translations, which is how these
went uncaught): lib/__tests__/builtin-themes.test.ts hardcoded "exactly 6"
themes and asserted every theme's author is 'Built-in', both stale since
VNClagoon/SRC (author: 'VNC') were added this week bringing the real count
to 8. Left the also-pre-existing, timing-sensitive
jmap-client-resilience.test.ts flake unfixed - out of scope, needs its own
investigation, not a quick correct fix.
Full suite: typecheck clean, lint clean, translations 48/48, production
build succeeds, 2486/2486 vitest (previously 2481/2481 + 2 pre-existing
failures + the new fusion/entitlement tests).
Three pieces built together tonight since they're naturally linked (the
server-class proxy is the real entitlement enforcement chokepoint):
1. Multi-key BYOK (public class): several named provider profiles
(name/baseUrl/model), each with its own key in lib/ai/key-store.ts
(keyed by profile id, not a single fixed 'public' slot). The "Try it"
pane lets you pick which saved profile answers each question - not one
fixed default.
2. `server` class, real: app/api/ai/server/{models,chat} proxy through
this app's own backend to AI_SERVER_BASE_URL - same-origin from the
browser, no CORS/OLLAMA_ORIGINS story at all, standing in tonight for
VNC's EU/CH-hosted infra with the real Ollama on this Mac (swapping to
the real instance tomorrow is a config change).
3. Real entitlement enforcement (lib/ai/entitlement.ts), scoped to `server`
only (not local/public, per the 2026-08-05 decisions): checkAndAssignSeat()
re-validates on every /api/ai/server/chat call - first use auto-assigns a
seat if any remain, further calls from an unlicensed user get a 402 with
a specific reason. recordUsage() appends to an append-only metering
ledger (timestamp/user/model/tokens/latency) that IS the billing record.
Admin data endpoints at /api/admin/ai/entitlement (seat total, revoke) -
the visual admin console is a separate, not-yet-built task.
Two real bugs found and fixed during verification, not just claimed fixed:
- /api/ai/policy never actually added 'server' to entitlement.classes even
when AI_SERVER_BASE_URL was set (only the type comment was updated) - the
Server radio option silently never appeared until this was caught live.
- The new routes used readStalwartAuthContext(0) (hardcoded slot, SSO/reauth-
specific) instead of getStalwartCredentials() (the general multi-slot
session resolver every other authenticated route uses) - reachable but
wrong, and would have hidden a real auth gap behind "works on my slot".
Verified end-to-end for real: built + ran the actual server, logged in via
the real (non-demo) auth flow, selected Server, listed the real Ollama
models through the proxy, asked "Reply with exactly the words: SERVER CLASS
WORKS" and got back exactly that - plus confirmed on disk (not just in the
UI) that data/admin-state/ai-entitlement.json recorded the seat assignment
and ai-metering.jsonl recorded real prompt/completion token counts and
latency from the actual model call. Rejection-path logic (seat limit
reached, zero seats configured, revocation) covered by 5 new unit tests
rather than a second live round trip. Full suite: typecheck clean, lint
clean, translations 48/48, production build succeeds.
QA pass on the encrypted mail index found two real gaps beyond what the
prior end-to-end fix pass caught:
1. store.ts's MailIndex.open() only wrapped SOME of the post-key pragma
calls in a try/catch before this: the first attempt's `key` pragma and
assertEncrypted() ran outside any try at all, and the wrong-key retry
repeated the same gap. Any pragma throwing there (SQLITE_BUSY, a full
disk on the first WAL write) leaked the native SQLite handle instead of
closing it. Factored the open+key+verify+pragma sequence into openKeyed(),
which guarantees a close before rethrowing on any failure, and reused it
for both the first attempt and the retry.
2. reindex.ts never removed a deleted contact or file from the index. The
`removed` field exists in the API and is fully tested at the store layer,
but nothing in the renderer populates it, so a deleted contact/file stayed
searchable - and retrievable by the AI feature - indefinitely. Mail and
calendar can't use the same fix (their queries are date-windowed, so an id
missing from one fetch may just be outside the window), but contacts/files
have no date filter - a catch-up fetch that comes back under its cap IS
the complete set, so anything locally indexed but absent from it is safely
known to be deleted. Added strayIdsAfterCatchUp() and wired it into the
catch-up path for those two types only.
Also read binding.ts, key.ts, paths.ts, jmap.ts, extract.ts, the FTS5
query builder, and both /api/offline/{search,reindex} routes end to end;
no other concrete bugs found there. Full findings reported separately.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Product decision 2026-08-05: exactly SRC (default) and VNClagoon ship as
selectable themes. Everything else (Qui, Nord, Catppuccin, Solarized,
Roundcube Elastic, Aurora Glass) is hidden via ThemePolicy.disabledBuiltinThemes
rather than deleted - cheap to re-enable later, zero code lost. Also removed
the hardcoded "Default/Bulwark" theme card from the Settings > Themes grid
(product wants exactly 2 theme choices, not 3).
Separately, found and fixed real "still says Bulwark" branding gaps:
- All 24 locale files: "Bulwark"/"Bulwark Mail"/"Bulwark Webmail" ->
"VNCmail+" in every user-facing string (verified via the translations
test, which only checks structural key parity across locales, not
content - a straight string swap is safe against it. 48/48 still pass).
- PWA manifest fallback app name, package.json description/author.
- Demo mode fixtures: the "Welcome to Bulwark Mail!" email and identity
signature a first-time demo user actually sees.
- The demo empty-state's logo was hardcoded to a literal Bulwark SVG file
regardless of active theme - a real bug, not just stale text, since
lib/theme-logo.ts's resolveThemeLogo() already exists and is already used
correctly by the login page and nav rail for exactly this (theme-aware
SRC mark / VNClagoon wordmark). Wired the same helper in here instead of
a hardcoded path.
NOT touched: internal code comments referencing "Bulwark" as historical/
attribution context (e.g. design-rationale comments explaining why a value
differs from Bulwark's original) - those are harmless and a full-repo sweep
of every comment wasn't the ask.
Verified: typecheck clean, lint clean, translations test 48/48 passing.
Tonight's GitHub Actions runs (Publish Docker Image workflow) succeeded on
every commit, confirmed by pulling the manifest directly: sha-147660a
exists in ghcr.io/brvncde-dotcom/vncmail-plus-dev and its digest matches
the `latest` tag exactly. Unlike the prior sha-d0a1cee6 pin (built locally,
manually `ctr images import`-ed onto each node - see 9b5870ca), this tag is
a real, publicly pullable registry image: no side-loading needed, survives
a node rebuild, and includes everything through tonight's AI work
(P0 scaffolding, real local-Ollama wiring, the CSP fix that made it
actually reachable, aiAssistantEnabled defaulting on).
Does NOT deploy anything by itself - ArgoCD's vncmail-dev Application is
still manual-sync (see deploy/argocd/vncmail-dev-app.yaml), and this
session has no kubectl/cluster access to trigger that sync or verify the
rollout. Whoever next syncs (or restarts the deployment) picks this up
automatically via imagePullPolicy: IfNotPresent, which now works as a real
cache rather than a hard dependency on the side-loaded image.
Real end-to-end verification (Playwright-driven real Electron app on this
Mac, against the actual local Ollama instance, not a mock) found the actual
blocker: production's connect-src CSP (`'self' https: wss:`) rejects plain
http:// entirely, so lib/ai/local-client.ts's loopback fetch to Ollama never
even attempted the network in a production build - Electron or browser
alike. This is almost certainly what looked like a browser-sandbox network
issue in the earlier (non-Electron) QA pass tonight too.
Fix is narrow, not a blanket http: relaxation: connect-src now additionally
allows `http://127.0.0.1:*` and `http://localhost:*` specifically. Loopback
has no network hop, so it doesn't reopen the mixed-content-style downgrade
risk the existing https-only production policy guards against - unlike
dev's blanket `http:` allowance, which stays dev-only.
Confirmed fixed: rebuilt (build:standalone + build:electron), launched the
real Electron app via Playwright's _electron, and got a genuine answer back
from the real local Ollama - "Test connection" showed Reachable (the real
success state, not the CORS-diagnostic fallback text), and asking "Reply
with exactly the words: LOCAL AI WORKS" returned exactly that, with the
correct "no local mail index in this session" banner alongside it (accurate
for a fresh Electron session with nothing synced yet).
Also flips FeatureGates.aiAssistantEnabled's default false->true: local now
genuinely works and ships free/unmetered (see lib/ai/types.ts), so there is
a real feature behind the tab, not an empty preview - matches tonight's
explicit "I want AI visible" instruction. An admin can still turn it off.
Verified: typecheck clean, lint clean (0 errors, pre-existing warnings
only), translations pass (48/48), full production build succeeds.
Decisions 2026-08-05 evening (reprioritizing docs/AI-ASSISTANT-CONCEPT.md's
original P1/P2 server-first sequencing to local-first, since a real Ollama
instance already runs on this Mac with a full model set):
- `local` ships free, no entitlement check — always available wherever
supportsLocalLlm() is true.
- `public` (BYOK) is available too, explicitly unmonitored for now — no
seats/metering/consent backend. This reverses the concept doc's decision
#1 (server-side-only key custody): the client holds its own key, matching
vncmail-native's existing pattern.
- `server` (VNC-hosted) stays unwired client-side; that infra is "this
MacBook tonight, the dev k8s cluster tomorrow."
New:
- lib/ai/local-client.ts: listLocalModels/testLocalConnection/chatLocal/
chatPublic, ported near-verbatim from vncmail-native's proven
src/api/ai.ts. Direct browser-side fetch, not proxied through this app's
own server — a server-side proxy would reach the *server's* loopback, not
the user's own laptop, which defeats the point of "local" once this app
is hosted remotely.
- lib/ai/key-store.ts: client-held BYOK storage (localStorage — this repo's
existing convention for client state, no OS keychain reachable from a
browser tab).
- lib/ai/local-settings.ts: isolated persistence for provider/model/base-URL
choices. Deliberately NOT folded into stores/settings-store.ts, which has
a hand-maintained export/import enumeration this prototype-scope state
doesn't belong in yet.
- Retrieval reuses this app's own already-built app/api/offline/search
(encrypted SQLite/FTS5 mail index) as context when available, and
degrades to unaugmented chat — not an error — when it 404s/503s (no index
in this session, e.g. plain browser rather than Electron).
Rewrote components/settings/ai-assistant-settings.tsx: provider picker,
local runtime config (base URL, model list/refresh, test connection with a
CORS-aware diagnostic per the concept doc's own note on the browser row),
public BYOK config (base URL, model, key, client-side consent toggle), and
a working Ask box.
Verified: typecheck clean, lint clean, translations pass, production build
succeeds. Live-tested against the real Ollama on this machine (confirmed
running: qwen2.5:32b, gemma4, deepseek-r1, llama3.2, hermes3, qwen3) via a
local server + demo-mode session — admin flag round-trips correctly, the
pane renders both provider options, and the CORS-diagnostic path fires
correctly on a real (if here environment-sandboxed, not Ollama-side)
connection failure. Full success end-to-end still wants a real, unsandboxed
browser tab against this Mac's loopback to close out.
Stefan-Sanger