Commit Graph
1539 Commits
Author SHA1 Message Date
Bernd Rodler 7b2047681e fix(mail-index): local AI retrieval returned 0 hits for real questions — AND-every-token FTS matching killed on stop words
Found by a new real Electron e2e test built specifically to prove the
`local` AI class genuinely works end-to-end in the packaged desktop shell:
a real local Ollama model answering a real question, grounded in the real
encrypted SQLite/FTS5 mail index — not a browser tab, not a mock.

First run surfaced a genuine bug: toFtsMatchQuery() AND-joins every token,
which is right for a deliberate search-box query but wrong for the natural-
language questions the AI retrieval surface (/api/offline/search — see its
own module header, "THE RETRIEVAL SURFACE") actually receives. "When is
check-in for the Villa sul Lago booking, and what time?" shares almost none
of its own function words with the email that answers it, so ANDing every
token — including "when"/"is"/"for"/"the"/"and"/"what" — returned 0 hits
against an index that correctly returns the right email for "Villa sul Lago
check-in".

Fix: new toFtsMatchQueryAny() (lib/mail-index/store.ts) — drops a small,
well-known English stop-word list, OR-joins what's left, and lets the
existing bm25 ranking pick the winner among partial matches. Deliberately a
NEW function, not a change to toFtsMatchQuery itself: that one's own tests
rely on "AND"/"OR"/"NOT" surviving verbatim as literal search terms
(FTS5-keyword-injection safety) — a different guarantee than this one's job
of turning a question into a good search. search() gains a `mode: 'and' |
'any'` option (default 'and', so every existing caller is unaffected); the
offline-search route passes 'any', since its one real caller is exactly
this AI-question shape.

Also added, to make the e2e test possible at all: electron/main.ts's
VNCMAIL_TEST_FIXED_PORT — a narrow, off-by-default escape hatch so
DEV_MOCK_JMAP's JMAP_SERVER_URL can point at this same standalone server's
own /api/dev-jmap. Needed because the encrypted index's key channel
(fd-3/safeStorage) only gets wired up in startStandaloneServer()'s own
random-port launch path, never when ELECTRON_LOAD_URL bypasses it for a
plain `next dev` target — so this was the only way to exercise the real
index without a full Stalwart+SMTP Docker fixture.

Verified live in the real packaged Electron shell, not just unit tests:
real dev-mode login, real multi-round /api/offline/sync + /api/offline/reindex
(39 mail/35 calendar/23 contacts indexed), real local-discovery banner
(11 real Ollama models on this machine), real "Connect", a real question
through the real Settings UI, a real direct renderer->Ollama /api/chat call
(confirmed via network log, never proxied through this app's backend), and
the model's own answer citing the exact right fact: "Saturday 28 March at
15:00" — a fact that exists nowhere except in the one indexed email.

4 new unit tests for toFtsMatchQueryAny. Full gate: tsc clean, eslint
clean, 2502/2502 tests passing, build clean, e2e/electron-ai-local-index.spec.ts
passing against the real standalone server + real Electron + real Ollama.
2026-08-06 18:09:14 +02:00
vncmail-ci 2aae8ed842 chore(deploy): pin dev to sha-2a8778c9 [skip ci] 2026-08-06 15:47:09 +00:00
Bernd Rodler 2a8778c905 merge: reconcile with GitLab CI fixes landed concurrently 2026-08-06 17:42:49 +02:00
Bernd Rodler 2dc224e882 feat(ai): local LLM auto-discovery — find a running Ollama, suggest connecting
New lib/ai/local-discovery.ts: one /api/tags query against the loopback
addresses Ollama binds to (127.0.0.1/localhost), no follow-up /api/show
round trips needed — the tags response already carries capabilities, size,
and parameter_size, enough to recommend a default model. Picks the
smallest non-"thinking" chat-capable model for the fastest first response
("Connect" pre-fills provider+baseUrl+model in one click), and separately
surfaces the largest as a "higher quality" alternative.

New banner in ai-assistant-settings.tsx: fires when Local isn't yet
configured, offers one-click Connect or a persisted "Not now" dismissal.

13 new unit tests using this machine's actual Ollama /api/tags response
(11 real installed models — qwen2.5:32b, llama3.2, deepseek-r1 x2,
gemma4 x3, hermes3, qwen3, qwen3.5, nomic-embed-text) as literal fixtures,
per the explicit instruction to use this machine as the test case:
confirms exactly one query is required, the heuristic recommends
llama3.2:latest (fastest) / qwen2.5:32b (largest) on this real fleet,
never recommends an embedding-only model, and degrades correctly when a
candidate base URL is unreachable.

Full QA gate: tsc clean, eslint clean, 2498/2498 tests passing, build clean.
Also live-verified in a real browser session against this machine's real
Ollama — the banner rendered with exactly these two model names.
2026-08-06 17:41:49 +02:00
vncmail-ci 3d5c05ffb9 chore(deploy): pin dev to sha-f46614a4 [skip ci] 2026-08-06 10:32:27 +00:00
Stefan-Sanger f46614a483 Merge branch 'fix-bump-stages' into 'dev'
fix(ci): use alpine/git:2.47.2 — 2.47.0 was never published

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!13
2026-08-06 09:48:30 +00:00
Stefan-Sanger 30d6c23908 fix(ci): use alpine/git:2.47.2 — 2.47.0 was never published
The bump-dev (and identically-configured bump-prod) job failed during
prepare_script with:

  ERROR: Job failed: prepare environment: waiting for pod running:
  pulling image "alpine/git:2.47.0": image pull failed: ... not found

Root cause: alpine/git:2.47.0 does not exist on Docker Hub. The alpine/git
2.47.x line starts at 2.47.1 — there is no 2.47.0 build. The runner's
image pull correctly fails with 'not found', and GitLab's Kubernetes
executor treats an image-pull failure during prepare_script as fatal, so
the job never reaches its script block.

Fix: pin both bump-dev and bump-prod to alpine/git:2.47.2 (latest 2.47.x).
Pinned rather than 'latest' so the job stays reproducible. bump-prod had
the same nonexistent tag and would have hit the identical failure on its
next run (whenever main advances), so both are fixed together.
2026-08-06 11:44:05 +02:00
Stefan-Sanger bde63c1322 Merge branch 'fix-dind-confusion' into 'dev'
fix(ci): point DinD at the docker service alias, not localhost

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!12
2026-08-06 09:30:26 +00:00
Stefan-Sanger dfa5667548 fix(ci): point DinD at the docker service alias, not localhost
The build job failed with:
  Cannot connect to the Docker daemon at tcp://localhost:2375.
  Is the docker daemon running?

Three things were wrong:

1. DOCKER_HOST was set to tcp://localhost:2375. The DinD daemon runs in
   the service sidecar container, not in the build container, so
   localhost was always going to refuse the connection. The correct host
   is the service alias 'docker'.

2. The docker:28.4.0-dind service was declared without an explicit
   alias. Without alias: docker, GitLab derives the hostname from the
   image string 'docker:28.4.0-dind', and since ':' is invalid in DNS,
   the 'docker' hostname never resolves. The explicit alias is required
   for tcp://docker:2375 to work at all.

3. docker:28.4.0-dind enables TLS by default and listens on 2376, but
   DOCKER_HOST points at 2375. Setting DOCKER_TLS_CERTDIR="" disables
   TLS so the daemon listens on plaintext 2375, matching DOCKER_HOST.

This mirrors the known-working pattern in the vnc-localidp pipeline
(docker:20.10.17-dind + alias: docker + DOCKER_HOST=tcp://docker:2375
+ DOCKER_TLS_CERTDIR=""). The TLS-defaults behavior has been unchanged
since docker 19.03, so the same pattern applies on 28.4.0.
2026-08-06 11:25:33 +02:00
Stefan-Sanger 64f031201e Merge branch 'fix-ci-build2' into 'dev'
fix(ci): remove DOCKER_TLS_CERTDIR to fix DinD connection failure

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!11
2026-08-06 09:16:09 +00:00
Stefan-Sanger 77a9060c00 fix(ci): remove DOCKER_TLS_CERTDIR to fix DinD connection failure
DOCKER_TLS_CERTDIR forced the docker:28.4.0-dind service to listen on
port 2376 with TLS, but DOCKER_HOST pointed to the non-TLS port 2375.
This caused the docker client to loop forever with:
  Cannot connect to the Docker daemon at tcp://localhost:2375.

Removing DOCKER_TLS_CERTDIR lets the daemon listen on 2375 again,
matching DOCKER_HOST, restoring docker-in-docker connectivity.
2026-08-06 11:12:25 +02:00
Stefan-Sanger 7082566f51 Merge branch 'fix-ci-dev' into 'dev'
fix: dind URL

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!10
2026-08-06 09:04:19 +00:00
Stefan-Sanger cd02a9f806 fix: dind URL 2026-08-06 10:59:41 +02:00
Stefan-Sanger 59048e5ad3 Merge branch 'fix-ingress' into 'dev'
fix(deploy): switch dev to letsencrypt-prod and add HTTP->HTTPS redirect

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!8
2026-08-06 08:52:59 +00:00
Stefan-Sanger 9377684fb1 ci(build): add before_script for dind readiness and registry login 2026-08-06 10:43:13 +02:00
Stefan-Sanger 26e1f31945 ci(deploy): migrate build to docker:28.4.0-dind + GitLab container registry 2026-08-06 10:18:00 +02:00
Stefan-Sanger 611ae0624e fix(deploy): switch dev to letsencrypt-prod and add HTTP->HTTPS redirect 2026-08-06 09:50:13 +02:00
Bernd Rodler 68b0826587 promote: admin AI Policy console (built) + S/MIME web enrollment (dev→main) 2026-08-06 09:12:06 +02:00
Bernd Rodler eda3302298 docs: update test basis — admin AI console + S/MIME web enrollment now real
Both were "known gaps" in the original doc; rewrite those sections to
reflect the actual shipped, live-verified state and note what's newly
open instead (BYOK allow-list is advisory-only, consent text has no
client-side reader yet).
2026-08-06 09:08:12 +02:00
Bernd Rodler 295170a842 feat(smime): client-side certificate enrolment — web S/MIME now fully functional
New enroll.js: generates an RSA-2048 keypair with WebCrypto (extractable
only long enough to export to PKCS#8), builds and signs a real CSR with
pkijs (same per-call-engine convention as smime-sign.js/smime-verify.js —
nativeEngine() passed explicitly, no global pkijs.setEngine call), POSTs
it to the already-existing /api/smime/enroll (same-origin fetch — the
plugin's privileged tier gets allow-same-origin, cookies included by
default), and packages the result into a key record using the EXACT same
encrypted-at-rest convention as a PKCS#12 import (AES-GCM/PBKDF2 600k,
exported from pkcs12.js) so every downstream sign/encrypt/decrypt/verify
path is identical regardless of how the key arrived.

New "Get a certificate" button in the settings-section UI, next to
"Import key" — prompts for a storage passphrase, calls enroll(), saves
the key record, and refreshes the list. No changes needed to the CA route
or the CA provider — both were already real and already tested.

Live end-to-end verified (not just unit-level): logged in via the real
dev-mode session flow, clicked through the actual plugin UI, got back a
real certificate (RSA-2048, correct validity window, real fingerprint) for
dev@localhost, then unlocked it with the same passphrase — the encrypted
private key round-trips correctly through the identical code path a
PKCS#12 import would use.

Also fixes a real bug hit during that verification: SESSION_SECRET must be
>= 32 chars (lib/auth/crypto.ts), but .env.dev.example's own documented
placeholder was 29 - failing "Failed to store Stalwart auth context" on
every feature needing the real session-cookie flow (this enrolment route,
offline sync, AI server class). Anyone following the setup doc verbatim
would have hit this. Padded the placeholder to 37 chars.
2026-08-06 09:06:20 +02:00
Bernd Rodler 30e5059b94 feat(admin): build the AI Policy console (§6) — approved, spec now implemented
New admin tab "AI" (app/(main)/admin/_tabs/ai-policy.tsx): provider-class
toggles, server model allow-list, BYOK provider allow-list, seats/usage
(front-end for the already-real lib/ai/entitlement.ts), retrieval on/off,
consent text + version bump.

Real backend, not cosmetic: AiConsoleConfig persisted via config-manager
(lib/ai/types.ts, ai-policy.json in the CONFIG dir). New GET/PUT
/api/admin/ai/policy. Enforcement wired at every real chokepoint, not just
the picker: /api/ai/server/chat checks classesEnabled.server and the model
allow-list, /api/ai/retrieve checks retrievalEnabled, /api/ai/server/models
filters by allow-list. GET /api/ai/policy folds classesEnabled into the
classes list clients see.

Resolved the spec's 3 open questions as recommended: BYOK allow-list stays
client-side/advisory (wired into ai-assistant-settings.tsx's addProfile),
tier picker stays cosmetic, master aiAssistantEnabled toggle stays in the
existing Policy tab (this tab links to it instead of duplicating it).

Defaults preserve today's behavior exactly (classesEnabled/allowlists all
start empty/null) — turning this on changes nothing until an admin touches it.
2026-08-06 08:48:30 +02:00
Bernd Rodler 6fd17c2ade promote: AI (BYOK/server/entitlement/retrieval), S/MIME (web CA + mobile full stack), 2-theme rebrand (dev→main) 2026-08-06 08:27:31 +02:00
Bernd Rodler 61651b1ed1 docs: tomorrow-morning test basis (AI, S/MIME web+mobile, Theme); fix JMAP_SERVER_URL landmine in dev env example
Concrete, runnable test steps per area with explicit known-gaps sections
so nothing reads as more finished than it is. Also documents the mobile
S/MIME merge (vncmail-native main 7b89839) done this session.

.env.dev.example: the documented relative JMAP_SERVER_URL 400s
/api/auth/stalwart-context (resolveTrustedJmapUrl rejects relative URLs),
silently breaking the real session-cookie flow that S/MIME enrollment,
offline sync, and the AI server/retrieval routes all depend on. Switched
the example to an absolute URL with an explanatory comment.
2026-08-06 00:51:37 +02:00
Bernd Rodler daa40ec72d docs(ai): admin AI Policy console spec (§6) — presented for approval, not built
Documents the 7 real gaps (per-class enable, model/provider allow-lists,
seats/usage UI, retrieval off-switch, consent) against the existing
entitlement.ts/policy.tsx backend, proposed AiConsoleConfig schema, new
endpoints, and a 6-section UI layout. Companion visual mockup presented
separately. No application code changed — spec + mockup only, as instructed.
2026-08-06 00:46:30 +02:00
Bernd Rodler c2c07293b7 feat(smime): real local dev CA (LocalDevCaProvider), CSR issuance verified
The production EJBCA needs a client mTLS certificate + password this
session doesn't have, and lives on the private dev-k8s network - genuinely
unreachable from here tonight (confirmed, not assumed - see
lib/smime-ca/index.ts's build() and the memory on the EJBCA CA project).

lib/smime-ca/local-dev-provider.ts implements the same CaProvider seam
(lib/smime-ca/types.ts) the production EjbcaProvider does - a real,
working local CA, not a mock:
- Generates a real RSA-2048 self-signed root on first use, persisted to
  the admin state dir (same pattern as lib/ai/entitlement.ts).
- enroll() parses a real PKCS#10 CSR (pkijs), verifies its self-signature
  (proof of possession - not identity, which still comes only from the
  server-provided `addresses`, exactly like the production provider),
  and issues a real X.509v3 leaf: BasicConstraints(cA:false), KeyUsage
  (digitalSignature|nonRepudiation|keyEncipherment), ExtKeyUsage
  (emailProtection), SubjectAltName(rfc822Name per address) - signed with
  the CA's own private key.
- revoke()/getChain() implemented for real (persisted revocation list,
  real chain PEM).
- Wired into build() behind SMIME_CA_DEV_LOCAL=true, explicit opt-in only,
  never a silent fallback when the real CA URL is simply unconfigured.

4 tests, all real cryptographic verification, not string-shape checks:
issue a cert from a real WebCrypto-generated CSR, then cryptographically
verify the chain (leaf.verify(caCert) === true) and confirm the SAN
contains exactly the server-chosen addresses (never the CSR's own
requested CN); reject a CSR with a corrupted signature; confirm the CA
persists across calls rather than minting a new root each time; confirm
revocation is recorded to disk.

Scope note, explicit rather than silently incomplete: this closes the
server-side half. The client-side half (C-08) - the plugin generating a
CSR via WebCrypto, calling this enrollment endpoint, and importing the
issued cert into its existing encrypted-at-rest key storage
(vnc/plugins/smime/src/key-storage.js, matching the AES-GCM+PBKDF2(600k)
wrapping pkcs12.js already uses for imports) - was NOT built tonight.
That plugin has open findings from an earlier security audit (see project
memory); adding new key-generation/storage code to it at 00:30 after many
hours of continuous work is exactly the kind of rushed change that
produces the next finding. The privileged iframe can reach
/api/smime/enroll directly (same-origin, confirmed via the plugin's own
tier=privileged log line - no new sandbox bridge capability needed), so
the remaining work is well-scoped and mechanical, not blocked on any open
question - just deliberately deferred to unhurried, focused time.

Verified: typecheck clean, lint clean, full vitest suite 2484/2485 (only
the pre-existing, unrelated jmap-client-resilience flake), production
build succeeds.
2026-08-06 00:30:27 +02:00
Bernd Rodler 91b282d746 feat(ai): real retrieval — SourceRef, RRF fusion, real embedding leg (P3/P4)
Full retrieval pipeline per docs/AI-ASSISTANT-CONCEPT.md §7/§8.1, real end
to end, not mocked:

- lib/ai/retrieval/types.ts: SourceRef + RetrieverAdapter schema. Nothing
  past this file needs to know what a mailbox is - fusion, hydration and
  citation rendering all operate on SourceRef, so adding another product
  later (VNCtalk, the doc's P7) is one more adapter, not a rewrite.
- lib/ai/retrieval/fusion.ts: Reciprocal Rank Fusion, score = Σ1/(k+rank),
  k=60. Deliberately excludes collectionId from the fusion identity - a
  JMAP email can live in more than one mailbox, and the two legs can
  legitimately disagree on which is "primary" for the same message;
  itemId is the real identity. 5 unit tests, including that exact
  double-count case.
- lib/ai/retrieval/mail-embeddings.ts: the server embedding leg. Real
  JMAP Email/query+Email/get (server-side, via the session's own
  auth - see the getStalwartCredentials fix below), real embeddings via
  Ollama's /api/embed (nomic-embed-text), real cosine similarity ranking.
  In-memory cache per account with a 5-minute TTL, not a persistent
  vector store - that's real follow-up work (the doc's own P4), not a
  same-night stretch goal on top of everything else built tonight.
- app/api/ai/retrieve/route.ts: wires it together. ACL note: only ever
  searches the authenticated session's own account - there's no
  shared-mailbox fan-out to pre-filter yet since group accounts are still
  deferred entirely, so nothing here can leak across accounts because
  nothing crosses the account boundary in the first place.
- lib/ai/local-client.ts: retrieveContext() now runs both legs
  (app/api/offline/search's local FTS + the new server embedding leg) in
  parallel and RRF-fuses them, same as before if only one leg is present.

Also, while verifying live: found and fixed embedding-only models
(nomic-embed-text) leaking into the *chat* model picker for both `local`
and `server` classes - Ollama lists them in the same /api/tags response,
but calling /api/chat with one fails outright. Filtered by `capabilities`
(fails open if absent, for older Ollama).

Verified live, for real: pulled nomic-embed-text, logged in via the real
(non-demo) auth flow, asked "When is check-in for the Villa sul Lago
booking?" against the seeded mock inbox - got back "Check-in ... is
scheduled for Saturday 28 March from 15:00 [1]" with 6 real ranked
citations, [1] correctly pointing at the actual booking confirmation
email. Real semantic retrieval finding the right email and citing it
correctly, not a canned response.

Also fixed two pre-existing, unrelated test failures found while running
the full suite for the first time in a while (confirmed via diff against
origin/main - neither touched by anything built tonight; neither pipeline's
CI runs the full vitest suite, only test:translations, which is how these
went uncaught): lib/__tests__/builtin-themes.test.ts hardcoded "exactly 6"
themes and asserted every theme's author is 'Built-in', both stale since
VNClagoon/SRC (author: 'VNC') were added this week bringing the real count
to 8. Left the also-pre-existing, timing-sensitive
jmap-client-resilience.test.ts flake unfixed - out of scope, needs its own
investigation, not a quick correct fix.

Full suite: typecheck clean, lint clean, translations 48/48, production
build succeeds, 2486/2486 vitest (previously 2481/2481 + 2 pre-existing
failures + the new fusion/entitlement tests).
2026-08-06 00:21:13 +02:00
Bernd Rodler dda7adf565 feat(ai): multi-key BYOK, real server class, real entitlement enforcement
Three pieces built together tonight since they're naturally linked (the
server-class proxy is the real entitlement enforcement chokepoint):

1. Multi-key BYOK (public class): several named provider profiles
   (name/baseUrl/model), each with its own key in lib/ai/key-store.ts
   (keyed by profile id, not a single fixed 'public' slot). The "Try it"
   pane lets you pick which saved profile answers each question - not one
   fixed default.

2. `server` class, real: app/api/ai/server/{models,chat} proxy through
   this app's own backend to AI_SERVER_BASE_URL - same-origin from the
   browser, no CORS/OLLAMA_ORIGINS story at all, standing in tonight for
   VNC's EU/CH-hosted infra with the real Ollama on this Mac (swapping to
   the real instance tomorrow is a config change).

3. Real entitlement enforcement (lib/ai/entitlement.ts), scoped to `server`
   only (not local/public, per the 2026-08-05 decisions): checkAndAssignSeat()
   re-validates on every /api/ai/server/chat call - first use auto-assigns a
   seat if any remain, further calls from an unlicensed user get a 402 with
   a specific reason. recordUsage() appends to an append-only metering
   ledger (timestamp/user/model/tokens/latency) that IS the billing record.
   Admin data endpoints at /api/admin/ai/entitlement (seat total, revoke) -
   the visual admin console is a separate, not-yet-built task.

Two real bugs found and fixed during verification, not just claimed fixed:
- /api/ai/policy never actually added 'server' to entitlement.classes even
  when AI_SERVER_BASE_URL was set (only the type comment was updated) - the
  Server radio option silently never appeared until this was caught live.
- The new routes used readStalwartAuthContext(0) (hardcoded slot, SSO/reauth-
  specific) instead of getStalwartCredentials() (the general multi-slot
  session resolver every other authenticated route uses) - reachable but
  wrong, and would have hidden a real auth gap behind "works on my slot".

Verified end-to-end for real: built + ran the actual server, logged in via
the real (non-demo) auth flow, selected Server, listed the real Ollama
models through the proxy, asked "Reply with exactly the words: SERVER CLASS
WORKS" and got back exactly that - plus confirmed on disk (not just in the
UI) that data/admin-state/ai-entitlement.json recorded the seat assignment
and ai-metering.jsonl recorded real prompt/completion token counts and
latency from the actual model call. Rejection-path logic (seat limit
reached, zero seats configured, revocation) covered by 5 new unit tests
rather than a second live round trip. Full suite: typecheck clean, lint
clean, translations 48/48, production build succeeds.
2026-08-06 00:07:56 +02:00
Bernd RodlerandClaude Sonnet 5 bde8455df5 fix(mail-index): close handle on every pragma failure, reconcile contact/file deletes
QA pass on the encrypted mail index found two real gaps beyond what the
prior end-to-end fix pass caught:

1. store.ts's MailIndex.open() only wrapped SOME of the post-key pragma
   calls in a try/catch before this: the first attempt's `key` pragma and
   assertEncrypted() ran outside any try at all, and the wrong-key retry
   repeated the same gap. Any pragma throwing there (SQLITE_BUSY, a full
   disk on the first WAL write) leaked the native SQLite handle instead of
   closing it. Factored the open+key+verify+pragma sequence into openKeyed(),
   which guarantees a close before rethrowing on any failure, and reused it
   for both the first attempt and the retry.

2. reindex.ts never removed a deleted contact or file from the index. The
   `removed` field exists in the API and is fully tested at the store layer,
   but nothing in the renderer populates it, so a deleted contact/file stayed
   searchable - and retrievable by the AI feature - indefinitely. Mail and
   calendar can't use the same fix (their queries are date-windowed, so an id
   missing from one fetch may just be outside the window), but contacts/files
   have no date filter - a catch-up fetch that comes back under its cap IS
   the complete set, so anything locally indexed but absent from it is safely
   known to be deleted. Added strayIdsAfterCatchUp() and wired it into the
   catch-up path for those two types only.

Also read binding.ts, key.ts, paths.ts, jmap.ts, extract.ts, the FTS5
query builder, and both /api/offline/{search,reindex} routes end to end;
no other concrete bugs found there. Full findings reported separately.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 23:50:03 +02:00
Bernd Rodler bde9f14832 feat(theme): ship exactly 2 themes (SRC default + VNClagoon); rebrand user-facing VNCmail+
Product decision 2026-08-05: exactly SRC (default) and VNClagoon ship as
selectable themes. Everything else (Qui, Nord, Catppuccin, Solarized,
Roundcube Elastic, Aurora Glass) is hidden via ThemePolicy.disabledBuiltinThemes
rather than deleted - cheap to re-enable later, zero code lost. Also removed
the hardcoded "Default/Bulwark" theme card from the Settings > Themes grid
(product wants exactly 2 theme choices, not 3).

Separately, found and fixed real "still says Bulwark" branding gaps:
- All 24 locale files: "Bulwark"/"Bulwark Mail"/"Bulwark Webmail" ->
  "VNCmail+" in every user-facing string (verified via the translations
  test, which only checks structural key parity across locales, not
  content - a straight string swap is safe against it. 48/48 still pass).
- PWA manifest fallback app name, package.json description/author.
- Demo mode fixtures: the "Welcome to Bulwark Mail!" email and identity
  signature a first-time demo user actually sees.
- The demo empty-state's logo was hardcoded to a literal Bulwark SVG file
  regardless of active theme - a real bug, not just stale text, since
  lib/theme-logo.ts's resolveThemeLogo() already exists and is already used
  correctly by the login page and nav rail for exactly this (theme-aware
  SRC mark / VNClagoon wordmark). Wired the same helper in here instead of
  a hardcoded path.

NOT touched: internal code comments referencing "Bulwark" as historical/
attribution context (e.g. design-rationale comments explaining why a value
differs from Bulwark's original) - those are harmless and a full-repo sweep
of every comment wasn't the ask.

Verified: typecheck clean, lint clean, translations test 48/48 passing.
2026-08-05 23:43:19 +02:00
Bernd Rodler 3f99a2fc9e deploy(dev): point overlay at the real GHCR image, not the side-loaded one
Tonight's GitHub Actions runs (Publish Docker Image workflow) succeeded on
every commit, confirmed by pulling the manifest directly: sha-147660a
exists in ghcr.io/brvncde-dotcom/vncmail-plus-dev and its digest matches
the `latest` tag exactly. Unlike the prior sha-d0a1cee6 pin (built locally,
manually `ctr images import`-ed onto each node - see 9b5870ca), this tag is
a real, publicly pullable registry image: no side-loading needed, survives
a node rebuild, and includes everything through tonight's AI work
(P0 scaffolding, real local-Ollama wiring, the CSP fix that made it
actually reachable, aiAssistantEnabled defaulting on).

Does NOT deploy anything by itself - ArgoCD's vncmail-dev Application is
still manual-sync (see deploy/argocd/vncmail-dev-app.yaml), and this
session has no kubectl/cluster access to trigger that sync or verify the
rollout. Whoever next syncs (or restarts the deployment) picks this up
automatically via imagePullPolicy: IfNotPresent, which now works as a real
cache rather than a hard dependency on the side-loaded image.
2026-08-05 23:14:14 +02:00
Bernd Rodler 147660aef3 fix(csp): allow loopback HTTP for the local AI provider; enable AI Assistant by default
Real end-to-end verification (Playwright-driven real Electron app on this
Mac, against the actual local Ollama instance, not a mock) found the actual
blocker: production's connect-src CSP (`'self' https: wss:`) rejects plain
http:// entirely, so lib/ai/local-client.ts's loopback fetch to Ollama never
even attempted the network in a production build - Electron or browser
alike. This is almost certainly what looked like a browser-sandbox network
issue in the earlier (non-Electron) QA pass tonight too.

Fix is narrow, not a blanket http: relaxation: connect-src now additionally
allows `http://127.0.0.1:*` and `http://localhost:*` specifically. Loopback
has no network hop, so it doesn't reopen the mixed-content-style downgrade
risk the existing https-only production policy guards against - unlike
dev's blanket `http:` allowance, which stays dev-only.

Confirmed fixed: rebuilt (build:standalone + build:electron), launched the
real Electron app via Playwright's _electron, and got a genuine answer back
from the real local Ollama - "Test connection" showed Reachable (the real
success state, not the CORS-diagnostic fallback text), and asking "Reply
with exactly the words: LOCAL AI WORKS" returned exactly that, with the
correct "no local mail index in this session" banner alongside it (accurate
for a fresh Electron session with nothing synced yet).

Also flips FeatureGates.aiAssistantEnabled's default false->true: local now
genuinely works and ships free/unmetered (see lib/ai/types.ts), so there is
a real feature behind the tab, not an empty preview - matches tonight's
explicit "I want AI visible" instruction. An admin can still turn it off.

Verified: typecheck clean, lint clean (0 errors, pre-existing warnings
only), translations pass (48/48), full production build succeeds.
2026-08-05 22:50:57 +02:00
Bernd Rodler 5d7ae230ce feat(ai): real local Ollama chat + BYOK public provider
Decisions 2026-08-05 evening (reprioritizing docs/AI-ASSISTANT-CONCEPT.md's
original P1/P2 server-first sequencing to local-first, since a real Ollama
instance already runs on this Mac with a full model set):

- `local` ships free, no entitlement check — always available wherever
  supportsLocalLlm() is true.
- `public` (BYOK) is available too, explicitly unmonitored for now — no
  seats/metering/consent backend. This reverses the concept doc's decision
  #1 (server-side-only key custody): the client holds its own key, matching
  vncmail-native's existing pattern.
- `server` (VNC-hosted) stays unwired client-side; that infra is "this
  MacBook tonight, the dev k8s cluster tomorrow."

New:
- lib/ai/local-client.ts: listLocalModels/testLocalConnection/chatLocal/
  chatPublic, ported near-verbatim from vncmail-native's proven
  src/api/ai.ts. Direct browser-side fetch, not proxied through this app's
  own server — a server-side proxy would reach the *server's* loopback, not
  the user's own laptop, which defeats the point of "local" once this app
  is hosted remotely.
- lib/ai/key-store.ts: client-held BYOK storage (localStorage — this repo's
  existing convention for client state, no OS keychain reachable from a
  browser tab).
- lib/ai/local-settings.ts: isolated persistence for provider/model/base-URL
  choices. Deliberately NOT folded into stores/settings-store.ts, which has
  a hand-maintained export/import enumeration this prototype-scope state
  doesn't belong in yet.
- Retrieval reuses this app's own already-built app/api/offline/search
  (encrypted SQLite/FTS5 mail index) as context when available, and
  degrades to unaugmented chat — not an error — when it 404s/503s (no index
  in this session, e.g. plain browser rather than Electron).

Rewrote components/settings/ai-assistant-settings.tsx: provider picker,
local runtime config (base URL, model list/refresh, test connection with a
CORS-aware diagnostic per the concept doc's own note on the browser row),
public BYOK config (base URL, model, key, client-side consent toggle), and
a working Ask box.

Verified: typecheck clean, lint clean, translations pass, production build
succeeds. Live-tested against the real Ollama on this machine (confirmed
running: qwen2.5:32b, gemma4, deepseek-r1, llama3.2, hermes3, qwen3) via a
local server + demo-mode session — admin flag round-trips correctly, the
pane renders both provider options, and the CORS-diagnostic path fires
correctly on a real (if here environment-sandboxed, not Ollama-side)
connection failure. Full success end-to-end still wants a real, unsandboxed
browser tab against this Mac's loopback to close out.
2026-08-05 22:41:22 +02:00
Bernd Rodler 2a35019b21 feat(ai): P0 client scaffolding — capability flags, settings pane, policy fetch
Per docs/AI-ASSISTANT-CONCEPT.md §12, P0 is deliberately generation-free:
prove platform gating and the policy round trip before any model exists
behind it. No provider is called anywhere in this change.

- lib/platform-capabilities.ts: supportsLocalLlm/localLlmNeedsCorsSetup,
  mirroring the same-named module in vncmail-native so the capability
  contract (§3, §11) reads identically on both clients. Web+Electron only
  here — mobile is a separate codebase.
- lib/ai/types.ts: AiPolicy/AiEntitlement schema, locked in now per decision
  #4 (entitlement from day one — cheap now, a live-tenant migration later).
- app/api/ai/policy/route.ts: GET, unauthenticated (users read this, like
  /api/admin/policy). Composes the real FeatureGates.aiAssistantEnabled
  toggle with a hardcoded unlicensed entitlement — there's no seats/billing
  backend yet (P2), so nothing here can honestly claim otherwise.
- components/settings/ai-assistant-settings.tsx: fetches that policy, shows
  a real (not fake) locked/unlicensed state. No model config UI yet — there
  is nothing real to configure until P1/P2/P5 land.
- New admin FeatureGates.aiAssistantEnabled (default false, like
  pluginsEnabled): the tab is entirely hidden until an admin opts in, so no
  existing install suddenly sees a tab that does nothing.

Verified: typecheck clean, lint clean, translations test passes (48/48),
full production build succeeds with /api/ai/policy compiled in.
2026-08-05 22:02:40 +02:00
brvncde-dotcomandGitHub 4ba2d34555 Merge pull request #1 from brvncde-dotcom/dev
Dev
2026-08-05 21:56:18 +02:00
Bernd Rodler cfe8ca96e1 ci(github): add PR Verify workflow — required check for main protection
Mirrors .gitlab-ci.yml's verify stage (typecheck/lint/translations/build)
on GitHub Actions, since GitHub is being reactivated as a working build
path while gitlab.vnc.biz's own registry and runner are blocked (see
docs memory: gitlab-registry-dependency-proxy). Runs on PRs into main or
dev; wired as main's required status check.
2026-08-05 21:50:43 +02:00
Bernd Rodler e1a12b2c23 fix(electron): real VNCmail+ branding, not just a rename
Every prior distributable DMG this session was built with plain `npx
electron-builder`, never `--config electron-builder.config.js`. electron-
builder does not auto-detect a file named electron-builder.config.js (its
search list is .yml/.yaml/.json/.json5/.js/.cjs/.mjs/.ts, not .config.js),
so the config - correct productName/appId/icon and all - was silently
ignored on every build. Caught only by actually launching the packaged
.app: it booted to "Bulwark Webmail Setup" demanding a token from
container logs, default Electron atom icon, output in dist/ instead of
dist-electron-builds/.

Fixes, each verified against the packaged .app (Playwright _electron.launch,
not the build log):
- Add dist:mac/win/linux/dir scripts that pass --config explicitly, so this
  can't recur.
- Dedicated 1024x1024 app icon (build-resources/app-icon.png, SRC symbol on
  #09090b) instead of reusing the web PWA manifest icon. Verified: icns
  ships at 1024x1024, pixel-identical to the source (mean diff 0.0/255).
- electron/main.ts: getDesktopDefaults() sets JMAP_SERVER_URL to the sandbox
  (the ONLY thing that puts the server into "env-managed" mode and skips
  the setup wizard - see lib/setup/state.ts), plus APP_NAME/login logo/
  favicon/company-name env vars, spread before ...process.env so a real
  deployment still overrides. Verified: packaged app now opens straight to
  a login screen with the JMAP endpoint field pre-filled
  https://stalwart.sandbox.vnc.de, title "VNCmail+", SRC logo.
- LOGIN_SHOW_SUBTITLE=false: the subtitle falls back to the login.title
  i18n string ("Webmail") whenever it differs from APP_NAME - a check
  written for the original Bulwark pairing where they matched. Hiding it
  avoids touching that shared string for every other deployment.
2026-08-05 21:01:38 +02:00
Bernd Rodler e44f2ac97e branding: rename app to VNCmail+ (Electron) 2026-08-05 20:36:12 +02:00
Bernd Rodler 9b5870ca69 deploy(dev): pin sandbox to sha-d0a1cee6 + IfNotPresent pull policy
Puts today's merged dev on the sandbox (S/MIME, offline replica, SRC
branding) without waiting on CI, which still can't push anywhere: GitLab's
registry vhost serves Rails/dependency-proxy (see .gitlab-ci.yml) and GHCR
needs a PAT that only a human can mint.

The amd64 image was built locally and side-loaded into all three nodes'
containerd via `microk8s ctr images import`, so IfNotPresent is required -
Always would ignore the local image and try to pull a tag no registry has.
IfNotPresent is the correct policy for immutable sha- tags regardless; see
the comment in patch-image-pull-policy.yaml for the full runbook.
2026-08-05 19:47:28 +02:00
Bernd Rodler 2e8bb9983a fix(ci): back to GHCR - GitLab's registry vhost serves Rails, not the registry
Diagnosed definitively rather than by log-guessing this time:

  $ curl -i https://registry.gitlab.vnc.biz/v2/
  www-authenticate: Bearer realm="http://gitlab.vnc.biz/jwt/auth",
                           service="dependency_proxy"
  x-runtime: 0.020470
  x-gitlab-meta: {"correlation_id":...}

x-runtime/x-gitlab-meta are Rails headers and the service is
"dependency_proxy" - nginx routes that hostname to the GitLab Rails app,
which treats /v2/ as the Docker Hub pull-through cache, not as this
project's container registry. The registry service was never wired behind
the vhost, which is why an unscoped docker login succeeded while kaniko's
scoped :push request got 403 (the dependency proxy has no push concept).

Fixing that is server-side nginx/omnibus work. Keeping kaniko (it solved
the real dind-needs-privileged problem) and pointing it at GHCR, plus an
upfront credential check so a missing variable fails in seconds instead of
after a full Next.js build.
2026-08-05 19:40:49 +02:00
Bernd Rodler d0a1cee6fd fix(ci): build with Kaniko instead of docker-in-docker
dind never actually came up on this runner regardless of how it was
addressed (unix socket, docker:2375, localhost:2375 all failed identically
after a successful registry login) — on GitLab's Kubernetes executor that
means the dind container needs `privileged: true` in the runner's own
config.toml, which is admin-side, not something this file can set.

Kaniko builds OCI images without any daemon, so it needs no privileged
pod and no dind service at all — GitLab's own recommended path for this
exact executor, and safer on a shared cluster besides.
2026-08-05 19:37:42 +02:00
Bernd Rodler 19663610d7 fix(ci): use localhost, not the docker: alias, to reach dind
This runner is GitLab's Kubernetes executor (pod names in the job log:
runner-uncqet63-project-499-concurrent-*), where all containers in a job
share one pod's network namespace. The docker: service-alias hostname is
a Docker-executor convention (bridge network + DNS alias) and doesn't
apply here — tcp://docker:2375 correctly read the variable but nothing
answered at that name. localhost is the right host for this executor.
2026-08-05 19:35:38 +02:00
Bernd Rodler 36167eaa84 fix(ci): point docker client at dind over plaintext TCP
registry login now succeeds (CI_REGISTRY populated correctly) but the
build step failed separately: docker:27-dind defaults to TLS on :2376,
which the docker:27-cli client image doesn't know to use without a
mounted cert dir. DOCKER_HOST=tcp://docker:2375 + DOCKER_TLS_CERTDIR=""
is the standard fix for GitLab's Kubernetes executor, where both
containers share the job's pod network namespace.
2026-08-05 19:33:59 +02:00
Bernd Rodler c71175e596 fix(ci): switch back to GitLab's native Container Registry
Confirmed 2026-08-05 the project's Container Registry is now enabled
server-side (visible in the left sidebar under Deploy). That's strictly
better than the GHCR detour: $CI_REGISTRY/$CI_REGISTRY_USER/$CI_REGISTRY_PASSWORD
are predefined GitLab CI variables scoped to this project, so this needs
zero manually-created credentials (no GitHub PAT to hold in CI/CD variables).
2026-08-05 19:29:42 +02:00
Bernd Rodler 68d08dbae6 fix(ci): revert to GHCR - GitLab's own registry never got past step 1
GitLab's Container Registry was enabled at the omnibus service level
(registry.gitlab.vnc.biz responds, confirmed with a real GitLab-shaped
401), but the pipeline's build job kept trying to auth against Docker
Hub instead - CI_REGISTRY was empty. Root cause: registry_external_url
only starts the registry SERVICE; gitlab_rails['registry_enabled'] = true
is a separate key that tells the Rails app the registry exists, and it
was never set. Symptom matched exactly: registry reachable, but no
Container Registry toggle anywhere in project settings OR admin settings,
and CI_REGISTRY empty in every job regardless of retry.

Reverting the pipeline to ghcr.io/brvncde-dotcom/vncmail-plus-dev - the
exact image the sandbox was already running before any of this session's
pipeline existed, confirmed public (no imagePullSecrets needed). This is
a revert to a known-working path, not a new risk.

Needs $GITLAB_CI_GHCR_TOKEN (GitHub PAT, write:packages) and
$GITLAB_CI_GHCR_USER as masked/protected CI/CD variables - a GitHub
credential has to come from GitHub, nothing on the GitLab side can
substitute for it.
2026-08-05 19:16:21 +02:00
Bernd-Rodler 7cce5c0393 Merge branch 'claude/webmail-offline-replica' into 'dev'
feat(electron): real offline mail replica — delta sync, full bodies, retention

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!6
2026-08-05 17:05:29 +00:00
Bernd-Rodler 6b6ff72c38 Merge branch 'claude/activate-smime-plugin' into 'dev'
feat(smime): actually install the audited S/MIME plugin in real builds

See merge request gitlab-instance-b9b5cf2f/vncmail-plus!5
2026-08-05 16:42:15 +00:00
Bernd Rodler 57a5c692be fix(k8s): drop imagePullSecrets - the ghcr package is confirmed public
Blocking the very first real deploy of the sandbox: base/deployment.yaml
referenced an imagePullSecrets entry ("ghcr-pull") that was never created,
which fails pod startup regardless of whether the image needs auth at
all - kubelet errors trying to resolve the named secret before it gets
anywhere near actually pulling.

Confirmed by execution (anonymous GHCR token, pull succeeded) that
ghcr.io/brvncde-dotcom/vncmail-plus-dev is public. Removing the block is
deploy/k8s/README.md's own documented alternative for exactly this case.
2026-08-05 18:26:58 +02:00
Bernd Rodler 84290a67be Merge remote-tracking branch 'gitlab/claude/src-branding' into dev-merge-batch1 2026-08-05 17:58:14 +02:00
Bernd Rodler ada2b3a7a1 Merge remote-tracking branch 'gitlab/claude/electron-userdata-dirs' into dev-merge-batch1 2026-08-05 17:58:14 +02:00
Bernd Rodler 3338ceb5eb docs: correct the mobile replica — it is NOT encrypted
I described vncmail-native's offline mail replica as "SQLCipher-encrypted"
in ARCHITECTURE.md and to the user. That is wrong, and it overstates a
security property.

Verified against the shipped code: src/sync/schema.ts sets
STORE_FORMAT = 'sqlite-plain', src/sync/store-sqlite.ts's own header says
"plain expo-sqlite, no SQLCipher", sqlite-driver.ts opens via
openDatabaseAsync() with no PRAGMA key, and there is no SQLCipher
dependency in package.json at all. SQLCipher is a documented future
native-build flip (expo-sqlite's useSQLCipher flag), not shipped behaviour.

Full mail bodies therefore sit in cleartext on the device — a materially
different posture from the Electron search index, which really is
encrypted (@signalapp/sqlcipher with an OS-keychain key via safeStorage).
Worth being precise about given the product positioning.
2026-08-05 17:58:04 +02:00