SRCmail/docs/TEST-BASIS-2026-08-06.md

Test basis — 2026-08-06 morning

Covers everything shipped overnight: AI Assistant (web), S/MIME (web + mobile), and the 2-theme rebrand. For each area: how to get into a testable state, concrete steps with expected results, and — importantly — what's still a known gap so nothing here gets mistaken for more finished than it is.

All web steps assume vncmail-plus running locally via the built-in mock JMAP server (fastest path, no real Stalwart needed):

cp .env.dev.example .env.local
# then set, in .env.local:
#   AI_SERVER_BASE_URL=http://127.0.0.1:11434   (real Ollama on this Mac)
#   SMIME_CA_DEV_LOCAL=true
npm run dev

Open http://localhost:3000 and log in via the "Anmelden" (dev-mode login) button, not "Demo starten". Demo mode is a pure client-side in-memory session with no server-side cookie — AI's server class, S/MIME enrollment, and offline routes all need the real cookie the login button sets, and will look broken (401s) under Demo mode for reasons that have nothing to do with the features themselves.

---

1. AI Assistant (web)

Settings → AI Assistant.

1.1 Local (Ollama on this Mac)

1. Select provider Local. Model list should populate from the running ollama serve (confirm with ollama list in a terminal first).
2. Ask a question in "Try it" with no special mail content, e.g. "reply with exactly the words LOCAL AI WORKS".
3. Expect: exact echo back, no seat/entitlement banner (local is free, unmetered, never reaches this app's own backend).

1.2 Server (this app's backend → your Ollama)

1. Select provider Server. Model list comes from /api/ai/server/models (same models as Local, proxied — confirms the same-origin proxy path works, not just direct-to-loopback).
2. Ask a first question. Expect: a small "seat assigned" notice on the first successful call for this user, none on subsequent calls.
3. Check data/admin-state/ai-metering.jsonl on disk — a new line should appear with real promptTokens/completionTokens/latencyMs from the actual Ollama call, not placeholders.
4. To test the entitlement ceiling: curl -X PUT localhost:3000/api/admin/ai/entitlement -H 'Content-Type: application/json' -d '{"seatsTotal":0}' (needs an admin session cookie), then retry a Server-class question as a different username. Expect: HTTP 402, "no licensed seats configured". Set seatsTotal back up afterwards.

1.3 Public (BYOK)

1. Add a profile: name, model, base URL, API key (any real provider you have a key for, or a fake key just to test the UI — the request itself will fail at the provider, not in this app).
2. Add a second profile with a different name/model.
3. In "Try it", use the "Answer with" selector to choose between the two profiles for the same question. Expect: each answers using its own configured model, no cross-talk.
4. Remove a profile from the list. Expect: it disappears immediately, and its key is gone from localStorage (DevTools → Application → Local Storage, key prefix vncmail.ai.key.).

1.4 Retrieval (grounded answers over your own mail)

The demo/mock inbox includes a "Villa sul Lago" booking confirmation email.

1. On Server or Local, ask: "When is check-in for the Villa sul Lago booking?"
2. Expect: a specific date/time, with a cited source referencing the actual email (not a generic non-answer). This exercises the full pipeline: real JMAP Email/query+Email/get, real Ollama embeddings (nomic-embed-text), cosine similarity, RRF fusion with the local text-match leg.

1.5 Known gaps — don't be surprised by these

• Tier (base/standard/pro) is stored but doesn't change behavior yet — only seatsTotal gates anything. The admin console (§1.6) labels the picker as cosmetic for exactly this reason.

1.6 Admin AI Policy console — now built, test as an admin

Approved and implemented after the spec+mockup review. Admin panel → AI (new tab, next to Policy).

1. Provider classes — toggle Server off, then try the Server class as a user. Expect: 403 "disabled by admin policy" instead of a normal answer (real enforcement in /api/ai/server/chat, not just a hidden radio button). Toggle back on.
2. Server model allow-list — switch to "Restrict to selected", add one model name, save. Expect: the model picker in Settings → AI Assistant now shows only that model, and a direct API call with a different model name gets 403 (checked at the chat chokepoint too).
3. Seats — change "Seats licensed", watch the "N of M assigned" count update; Revoke a seat and confirm that user gets 402 again on next use (frees the seat for someone else, matches checkAndAssignSeat's first-use semantics).
4. Usage — real rows from ai-metering.jsonl, not placeholders — should match whatever calls you made in §1.2.
5. Retrieval & consent — toggle retrieval off, then ask a mail-grounded question. Expect: 403 from /api/ai/retrieve, plain chat still works. Edit the consent text, bump the version — no user-facing consent prompt is wired up to read this yet (that's the one open item: the text/version are stored and served over /api/ai/policy, but no UI currently shows it to end users before first BYOK use — worth flagging if that matters before wider rollout).

1.7 Known gaps, updated

• BYOK provider allow-listing is real now but client-side/advisory only — a user's own browser checks it at profile-save time (ai-assistant-settings.tsx), there's no server-side network boundary enforcing it. Matches the spec's §6.1 recommendation; revisit if that needs to become a hard boundary.
• The consent text/version admins set has no reader yet on the client side (see §1.6 point 5) — stored and served, not yet displayed.

---

2. S/MIME — web

2.1 What to test today — now the full UI flow

With SMIME_CA_DEV_LOCAL=true set, enrollment is fully wired end to end and live-verified (not just unit-tested): Settings → S/MIME → Get a certificate → enter a storage passphrase → a real certificate comes back (RSA-2048, correct validity window, real fingerprint) and appears in "Your keys". Unlock with the same passphrase round-trips correctly through the identical encrypted-at-rest path a PKCS#12 import would use.

1. Log in via "Anmelden" (real session-cookie flow, not "Demo starten" — see the caveat at the top of this doc).
2. Settings → S/MIME → Get a certificate.
3. Enter any passphrase, confirm.
4. Expect: a new key card appears — RSA-2048 · valid <today> – <+397 days>, a real SHA-256 fingerprint, Unlock/Delete buttons.
5. Click Unlock, enter the same passphrase. Expect: button flips to Lock — the private key decrypted correctly.
6. Compose a message to yourself, toggle Sign/Encrypt, send, then open it — this exercises the CA-issued cert through the plugin's existing sign/encrypt/decrypt/verify code paths (unchanged by tonight's work, since the key record shape is identical to a PKCS#12 import's).

The CA-issuance backend itself is additionally covered by 4 independent unit tests (CSR signature verification, full chain verification via leaf.verify(caCert), correct SAN/rfc822Name addresses, emailProtection EKU present) — see lib/smime-ca/__tests__/local-dev-provider.test.ts.

2.2 Known gap — real but narrow

No cross-testing yet between a web-CA-issued certificate and the mobile S/MIME implementation (§3) — worth doing once both have been used interactively at least once, not before. Also unchanged from yesterday: SMIME_CA_DEV_LOCAL is explicitly a stand-in for the real EJBCA (needs a client mTLS credential this environment doesn't have) — switching to production CA is a config change (SMIME_CA_URL + credentials), not a code change.

---

3. S/MIME — mobile (vncmail-native)

Unlike web, mobile got the full stack, merged to main this session (7b89839, combining the S/MIME work with the AI/offline-sync work that landed on main in parallel): real CMS sign/verify/encrypt/decrypt on node-forge primitives (wire-compatible with the audited webmail plugin, verified byte-for-byte against OpenSSL), PKCS#12 import, keys wrapped AES-256-GCM/PBKDF2 in expo-secure-store (Keychain/Keystore — never AsyncStorage in the clear).

3.1 Test steps

1. Build and run on a simulator/device (npm run ios / npm run android from vncmail-native).
2. Settings → S/MIME → Import certificate. Use a real .p12 file (an OpenSSL-generated one works fine for testing) and its passphrase.
3. Expect: cert imported, persists across an app restart, a wrong passphrase is rejected, the right one unlocks it.
4. Compose a new email to yourself → enable Sign (and/or Encrypt) → send.
5. Open the received message. Expect: a signature banner showing verified/signed-by, and if encrypted, the body decrypts and displays normally.
6. Try receiving a deliberately malformed/unauthenticated-cipher message (or just trust the 259-line hardening test suite — src/lib/__tests__/smime-hardening.test.ts — covers this without needing to hand-craft one). Expect: unauthenticated content renders as inert plaintext, never live HTML.

3.2 Known-good, already verified

479+ unit tests (834 after the merge with the AI/sync work) pass, including a full round-trip test. typecheck is clean. Also verified live on an Android emulator during that session: real OpenSSL-3-generated .p12 imported through the actual device file picker.

3.3 Known gap

No cross-checking was done this session between the mobile implementation and the CA now issuing certs on web (§2) — they were built independently and haven't been tested importing a web-CA-issued certificate into the mobile app. Worth doing once web's Enroll UI (§2.2) exists, not before.

---

4. Theme

Settings → Themes. Exactly 2 built-in themes now: SRC (default, Swiss red #D52B1E) and VNClagoon (navy/cyan). The other 6 generic built-ins (Nord, Catppuccin, Solarized, Roundcube Elastic, Aurora Glass, plus the old default) are hidden via admin theme policy, not deleted — an admin could re-enable them, a normal user can't see them.

4.1 Test steps

1. Confirm SRC is active on first load (no theme ever selected before).
2. Switch to VNClagoon. Expect: a "Theme activated" toast, and the chrome genuinely re-skins (navy/cyan replaces red throughout — sidebar, buttons, unread markers, not just an accent color here and there).
3. Switch back to SRC. Expect: same, in reverse.
4. Search the UI (empty-state illustrations, manifest/PWA name, login page, locale strings in a couple of languages) for the string "Bulwark" — all 24 locale files were rebranded to "VNCmail+" this session, plus app/manifest.ts, demo fixtures, and the inbox empty-state logo, which now resolves through the same theme-logo system every other themed surface uses instead of a hardcoded path.

4.2 Known gap

Not exhaustively re-checked for every locale/every screen — the 24-file find-and-replace was mechanical (script-driven) and spot-checked, not manually walked screen-by-screen in each language. If a stray "Bulwark" turns up somewhere, it's most likely a screen that wasn't in scope of the locale-string sweep (e.g. a hardcoded string in a component rather than a translation key) — same category as the logo fix above, worth a quick grep (grep -ril bulwark) if one surfaces.

---

5. Offline / mail-index

Already QA'd and fixed this session (not re-listed as a to-do): a leaked SQLite handle on PRAGMA failure, and contacts/files that weren't being removed from the local index on delete. Both fixed and covered by tests — no separate manual test needed tomorrow unless something else turns up.