Commit Graph
2 Commits
Author SHA1 Message Date
Bernd Rodler dda7adf565 feat(ai): multi-key BYOK, real server class, real entitlement enforcement
Three pieces built together tonight since they're naturally linked (the
server-class proxy is the real entitlement enforcement chokepoint):

1. Multi-key BYOK (public class): several named provider profiles
   (name/baseUrl/model), each with its own key in lib/ai/key-store.ts
   (keyed by profile id, not a single fixed 'public' slot). The "Try it"
   pane lets you pick which saved profile answers each question - not one
   fixed default.

2. `server` class, real: app/api/ai/server/{models,chat} proxy through
   this app's own backend to AI_SERVER_BASE_URL - same-origin from the
   browser, no CORS/OLLAMA_ORIGINS story at all, standing in tonight for
   VNC's EU/CH-hosted infra with the real Ollama on this Mac (swapping to
   the real instance tomorrow is a config change).

3. Real entitlement enforcement (lib/ai/entitlement.ts), scoped to `server`
   only (not local/public, per the 2026-08-05 decisions): checkAndAssignSeat()
   re-validates on every /api/ai/server/chat call - first use auto-assigns a
   seat if any remain, further calls from an unlicensed user get a 402 with
   a specific reason. recordUsage() appends to an append-only metering
   ledger (timestamp/user/model/tokens/latency) that IS the billing record.
   Admin data endpoints at /api/admin/ai/entitlement (seat total, revoke) -
   the visual admin console is a separate, not-yet-built task.

Two real bugs found and fixed during verification, not just claimed fixed:
- /api/ai/policy never actually added 'server' to entitlement.classes even
  when AI_SERVER_BASE_URL was set (only the type comment was updated) - the
  Server radio option silently never appeared until this was caught live.
- The new routes used readStalwartAuthContext(0) (hardcoded slot, SSO/reauth-
  specific) instead of getStalwartCredentials() (the general multi-slot
  session resolver every other authenticated route uses) - reachable but
  wrong, and would have hidden a real auth gap behind "works on my slot".

Verified end-to-end for real: built + ran the actual server, logged in via
the real (non-demo) auth flow, selected Server, listed the real Ollama
models through the proxy, asked "Reply with exactly the words: SERVER CLASS
WORKS" and got back exactly that - plus confirmed on disk (not just in the
UI) that data/admin-state/ai-entitlement.json recorded the seat assignment
and ai-metering.jsonl recorded real prompt/completion token counts and
latency from the actual model call. Rejection-path logic (seat limit
reached, zero seats configured, revocation) covered by 5 new unit tests
rather than a second live round trip. Full suite: typecheck clean, lint
clean, translations 48/48, production build succeeds.
2026-08-06 00:07:56 +02:00
Bernd Rodler 5d7ae230ce feat(ai): real local Ollama chat + BYOK public provider
Decisions 2026-08-05 evening (reprioritizing docs/AI-ASSISTANT-CONCEPT.md's
original P1/P2 server-first sequencing to local-first, since a real Ollama
instance already runs on this Mac with a full model set):

- `local` ships free, no entitlement check — always available wherever
  supportsLocalLlm() is true.
- `public` (BYOK) is available too, explicitly unmonitored for now — no
  seats/metering/consent backend. This reverses the concept doc's decision
  #1 (server-side-only key custody): the client holds its own key, matching
  vncmail-native's existing pattern.
- `server` (VNC-hosted) stays unwired client-side; that infra is "this
  MacBook tonight, the dev k8s cluster tomorrow."

New:
- lib/ai/local-client.ts: listLocalModels/testLocalConnection/chatLocal/
  chatPublic, ported near-verbatim from vncmail-native's proven
  src/api/ai.ts. Direct browser-side fetch, not proxied through this app's
  own server — a server-side proxy would reach the *server's* loopback, not
  the user's own laptop, which defeats the point of "local" once this app
  is hosted remotely.
- lib/ai/key-store.ts: client-held BYOK storage (localStorage — this repo's
  existing convention for client state, no OS keychain reachable from a
  browser tab).
- lib/ai/local-settings.ts: isolated persistence for provider/model/base-URL
  choices. Deliberately NOT folded into stores/settings-store.ts, which has
  a hand-maintained export/import enumeration this prototype-scope state
  doesn't belong in yet.
- Retrieval reuses this app's own already-built app/api/offline/search
  (encrypted SQLite/FTS5 mail index) as context when available, and
  degrades to unaugmented chat — not an error — when it 404s/503s (no index
  in this session, e.g. plain browser rather than Electron).

Rewrote components/settings/ai-assistant-settings.tsx: provider picker,
local runtime config (base URL, model list/refresh, test connection with a
CORS-aware diagnostic per the concept doc's own note on the browser row),
public BYOK config (base URL, model, key, client-side consent toggle), and
a working Ask box.

Verified: typecheck clean, lint clean, translations pass, production build
succeeds. Live-tested against the real Ollama on this machine (confirmed
running: qwen2.5:32b, gemma4, deepseek-r1, llama3.2, hermes3, qwen3) via a
local server + demo-mode session — admin flag round-trips correctly, the
pane renders both provider options, and the CORS-diagnostic path fires
correctly on a real (if here environment-sandboxed, not Ollama-side)
connection failure. Full success end-to-end still wants a real, unsandboxed
browser tab against this Mac's loopback to close out.
2026-08-05 22:41:22 +02:00