docs: update test basis — admin AI console + S/MIME web enrollment now real
Both were "known gaps" in the original doc; rewrite those sections to reflect the actual shipped, live-verified state and note what's newly open instead (BYOK allow-list is advisory-only, consent text has no client-side reader yet).
This commit is contained in:
Bernd Rodler
@@ -80,67 +80,87 @@ The demo/mock inbox includes a "Villa sul Lago" booking confirmation email.
|
||||
|
||||
### 1.5 Known gaps — don't be surprised by these
|
||||
|
||||
- **No admin UI yet for any of the above.** Seat totals, model allow-lists,
|
||||
per-class on/off, usage — all real on the backend (§1.2, §1.3) but only
|
||||
reachable via `curl`/direct API calls today. A full spec + visual mockup
|
||||
for the admin console exists at
|
||||
[`docs/ADMIN-AI-POLICY-CONSOLE-SPEC.md`](ADMIN-AI-POLICY-CONSOLE-SPEC.md)
|
||||
— presented for approval, **intentionally not built** pending sign-off.
|
||||
- **BYOK provider allow-listing does not exist at all yet** — a user can
|
||||
point a Public profile at literally any URL. Not a regression, just never
|
||||
built; see the spec's §6.1 for the proposed (client-side-only) approach.
|
||||
- Tier (base/standard/pro) is stored but doesn't change behavior yet — only
|
||||
`seatsTotal` gates anything.
|
||||
`seatsTotal` gates anything. The admin console (§1.6) labels the picker
|
||||
as cosmetic for exactly this reason.
|
||||
|
||||
### 1.6 Admin AI Policy console — now built, test as an admin
|
||||
|
||||
Approved and implemented after the spec+mockup review. Admin panel → **AI**
|
||||
(new tab, next to Policy).
|
||||
|
||||
1. **Provider classes** — toggle Server off, then try the Server class as a
|
||||
user. **Expect:** 403 "disabled by admin policy" instead of a normal
|
||||
answer (real enforcement in `/api/ai/server/chat`, not just a hidden
|
||||
radio button). Toggle back on.
|
||||
2. **Server model allow-list** — switch to "Restrict to selected", add one
|
||||
model name, save. **Expect:** the model picker in Settings → AI
|
||||
Assistant now shows only that model, *and* a direct API call with a
|
||||
different model name gets 403 (checked at the chat chokepoint too).
|
||||
3. **Seats** — change "Seats licensed", watch the "N of M assigned" count
|
||||
update; **Revoke** a seat and confirm that user gets 402 again on next
|
||||
use (frees the seat for someone else, matches `checkAndAssignSeat`'s
|
||||
first-use semantics).
|
||||
4. **Usage** — real rows from `ai-metering.jsonl`, not placeholders — should
|
||||
match whatever calls you made in §1.2.
|
||||
5. **Retrieval & consent** — toggle retrieval off, then ask a mail-grounded
|
||||
question. **Expect:** 403 from `/api/ai/retrieve`, plain chat still
|
||||
works. Edit the consent text, bump the version — no user-facing consent
|
||||
*prompt* is wired up to read this yet (that's the one open item: the
|
||||
text/version are stored and served over `/api/ai/policy`, but no UI
|
||||
currently shows it to end users before first BYOK use — worth flagging
|
||||
if that matters before wider rollout).
|
||||
|
||||
### 1.7 Known gaps, updated
|
||||
|
||||
- BYOK provider allow-listing is real now but **client-side/advisory
|
||||
only** — a user's own browser checks it at profile-save time
|
||||
(`ai-assistant-settings.tsx`), there's no server-side network boundary
|
||||
enforcing it. Matches the spec's §6.1 recommendation; revisit if that
|
||||
needs to become a hard boundary.
|
||||
- The consent text/version admins set has no reader yet on the client side
|
||||
(see §1.6 point 5) — stored and served, not yet displayed.
|
||||
|
||||
---
|
||||
|
||||
## 2. S/MIME — web
|
||||
|
||||
### 2.1 What to test today
|
||||
### 2.1 What to test today — now the full UI flow
|
||||
|
||||
With `SMIME_CA_DEV_LOCAL=true` set, the CA-issuance backend is real and
|
||||
independently verified (4 passing tests: CSR signature verification, full
|
||||
chain verification via `leaf.verify(caCert)`, correct SAN/rfc822Name
|
||||
addresses, `emailProtection` EKU present — see
|
||||
`lib/smime-ca/__tests__/local-dev-provider.test.ts`).
|
||||
With `SMIME_CA_DEV_LOCAL=true` set, enrollment is fully wired end to end and
|
||||
live-verified (not just unit-tested): Settings → S/MIME → **Get a
|
||||
certificate** → enter a storage passphrase → a real certificate comes back
|
||||
(RSA-2048, correct validity window, real fingerprint) and appears in "Your
|
||||
keys". **Unlock** with the same passphrase round-trips correctly through
|
||||
the identical encrypted-at-rest path a PKCS#12 import would use.
|
||||
|
||||
There is **no user-facing "Enroll" button wired up yet** — that was a
|
||||
deliberate scope decision made overnight (see the commit message on
|
||||
`c2c07293`), not an oversight: this plugin has known, still-open
|
||||
security-audit findings, and generating brand-new CSR/key-import code
|
||||
against it at 1am risked introducing a new one rather than closing an old
|
||||
one. So today, test the backend directly:
|
||||
1. Log in via "Anmelden" (real session-cookie flow, not "Demo starten" —
|
||||
see the caveat at the top of this doc).
|
||||
2. Settings → S/MIME → **Get a certificate**.
|
||||
3. Enter any passphrase, confirm.
|
||||
4. **Expect:** a new key card appears — `RSA-2048 · valid <today> – <+397
|
||||
days>`, a real SHA-256 fingerprint, Unlock/Delete buttons.
|
||||
5. Click **Unlock**, enter the same passphrase. **Expect:** button flips to
|
||||
**Lock** — the private key decrypted correctly.
|
||||
6. Compose a message to yourself, toggle Sign/Encrypt, send, then open it —
|
||||
this exercises the CA-issued cert through the plugin's existing
|
||||
sign/encrypt/decrypt/verify code paths (unchanged by tonight's work,
|
||||
since the key record shape is identical to a PKCS#12 import's).
|
||||
|
||||
```bash
|
||||
# Generate a throwaway CSR (any tool - openssl shown here):
|
||||
openssl req -new -newkey rsa:2048 -nodes -keyout /tmp/test.key \
|
||||
-out /tmp/test.csr -subj "/CN=Test User"
|
||||
The CA-issuance backend itself is additionally covered by 4 independent
|
||||
unit tests (CSR signature verification, full chain verification via
|
||||
`leaf.verify(caCert)`, correct SAN/rfc822Name addresses, `emailProtection`
|
||||
EKU present) — see `lib/smime-ca/__tests__/local-dev-provider.test.ts`.
|
||||
|
||||
# Call the enrollment route directly (needs an authenticated session cookie
|
||||
# from the "Anmelden" login above - copy it from DevTools → Application →
|
||||
# Cookies, or use a logged-in curl session):
|
||||
curl -X POST localhost:3000/api/smime/enroll \
|
||||
-H 'Content-Type: application/json' \
|
||||
--cookie "<your session cookie>" \
|
||||
-d "{\"csrPem\": $(python3 -c 'import json,sys; print(json.dumps(open("/tmp/test.csr").read()))'), \"addresses\": [\"you@example.com\"]}"
|
||||
```
|
||||
### 2.2 Known gap — real but narrow
|
||||
|
||||
**Expect:** a JSON response with `certificatePem`, `chainPem`, `serialNumber`,
|
||||
`notAfter`. Verify it's a real cert: `openssl x509 -in <(echo "$certificatePem") -noout -text`
|
||||
should show issuer `VNCmail+ LOCAL DEV S/MIME CA`, your `you@example.com` in
|
||||
Subject Alternative Name, and Extended Key Usage including `E-mail
|
||||
Protection`.
|
||||
|
||||
### 2.2 Known gap — the important one
|
||||
|
||||
**Nobody can actually enroll from the UI today.** The plugin's Settings
|
||||
section still has no "Get a certificate" button, no client-side CSR
|
||||
generation, and no wiring to import an issued cert into its existing
|
||||
encrypted-at-rest key storage (`vnc/plugins/smime/src/pkcs12.js`'s
|
||||
AES-GCM+PBKDF2 convention). The plugin itself is confirmed *mounted*
|
||||
(privileged iframe, same-origin fetch works without a bridge) with 0 certs
|
||||
imported — that's expected, not a bug. This is real, well-scoped follow-up
|
||||
work, not blocked on anything — just deliberately not rushed overnight.
|
||||
No cross-testing yet between a **web-CA-issued** certificate and the
|
||||
**mobile** S/MIME implementation (§3) — worth doing once both have been
|
||||
used interactively at least once, not before. Also unchanged from
|
||||
yesterday: `SMIME_CA_DEV_LOCAL` is explicitly a stand-in for the real
|
||||
EJBCA (needs a client mTLS credential this environment doesn't have) —
|
||||
switching to production CA is a config change (`SMIME_CA_URL` +
|
||||
credentials), not a code change.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user