docs: update test basis — admin AI console + S/MIME web enrollment now real
Both were "known gaps" in the original doc; rewrite those sections to reflect the actual shipped, live-verified state and note what's newly open instead (BYOK allow-list is advisory-only, consent text has no client-side reader yet).
This commit is contained in:
Bernd Rodler
@@ -80,67 +80,87 @@ The demo/mock inbox includes a "Villa sul Lago" booking confirmation email.
|
|||||||
|
|
||||||
### 1.5 Known gaps — don't be surprised by these
|
### 1.5 Known gaps — don't be surprised by these
|
||||||
|
|
||||||
- **No admin UI yet for any of the above.** Seat totals, model allow-lists,
|
|
||||||
per-class on/off, usage — all real on the backend (§1.2, §1.3) but only
|
|
||||||
reachable via `curl`/direct API calls today. A full spec + visual mockup
|
|
||||||
for the admin console exists at
|
|
||||||
[`docs/ADMIN-AI-POLICY-CONSOLE-SPEC.md`](ADMIN-AI-POLICY-CONSOLE-SPEC.md)
|
|
||||||
— presented for approval, **intentionally not built** pending sign-off.
|
|
||||||
- **BYOK provider allow-listing does not exist at all yet** — a user can
|
|
||||||
point a Public profile at literally any URL. Not a regression, just never
|
|
||||||
built; see the spec's §6.1 for the proposed (client-side-only) approach.
|
|
||||||
- Tier (base/standard/pro) is stored but doesn't change behavior yet — only
|
- Tier (base/standard/pro) is stored but doesn't change behavior yet — only
|
||||||
`seatsTotal` gates anything.
|
`seatsTotal` gates anything. The admin console (§1.6) labels the picker
|
||||||
|
as cosmetic for exactly this reason.
|
||||||
|
|
||||||
|
### 1.6 Admin AI Policy console — now built, test as an admin
|
||||||
|
|
||||||
|
Approved and implemented after the spec+mockup review. Admin panel → **AI**
|
||||||
|
(new tab, next to Policy).
|
||||||
|
|
||||||
|
1. **Provider classes** — toggle Server off, then try the Server class as a
|
||||||
|
user. **Expect:** 403 "disabled by admin policy" instead of a normal
|
||||||
|
answer (real enforcement in `/api/ai/server/chat`, not just a hidden
|
||||||
|
radio button). Toggle back on.
|
||||||
|
2. **Server model allow-list** — switch to "Restrict to selected", add one
|
||||||
|
model name, save. **Expect:** the model picker in Settings → AI
|
||||||
|
Assistant now shows only that model, *and* a direct API call with a
|
||||||
|
different model name gets 403 (checked at the chat chokepoint too).
|
||||||
|
3. **Seats** — change "Seats licensed", watch the "N of M assigned" count
|
||||||
|
update; **Revoke** a seat and confirm that user gets 402 again on next
|
||||||
|
use (frees the seat for someone else, matches `checkAndAssignSeat`'s
|
||||||
|
first-use semantics).
|
||||||
|
4. **Usage** — real rows from `ai-metering.jsonl`, not placeholders — should
|
||||||
|
match whatever calls you made in §1.2.
|
||||||
|
5. **Retrieval & consent** — toggle retrieval off, then ask a mail-grounded
|
||||||
|
question. **Expect:** 403 from `/api/ai/retrieve`, plain chat still
|
||||||
|
works. Edit the consent text, bump the version — no user-facing consent
|
||||||
|
*prompt* is wired up to read this yet (that's the one open item: the
|
||||||
|
text/version are stored and served over `/api/ai/policy`, but no UI
|
||||||
|
currently shows it to end users before first BYOK use — worth flagging
|
||||||
|
if that matters before wider rollout).
|
||||||
|
|
||||||
|
### 1.7 Known gaps, updated
|
||||||
|
|
||||||
|
- BYOK provider allow-listing is real now but **client-side/advisory
|
||||||
|
only** — a user's own browser checks it at profile-save time
|
||||||
|
(`ai-assistant-settings.tsx`), there's no server-side network boundary
|
||||||
|
enforcing it. Matches the spec's §6.1 recommendation; revisit if that
|
||||||
|
needs to become a hard boundary.
|
||||||
|
- The consent text/version admins set has no reader yet on the client side
|
||||||
|
(see §1.6 point 5) — stored and served, not yet displayed.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 2. S/MIME — web
|
## 2. S/MIME — web
|
||||||
|
|
||||||
### 2.1 What to test today
|
### 2.1 What to test today — now the full UI flow
|
||||||
|
|
||||||
With `SMIME_CA_DEV_LOCAL=true` set, the CA-issuance backend is real and
|
With `SMIME_CA_DEV_LOCAL=true` set, enrollment is fully wired end to end and
|
||||||
independently verified (4 passing tests: CSR signature verification, full
|
live-verified (not just unit-tested): Settings → S/MIME → **Get a
|
||||||
chain verification via `leaf.verify(caCert)`, correct SAN/rfc822Name
|
certificate** → enter a storage passphrase → a real certificate comes back
|
||||||
addresses, `emailProtection` EKU present — see
|
(RSA-2048, correct validity window, real fingerprint) and appears in "Your
|
||||||
`lib/smime-ca/__tests__/local-dev-provider.test.ts`).
|
keys". **Unlock** with the same passphrase round-trips correctly through
|
||||||
|
the identical encrypted-at-rest path a PKCS#12 import would use.
|
||||||
|
|
||||||
There is **no user-facing "Enroll" button wired up yet** — that was a
|
1. Log in via "Anmelden" (real session-cookie flow, not "Demo starten" —
|
||||||
deliberate scope decision made overnight (see the commit message on
|
see the caveat at the top of this doc).
|
||||||
`c2c07293`), not an oversight: this plugin has known, still-open
|
2. Settings → S/MIME → **Get a certificate**.
|
||||||
security-audit findings, and generating brand-new CSR/key-import code
|
3. Enter any passphrase, confirm.
|
||||||
against it at 1am risked introducing a new one rather than closing an old
|
4. **Expect:** a new key card appears — `RSA-2048 · valid <today> – <+397
|
||||||
one. So today, test the backend directly:
|
days>`, a real SHA-256 fingerprint, Unlock/Delete buttons.
|
||||||
|
5. Click **Unlock**, enter the same passphrase. **Expect:** button flips to
|
||||||
|
**Lock** — the private key decrypted correctly.
|
||||||
|
6. Compose a message to yourself, toggle Sign/Encrypt, send, then open it —
|
||||||
|
this exercises the CA-issued cert through the plugin's existing
|
||||||
|
sign/encrypt/decrypt/verify code paths (unchanged by tonight's work,
|
||||||
|
since the key record shape is identical to a PKCS#12 import's).
|
||||||
|
|
||||||
```bash
|
The CA-issuance backend itself is additionally covered by 4 independent
|
||||||
# Generate a throwaway CSR (any tool - openssl shown here):
|
unit tests (CSR signature verification, full chain verification via
|
||||||
openssl req -new -newkey rsa:2048 -nodes -keyout /tmp/test.key \
|
`leaf.verify(caCert)`, correct SAN/rfc822Name addresses, `emailProtection`
|
||||||
-out /tmp/test.csr -subj "/CN=Test User"
|
EKU present) — see `lib/smime-ca/__tests__/local-dev-provider.test.ts`.
|
||||||
|
|
||||||
# Call the enrollment route directly (needs an authenticated session cookie
|
### 2.2 Known gap — real but narrow
|
||||||
# from the "Anmelden" login above - copy it from DevTools → Application →
|
|
||||||
# Cookies, or use a logged-in curl session):
|
|
||||||
curl -X POST localhost:3000/api/smime/enroll \
|
|
||||||
-H 'Content-Type: application/json' \
|
|
||||||
--cookie "<your session cookie>" \
|
|
||||||
-d "{\"csrPem\": $(python3 -c 'import json,sys; print(json.dumps(open("/tmp/test.csr").read()))'), \"addresses\": [\"you@example.com\"]}"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Expect:** a JSON response with `certificatePem`, `chainPem`, `serialNumber`,
|
No cross-testing yet between a **web-CA-issued** certificate and the
|
||||||
`notAfter`. Verify it's a real cert: `openssl x509 -in <(echo "$certificatePem") -noout -text`
|
**mobile** S/MIME implementation (§3) — worth doing once both have been
|
||||||
should show issuer `VNCmail+ LOCAL DEV S/MIME CA`, your `you@example.com` in
|
used interactively at least once, not before. Also unchanged from
|
||||||
Subject Alternative Name, and Extended Key Usage including `E-mail
|
yesterday: `SMIME_CA_DEV_LOCAL` is explicitly a stand-in for the real
|
||||||
Protection`.
|
EJBCA (needs a client mTLS credential this environment doesn't have) —
|
||||||
|
switching to production CA is a config change (`SMIME_CA_URL` +
|
||||||
### 2.2 Known gap — the important one
|
credentials), not a code change.
|
||||||
|
|
||||||
**Nobody can actually enroll from the UI today.** The plugin's Settings
|
|
||||||
section still has no "Get a certificate" button, no client-side CSR
|
|
||||||
generation, and no wiring to import an issued cert into its existing
|
|
||||||
encrypted-at-rest key storage (`vnc/plugins/smime/src/pkcs12.js`'s
|
|
||||||
AES-GCM+PBKDF2 convention). The plugin itself is confirmed *mounted*
|
|
||||||
(privileged iframe, same-origin fetch works without a bridge) with 0 certs
|
|
||||||
imported — that's expected, not a bug. This is real, well-scoped follow-up
|
|
||||||
work, not blocked on anything — just deliberately not rushed overnight.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user