Bernd Rodler
15b189357e
Written from direct SSH inspection of both real clusters (node1-3 prod HA, dev-k8s-1-3 dev) done while building the GitLab CI + ArgoCD pipeline (MR !1) - not re-derived from the aspirational docs/manifests that predated that inspection. ARCHITECTURE.md: system diagram (clients, both clusters, Stalwart, EJBCA CA, the CI+ArgoCD flow) plus the storage-coupling fact that everything else hinges on - 4 RWO PVCs + strategy:Recreate is why the app is single-replica today. SANDBOX-DEV-MANUAL.md: day-to-day branch/MR/CI/ArgoCD flow, one-time bootstrap, troubleshooting, and what's explicitly out of scope for normal dev work (the CA, the still-inert prod overlay). PRODUCTION-SCALE-OUT-PLAN.md: phased path to a 100k+-user production deployment on node1-3 - breaking the storage coupling first (rook-ceph CephFS RWX as the fast path, migrating mutable state into the already-installed-but-unused CNPG Postgres as the correct one), then autoscaling, Stalwart's own scaling track, networking/edge, the observability gap (none found on either cluster), security hardening, load testing, DR, and the go-live sequence. Includes a "scale at any time" manual lever, not just HPA.
132 lines
5.6 KiB
Markdown
132 lines
5.6 KiB
Markdown
# VNCmail+ — Sandbox / Dev Manual
|
|
|
|
Practical, day-to-day guide for developing VNCmail+ and getting changes into
|
|
the sandbox (`dev-k8s-1/2/3` cluster). For the big picture see
|
|
[ARCHITECTURE.md](ARCHITECTURE.md); for how to eventually go live see
|
|
[PRODUCTION-SCALE-OUT-PLAN.md](PRODUCTION-SCALE-OUT-PLAN.md).
|
|
|
|
## 1. Repo & branches
|
|
|
|
- **Canonical remote**: `gitlab.vnc.biz/gitlab-instance-b9b5cf2f/vncmail-plus`
|
|
(GitHub `origin` is a passive mirror — never push feature work there).
|
|
- `main` = production (protected, fast-forward-only from `dev`, no direct pushes).
|
|
- `dev` = integration/default branch (protected, MR-required).
|
|
- `vnc/*` or `feature/*` = your working branches → MR into `dev`.
|
|
|
|
```bash
|
|
git clone git@gitlab.vnc.biz:gitlab-instance-b9b5cf2f/vncmail-plus.git
|
|
cd vncmail-plus
|
|
git checkout -b vnc/my-change dev
|
|
```
|
|
|
|
## 2. Local development
|
|
|
|
```bash
|
|
npm ci
|
|
cp .env.dev.example .env.local # built-in mock JMAP server, DEV_MOCK_JMAP=true
|
|
npm run dev # http://localhost:3000, log in with any username/password
|
|
```
|
|
|
|
The mock JMAP server (`/api/dev-jmap`) means you don't need a real Stalwart
|
|
instance for UI work. Useful scripts:
|
|
|
|
```bash
|
|
npm run typecheck # tsc --noEmit
|
|
npm run lint # eslint .
|
|
npm run test:translations # vitest, fast
|
|
npm run test:integration # bash integration/run-tests.sh — spins up a REAL
|
|
# Stalwart via docker-compose (integration/), slower
|
|
```
|
|
|
|
For Electron:
|
|
|
|
```bash
|
|
npm run electron:dev # build:standalone + build:electron + launch
|
|
npm run test:electron # Playwright, no OS permissions needed (Electron CDP)
|
|
```
|
|
|
|
## 3. Opening a change
|
|
|
|
1. Push your branch, open a Merge Request into `dev` on GitLab.
|
|
2. The `verify` CI job runs automatically: typecheck, lint, unit tests, build.
|
|
**This is a required check** — it never pushes an image or touches any
|
|
cluster, just proves the branch builds.
|
|
3. Get it reviewed, merge.
|
|
|
|
## 4. What happens after merge — the pipeline
|
|
|
|
```
|
|
merge to dev
|
|
→ CI `build`: docker build, push registry.gitlab.vnc.biz/.../vncmail-plus:sha-<sha>
|
|
→ CI `bump-dev`: commits that tag into
|
|
deploy/k8s/overlays/dev/image-tag/kustomization.yaml (a small file CI
|
|
owns — don't hand-edit it, your edit will be overwritten on the next push)
|
|
→ ArgoCD's `vncmail-dev` Application notices the git change and syncs
|
|
```
|
|
|
|
CI never runs `kubectl` and holds no cluster credentials — it only talks to
|
|
the registry and to this git repo. ArgoCD (already running on `dev-k8s`,
|
|
found idle when this pipeline was built) does the actual applying.
|
|
|
|
**Until the one-time bootstrap below is done**, `vncmail-dev`'s sync policy
|
|
is manual on purpose — check its status:
|
|
|
|
```bash
|
|
ssh dev-k8s-1 # or dev-k8s-2 / dev-k8s-3
|
|
export PATH=/snap/bin:$PATH
|
|
microk8s kubectl -n argocd get application vncmail-dev
|
|
```
|
|
|
|
Or the UI: `https://argo.devcluster.vnc.de` (`admin` / see
|
|
`kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d` —
|
|
rotate after first login).
|
|
|
|
## 5. One-time bootstrap (already done or being done — see MR !1 / VNCMAIL-SETUP.md)
|
|
|
|
Secrets are **never** managed by CI or ArgoCD — created once, by hand:
|
|
|
|
```bash
|
|
kubectl create secret docker-registry ghcr-pull -n vncmail ... # or make the registry package public
|
|
cp deploy/k8s/overlays/dev/secret.example.yaml secret.yaml # edit SESSION_SECRET
|
|
kubectl apply -f secret.yaml
|
|
```
|
|
|
|
Then a first manual Sync in the ArgoCD UI. Once that's clean, flip
|
|
`deploy/argocd/vncmail-dev-app.yaml`'s `automated:` block on and re-apply —
|
|
from then on, every merge to `dev` deploys itself.
|
|
|
|
## 6. Checking on the running sandbox
|
|
|
|
```bash
|
|
ssh dev-k8s-1
|
|
export PATH=/snap/bin:$PATH
|
|
microk8s kubectl -n vncmail get pods,pvc,ingress
|
|
microk8s kubectl -n vncmail logs deploy/vncmail-plus --tail=100 -f
|
|
microk8s kubectl -n vncmail rollout status deploy/vncmail-plus
|
|
```
|
|
|
|
No local kubeconfig is assumed — everything above is run over `ssh` directly
|
|
on a cluster node (`node1/2/3` for prod, `dev-k8s-1/2/3` for dev), using the
|
|
`microk8s.kubectl` binaries installed there (put `/snap/bin` on `PATH`).
|
|
|
|
## 7. Troubleshooting
|
|
|
|
| Symptom | Likely cause |
|
|
|---|---|
|
|
| ArgoCD shows `vncmail-dev` as `ComparisonError` / SSH handshake failed | The ArgoCD deploy key hasn't been added to GitLab yet (Project → Settings → Repository → Deploy keys) |
|
|
| `bump-dev`/`bump-prod` CI job fails to push | `CI_JOB_TOKEN` self-push isn't enabled (Settings → CI/CD → Job token permissions), and no `GITLAB_PUSH_TOKEN` variable is set as a fallback |
|
|
| Pod `ImagePullBackOff` | Registry pull secret missing/expired, or package still private |
|
|
| Pod `CrashLoopBackOff`, `EACCES` on `/app/data` | `securityContext.fsGroup: 1001` must stay set — some storage drivers also need it on the PVC itself |
|
|
| Ingress has no address / no cert | Wrong `ingressClassName` (must be `traefik` on both real clusters) or a missing `ClusterIssuer` — `node1-3` (prod) has **none** configured today |
|
|
| "Ein Fehler ist aufgetreten" on login | Use the full email address (`user@sandbox.vnc.de`), not a bare username — Stalwart auths on the full address |
|
|
|
|
## 8. Don't touch (out of scope for day-to-day dev)
|
|
|
|
- `deploy/k8s/ca/` (EJBCA internal CA) — separate namespace `vnc-ca`, own
|
|
README, root-key ceremony is a manual human-only runbook. Never wire CI or
|
|
ArgoCD automation into it.
|
|
- `overlays/prod/` and `deploy/argocd/vncmail-prod-app.yaml` — scaffolded,
|
|
deliberately inert (placeholder hostname, no prod Stalwart, `node1-3` not
|
|
yet registered with ArgoCD). See [PRODUCTION-SCALE-OUT-PLAN.md](PRODUCTION-SCALE-OUT-PLAN.md)
|
|
for what has to happen before any of that becomes real.
|