Files
SRCmail/VNCMAIL-SETUP.md
T
Bernd Rodler 177b2aca57 feat(ci): pivot to ArgoCD GitOps, fix Traefik ingress after real-cluster check
Direct SSH access to the actual clusters (node1-3 "prod" HA, dev-k8s-1-3
"dev") revealed two things that made the previous design wrong:

1. Neither cluster has vncmail/vnc-ca namespaces or a bulwark ingress at
   all - the "live sandbox" referenced in this repo's docs/manifests was
   never actually applied anywhere. Both ingress.yaml's ingressClassName
   (public) and cert-manager issuer (letsencrypt-prod) were also wrong:
   both clusters run Traefik (class is literally named `traefik`), and
   only dev-k8s has any ClusterIssuer at all (`letsencrypt-staging`).
   node1-3 has zero ClusterIssuers configured.

2. dev-k8s already has ArgoCD installed, idle, zero Applications - more
   idiomatic to use it than have GitLab Runner execute kubectl directly.

Pivots .gitlab-ci.yml: build+push image, then commit the tag into a small
per-overlay Component (overlays/{dev,prod}/image-tag/) that ArgoCD's
Application watches - CI never touches the cluster, only the registry and
this repo. dev's Application (vncmail-dev) is registered and applied
already (manual sync for now, until the one-time namespace secret
bootstrap is done - see VNCMAIL-SETUP.md). prod's Application is
scaffolded in deploy/argocd/ but deliberately not applied - it targets a
different cluster (node1-3) that isn't registered with ArgoCD yet, and
there's still no real prod hostname/Stalwart/ClusterIssuer.

Fixes base/ingress.yaml to the real ingressClassName: traefik (was the
nginx-style `public`, which doesn't exist on either cluster) and gives
each overlay its own cert-manager issuer patch instead of one hardcoded
value, since dev and prod need different (or, for prod, nonexistent)
issuers.
2026-08-05 13:06:22 +02:00

8.8 KiB

VNCmail+ — setup & deploy runbook

VNCmail+ is VNC's fork of Bulwark, a Next.js (App Router) JMAP webmail client for Stalwart. Stalwart is the source of truth; VNCmail+ is the UI. It deploys as a container on Kubernetes (microk8s) at vncmail.sandbox.vnc.de — see deploy/k8s/.

License: AGPL-3.0. Serving a modified VNCmail+ to users over the network obligates VNC to offer those users the corresponding source. Keeping this fork public (with a "Source" link in the imprint/UI) satisfies that. Loop in legal before a public/customer-facing launch if a closed fork is ever desired.

Architecture — why a container, not Vercel

  • Bulwark is a stateful, long-lived server: it persists settings-sync, admin config/state, and telemetry to a local data directory (/app/data/*).
  • Vercel serverless was tried and dropped — its filesystem is read-only except /tmp, so Bulwark's mkdir ./data crashes (ENOENT /var/task/data). You cannot point its data dirs at a remote host either (they're POSIX paths, not URLs). Bulwark's native model is a container + persistent volumes.
  • So VNCmail+ runs as a Docker image with 4 persistent volumes, exactly like the existing bulwark.sandbox.vnc.de.
  • JMAP calls go through server-side /api/* routes (proxy.ts) → server-to- server to Stalwart, no browser CORS. Config is runtime-read.

Branches (dev-first)

Branch Role
main Production. Only updated by git merge --ff-only dev, then an explicit manual promote in CI. No prod environment exists yet — see "CI/CD" below.
dev Integration + QA — default working branch. Every push auto-builds and auto-deploys to the sandbox (vncmail.sandbox.vnc.de).
vnc/* Feature branches for UI work (branch off dev, MR into dev — required, gated by CI).

All VNC customization lives under vnc/ (see vnc/VNC-CHANGES.md).

CI/CD — GitLab (canonical) + ArgoCD GitOps, Vercel-style dev→prod

Multiple developers work on this repo now. .gitlab-ci.yml on gitlab.vnc.biz (the canonical remote — GitHub origin is a passive mirror, not where CI or deploys happen) builds images and bumps a tag in git; ArgoCD does the actual deploying — already installed and idle on the dev-k8s-1/2/3 cluster, discovered when standing this up. GitLab CI needs zero cluster credentials as a result.

Two real clusters, confirmed by direct inspection:

Cluster Role Notes
dev-k8s-1/2/3 dev/sandbox ~hours old when set up here. Traefik, metallb, cert-manager (letsencrypt-staging issuer only), ArgoCD already running.
node1/node2/node3 prod (HA) Older, rook-ceph+traefik+metallb+cert-manager, but zero apps and zero ClusterIssuers — genuinely a clean slate.

Neither cluster had a vncmail namespace, vnc-ca namespace, or bulwark ingress — the "live sandbox at vncmail.sandbox.vnc.de" referenced earlier in this doc's history was aspirational (manifests + docs existed, nothing was ever actually applied). The ingress manifests also assumed nginx (class: public, an nginx body-size annotation) — fixed to Traefik's real ingressClassName: traefik (Traefik has no default body-size cap, so no replacement annotation is needed).

Flow:

  1. MR into devverify stage (typecheck/lint/unit test/build). Required check — no push, no deploy.
  2. Merge to devbuild pushes one image, registry.gitlab.vnc.biz/.../vncmail-plus:sha-<sha>, then bump-dev commits that tag into deploy/k8s/overlays/dev/image-tag/kustomization.yaml ([skip ci]). ArgoCD's vncmail-dev Application picks up the git change.
  3. Merge to main (fast-forward only, see below) → bump-prod points overlays/prod/image-tag/ at that same tag — no rebuild. The actual promotion gate is a human clicking Sync on the vncmail-prod ArgoCD Application, which is permanently manual-sync (never automated) — that's the Vercel-style "Promote to Production" button, just living in ArgoCD's UI instead of GitLab's.

What's left to wire up (one-time, human steps)

  1. Add the ArgoCD deploy key to GitLab — Project → Settings → Repository → Deploy keys → add (read-only is enough):
    ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOURjX/Y9zfB785DyLEF1GUq4HhWujrqeXag8oxdMciq argocd@dev-k8s (vncmail-plus read-only)
    
    Until this is added, vncmail-dev's ArgoCD Application (already created, kubectl -n argocd get application vncmail-dev) shows a benign ComparisonError (SSH handshake failing) — expected, not a bug.
  2. Let CI push tag-bumps back to this repo — either enable "this project can be accessed by CI/CD job tokens from other projects" → actually simpler: Settings → CI/CD → Job token permissions → allow this project's own job token to push to itself, OR create a Project Access Token (write_repository scope) and add it as a masked CI/CD variable GITLAB_PUSH_TOKEN (the pipeline tries that first, falls back to CI_JOB_TOKEN).
  3. One-time namespace bootstrap (CI/ArgoCD deliberately never manage secret contents — see deploy/k8s/README.md §3):
    # against dev-k8s (ArgoCD's CreateNamespace=true will make `vncmail` on
    # first sync, or create it yourself first — either order works)
    kubectl create secret docker-registry ghcr-pull -n vncmail ...   # or make the GHCR package public
    cp deploy/k8s/overlays/dev/secret.example.yaml secret.yaml   # edit SESSION_SECRET
    kubectl apply -f secret.yaml
    
  4. First sync — ArgoCD UI at https://argo.devcluster.vnc.de (username admin, password: kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d — rotate it after logging in once) → vncmail-dev → Sync. Once that's clean, flip deploy/argocd/vncmail-dev-app.yaml's commented-out automated: block on and re-apply, so dev auto-syncs on every push from then on.
  5. Production (later, deliberately not wired yet): decide a real hostname, stand up prod Stalwart, register node1-3 as an ArgoCD-managed cluster, apply deploy/argocd/vncmail-prod-app.yaml, fill in real overlays/prod values, create a real ClusterIssuer on node1-3 (there isn't one today), then click Sync once — deliberately not before.

Historical note: the old -dev/-beta GHCR image-name split (.github/workflows/docker-publish.yml) is retired by this — one image name now, environment lives only in the tag.

Deploy (Kubernetes / microk8s)

Full runbook: deploy/k8s/README.md. In short:

  1. CI (above) builds and pushes the image, one name/many tags, to GitLab's registry.
  2. kubectl apply -k deploy/k8s/overlays/dev (or overlays/prod, once real) — base manifests (namespace, 4 PVCs, deployment, service, ingress) live in deploy/k8s/base/, environment differences (namespace, hostname, replica count) are overlay patches.
  3. DNS + a secret.yaml (from the overlay's secret.example.yaml, gitignored, created once by hand — CI never manages secret contents) + an image-pull secret are the remaining manual, human, one-time steps per environment.

Runs alongside the existing bulwark.sandbox.vnc.de. Match your cluster's StorageClass / IngressClass / cert issuer to bulwark's (see the runbook).

Deploy workflow (dev-first — ALWAYS)

Same flow as every other VNC/SRC repo, now enforced structurally by CI rather than by convention:

  1. Work on dev (or vnc/* → MR into dev, CI-gated). Merge → auto-builds and auto-deploys to vncmail.sandbox.vnc.de. QA there.
  2. Promote to production only on explicit go-live — merge devmain:
    git log dev..main            # MUST be empty — main must have nothing dev lacks (else prod would revert)
    git checkout main && git merge --ff-only dev
    git push gitlab main         # never GitHub — opens the manual `promote` job, does not run it
    git checkout dev
    
    Then click promote in the GitLab pipeline UI (protected production environment — requires the right role) once prod actually exists (see "CI/CD" above). Never push straight to main. Never let a dev→main merge silently revert prod.

Syncing upstream (Bulwark releases)

Bring upstream into dev (NOT main), integrate + QA on the dev image, then promote as above:

git fetch upstream
git checkout dev && git merge upstream/main    # resolve conflicts via vnc/VNC-CHANGES.md; QA on preview

Auth

Basic auth via Stalwart is the default — users sign in with their @sandbox.vnc.de address + password; VNCmail+ authenticates them over JMAP. No extra config. (SSO via vncdirectory/OIDC is a later option — see vnc/vercel.env.template.)