Bernd Rodler
3512f935d1
Multiple developers now work on this repo, and the only working deploy
trigger required pushing to GitHub - which contradicts the standing
GitLab-canonical policy for this repo - while every actual deploy was a
manual kubectl run against one environment (no prod exists at all).
Restructures deploy/k8s/ into base/ + overlays/{dev,prod}: overlays/dev
is a verified byte-for-byte no-op for the live sandbox (kubectl kustomize
diff against the old flat layout is empty), overlays/prod is scaffolded
but inert (placeholder hostname + JMAP_SERVER_URL, since neither a prod
hostname decision nor a prod Stalwart exist yet). deploy/k8s/ca/ (the
EJBCA internal CA) is untouched and never referenced by either overlay.
Adds .gitlab-ci.yml: verify (MR gate, no push/deploy) -> build+deploy-dev
(automatic on push to dev, one image name/tag-only environments, fixing
the old -dev/-beta naming split) -> promote (manual, protected
`production` environment, retags the exact dev digest via
`docker buildx imagetools create` - never rebuilds - and is left as a
documented TODO for the actual `kubectl apply` until prod is real).
Updates VNCMAIL-SETUP.md and deploy/k8s/README.md to describe the new
flow and correct the aspirational promotion description that assumed a
"production image" CI never actually built.
Also fixes a pre-existing lint error (no-control-regex false positive on
an intentional DN-sanitizing character class in lib/smime-ca/ejbca.ts)
that was blocking this commit's pre-commit hook - unrelated to this
change otherwise, confirmed already present on dev before this branch.
Runner/RBAC/registry setup is an infra prerequisite this commit cannot
provide - documented in the pipeline plan, not part of this diff.
169 lines
7.2 KiB
Markdown
169 lines
7.2 KiB
Markdown
# VNCmail+ — Admin Deployment Guide (microk8s)
|
||
|
||
Deploy VNCmail+ (VNC's Bulwark fork) as a container at **`vncmail.sandbox.vnc.de`**,
|
||
**alongside** the existing `bulwark.sandbox.vnc.de`.
|
||
|
||
> Why a container (not Vercel): Bulwark is stateful — it writes settings/admin/
|
||
> telemetry to `/app/data`, which needs persistent volumes.
|
||
|
||
## Structure — base + overlays
|
||
|
||
```
|
||
deploy/k8s/
|
||
base/ # shared manifest shapes (namespace-agnostic)
|
||
overlays/
|
||
dev/ # the live sandbox — vncmail.sandbox.vnc.de, namespace vncmail
|
||
prod/ # scaffolded, NOT YET LIVE — see "Production status" below
|
||
ca/ # separate, isolated EJBCA internal CA — see ca/README.md.
|
||
# Never composed with base/ or either overlay above.
|
||
```
|
||
|
||
`kubectl apply -k overlays/dev` (or `overlays/prod`, once real) instead of
|
||
applying `base/` directly — `base/` alone has no namespace and won't apply
|
||
meaningfully on its own.
|
||
|
||
## Routine deploys go through CI now
|
||
|
||
As of the GitLab CI/CD pipeline (`.gitlab-ci.yml`, see `../../VNCMAIL-SETUP.md`
|
||
§ CI/CD), **pushing to `dev` auto-builds and auto-deploys** — you should not
|
||
normally need to run `kubectl apply` for the sandbox by hand anymore. This
|
||
guide's manual steps below are for first-time setup, the one-time secret
|
||
creation CI deliberately never automates, and troubleshooting.
|
||
|
||
## Production status
|
||
|
||
**There is no production VNCmail+ deployment yet.** `overlays/prod/` exists
|
||
in the repo but is inert: its ingress hostname and its secret's
|
||
`JMAP_SERVER_URL` are both obvious placeholders (`vncmail.CHANGEME.invalid` /
|
||
`https://REPLACE-ME-prod-stalwart-not-yet-deployed.invalid`) that will fail
|
||
loudly rather than silently deploy against the wrong backend. Applying it
|
||
requires, in order: a real prod Stalwart instance to exist, a real hostname
|
||
decision, DNS, a real `secret.yaml`, and the `.gitlab-ci.yml` `promote` job's
|
||
`kubectl apply` step (currently a TODO placeholder) filled in. None of that
|
||
is CI's job to decide — it's an explicit, human-triggered event.
|
||
|
||
---
|
||
|
||
## 1. What you are deploying (per overlay)
|
||
|
||
| # | Object | File | Purpose |
|
||
|---|--------|------|---------|
|
||
| 1 | Namespace | `overlays/<env>/namespace.yaml` | Isolates the app (`vncmail` for dev, `vncmail-prod` for prod) |
|
||
| 2 | 4× PersistentVolumeClaim | `base/pvc.yaml` | `/app/data/{settings,admin,admin-state,telemetry}` |
|
||
| 3 | Secret `vncmail-env` | `overlays/<env>/secret.yaml` *(you create it)* | App config (JMAP URL, session secret, branding) |
|
||
| 4 | Image-pull secret | *(you create it — command below)* | Pull the (currently private) image |
|
||
| 5 | Deployment `vncmail-plus` | `base/deployment.yaml` (+ overlay patches) | The app pod |
|
||
| 6 | Service `vncmail-plus` | `base/service.yaml` | ClusterIP :80 → pod :3000 |
|
||
| 7 | Ingress `vncmail-plus` | `base/ingress.yaml` (+ overlay patches for prod) | TLS host |
|
||
|
||
**Image:** CI builds and pushes to `registry.gitlab.vnc.biz/gitlab-instance-b9b5cf2f/vncmail-plus`
|
||
(tag `sha-<sha>` per deploy, moving pointers `dev-latest`/`prod-latest`). The
|
||
`ghcr.io/brvncde-dotcom/vncmail-plus-dev` image referenced in `base/deployment.yaml`
|
||
is a legacy default only — CI overrides it per-deploy via `kubectl set image`,
|
||
so what's committed there never needs to track what's actually running.
|
||
|
||
---
|
||
|
||
## 2. Pre-flight — confirm 3 cluster values (2 min)
|
||
|
||
The manifests use microk8s defaults. **Copy the exact values the existing
|
||
Bulwark uses** so VNCmail+ matches your cluster:
|
||
|
||
```bash
|
||
# Find bulwark's ingress and read off its class + cert-manager annotations:
|
||
kubectl get ingress -A | grep -i bulwark
|
||
kubectl get ingress <bulwark-ingress-name> -n <bulwark-ns> -o yaml
|
||
|
||
# List available storage classes and ingress classes:
|
||
kubectl get sc
|
||
kubectl get ingressclass
|
||
kubectl get clusterissuer # cert-manager issuers (if used)
|
||
```
|
||
|
||
Then edit if they differ from the defaults below (in `base/`, so both overlays
|
||
pick up the fix):
|
||
|
||
| Value | Default in manifests | File to edit |
|
||
|-------|----------------------|--------------|
|
||
| StorageClass | `microk8s-hostpath` | `base/pvc.yaml` (all 4) |
|
||
| IngressClass | `public` | `base/ingress.yaml` |
|
||
| cert-manager issuer | `letsencrypt-prod` | `base/ingress.yaml` |
|
||
|
||
---
|
||
|
||
## 3. First-time setup (one-time, per environment — CI never does this)
|
||
|
||
```bash
|
||
cd deploy/k8s/overlays/dev # or overlays/prod, once real
|
||
|
||
# a) Image-pull secret — the registry package is private.
|
||
kubectl create secret docker-registry ghcr-pull \
|
||
--namespace vncmail \
|
||
--docker-server=ghcr.io \
|
||
--docker-username=brvncde-dotcom \
|
||
--docker-password='<GITHUB_PAT_read:packages>' \
|
||
--docker-email=br@vnc.biz
|
||
# Once CI has cut over to registry.gitlab.vnc.biz, this becomes a
|
||
# docker-registry secret for that registry instead — see VNCMAIL-SETUP.md.
|
||
|
||
# b) App config secret — copy the template, set a real SESSION_SECRET, apply.
|
||
cp secret.example.yaml secret.yaml
|
||
# edit secret.yaml: SESSION_SECRET: "$(openssl rand -base64 32)"
|
||
kubectl apply -f secret.yaml
|
||
|
||
# c) Everything else (namespace, PVCs, Deployment, Service, Ingress)
|
||
kubectl apply -k .
|
||
```
|
||
|
||
> Alternative to (a): make the registry package public, then delete the
|
||
> `imagePullSecrets:` block from `base/deployment.yaml`.
|
||
|
||
After this one-time setup, routine deploys to `dev` happen automatically via
|
||
CI on every push — see "Routine deploys go through CI now" above. This
|
||
section is for first-time bring-up (or `overlays/prod`, once it's real) and
|
||
troubleshooting, not the everyday path.
|
||
|
||
---
|
||
|
||
## 4. Verify
|
||
|
||
```bash
|
||
kubectl -n vncmail rollout status deploy/vncmail-plus # -> successfully rolled out
|
||
kubectl -n vncmail get pods,pvc,ingress
|
||
|
||
# DNS: point vncmail.sandbox.vnc.de at the same ingress IP as bulwark.sandbox.vnc.de.
|
||
# cert-manager issues TLS once DNS resolves. Then:
|
||
curl -sI https://vncmail.sandbox.vnc.de/api/health # -> HTTP/2 200
|
||
```
|
||
|
||
Open `https://vncmail.sandbox.vnc.de` and log in with a **full** email address
|
||
(e.g. `bernd.rodler@sandbox.vnc.de`) — Stalwart authenticates the full email, not
|
||
a bare username.
|
||
|
||
---
|
||
|
||
## 5. Update to a new build
|
||
|
||
Normally you don't — CI's `deploy-dev` job does this automatically on every
|
||
push to `dev`. To do it by hand (e.g. troubleshooting):
|
||
|
||
```bash
|
||
kubectl -n vncmail set image deploy/vncmail-plus \
|
||
vncmail-plus=registry.gitlab.vnc.biz/gitlab-instance-b9b5cf2f/vncmail-plus:sha-<sha>
|
||
```
|
||
|
||
Rollback: `kubectl -n vncmail rollout undo deploy/vncmail-plus`
|
||
|
||
---
|
||
|
||
## 6. Troubleshooting
|
||
|
||
| Symptom | Cause / fix |
|
||
|---------|-------------|
|
||
| Pod `ImagePullBackOff` | `ghcr-pull` secret missing/expired, or package still private. Recreate the secret (§3a) or make the package public. |
|
||
| Pod `CrashLoopBackOff`, logs show `EACCES`/permission on `/app/data` | Volume not writable by uid 1001. `securityContext.fsGroup: 1001` is set in `base/deployment.yaml` — keep it; some storage drivers also need it on the PVC. |
|
||
| PVC stuck `Pending` | Wrong `storageClassName` in `base/pvc.yaml`. Set it to one from `kubectl get sc`. |
|
||
| Ingress has no address / no cert | Wrong `ingressClassName` or cert issuer. Match bulwark's (§2). Check `kubectl -n vncmail describe ingress vncmail-plus`. |
|
||
| Login shows "Ein Fehler ist aufgetreten" | Use the **full** email (`user@sandbox.vnc.de`), not a bare username. |
|
||
| Can't reach Stalwart | Check `JMAP_SERVER_URL` in the secret = `https://stalwart.sandbox.vnc.de`. |
|