SRCmail/deploy/k8s/README.md

# VNCmail+ — Admin Deployment Guide (microk8s)

Deploy VNCmail+ (VNC's Bulwark fork) as a container at **`vncmail.sandbox.vnc.de`**,
**alongside** the existing `bulwark.sandbox.vnc.de`.

> Why a container (not Vercel): Bulwark is stateful — it writes settings/admin/
> telemetry to `/app/data`, which needs persistent volumes.

## Structure — base + overlays

```
deploy/k8s/
  base/               # shared manifest shapes (namespace-agnostic)
  overlays/
    dev/              # the live sandbox — vncmail.sandbox.vnc.de, namespace vncmail
    prod/             # scaffolded, NOT YET LIVE — see "Production status" below
  ca/                 # separate, isolated EJBCA internal CA — see ca/README.md.
                       # Never composed with base/ or either overlay above.
```

`kubectl apply -k overlays/dev` (or `overlays/prod`, once real) instead of
applying `base/` directly — `base/` alone has no namespace and won't apply
meaningfully on its own.

## Routine deploys go through CI + ArgoCD now

As of the GitLab CI/CD pipeline (`.gitlab-ci.yml`, see `../../VNCMAIL-SETUP.md`
§ CI/CD), **pushing to `dev` auto-builds and bumps the deploy tag; ArgoCD's
`vncmail-dev` Application applies it** — you should not normally need to run
`kubectl apply` for the sandbox by hand anymore, and CI never touches the
cluster directly (it only ever talks to the registry and to this git repo).
This guide's manual steps below are for first-time setup, the one-time
secret creation CI/ArgoCD deliberately never automate, and troubleshooting.

## Production status

**There is no production VNCmail+ deployment yet.** `overlays/prod/` exists
in the repo but is inert: its ingress hostname and its secret's
`JMAP_SERVER_URL` are both obvious placeholders (`vncmail.CHANGEME.invalid` /
`https://REPLACE-ME-prod-stalwart-not-yet-deployed.invalid`) that will fail
loudly rather than silently deploy against the wrong backend. Applying it
requires, in order: a real prod Stalwart instance to exist, a real hostname
decision, DNS, a real `secret.yaml`, and the `.gitlab-ci.yml` `promote` job's
`kubectl apply` step (currently a TODO placeholder) filled in. None of that
is CI's job to decide — it's an explicit, human-triggered event.

---

## 1. What you are deploying (per overlay)

| # | Object | File | Purpose |
|---|--------|------|---------|
| 1 | Namespace | `overlays/<env>/namespace.yaml` | Isolates the app (`vncmail` for dev, `vncmail-prod` for prod) |
| 2 | 4× PersistentVolumeClaim | `base/pvc.yaml` | `/app/data/{settings,admin,admin-state,telemetry}` |
| 3 | Secret `vncmail-env` | `overlays/<env>/secret.yaml` *(you create it)* | App config (JMAP URL, session secret, branding) |
| 4 | Image-pull secret | *(you create it — command below)* | Pull the (currently private) image |
| 5 | Deployment `vncmail-plus` | `base/deployment.yaml` (+ overlay patches) | The app pod |
| 6 | Service `vncmail-plus` | `base/service.yaml` | ClusterIP :80 → pod :3000 |
| 7 | Ingress `vncmail-plus` | `base/ingress.yaml` (+ overlay patches for prod) | TLS host |

**Image:** CI builds and pushes to `registry.gitlab.vnc.biz/gitlab-instance-b9b5cf2f/vncmail-plus`
(tag `sha-<sha>` per deploy, moving pointer `dev-latest`). The generic `vncmail-plus`
image name in `base/deployment.yaml` is a placeholder — kustomize's image-tag
Component replaces it with the real registry path on every deploy.

---

## 2. Pre-flight — confirm 3 cluster values (2 min)

The manifests use microk8s defaults. **Copy the exact values the existing
Bulwark uses** so VNCmail+ matches your cluster:

```bash
# Find bulwark's ingress and read off its class + cert-manager annotations:
kubectl get ingress -A | grep -i bulwark
kubectl get ingress <bulwark-ingress-name> -n <bulwark-ns> -o yaml

# List available storage classes and ingress classes:
kubectl get sc
kubectl get ingressclass
kubectl get clusterissuer      # cert-manager issuers (if used)
```

Then edit if they differ from the defaults below (in `base/`, so both overlays
pick up the fix):

| Value | Default in manifests | File to edit |
|-------|----------------------|--------------|
| StorageClass | `microk8s-hostpath` | `base/pvc.yaml` (all 4) |
| IngressClass | `public` | `base/ingress.yaml` |
| cert-manager issuer | `letsencrypt-prod` | `base/ingress.yaml` |

---

## 3. First-time setup (one-time, per environment — CI never does this)

```bash
cd deploy/k8s/overlays/dev     # or overlays/prod, once real

# a) Image-pull secret — the GitLab registry requires authentication.
#    Use a project deploy token with `read_registry` scope, or the CI job
#    token (short-lived — better for CI, not for long-running clusters).
kubectl create secret docker-registry gitlab-registry \
  --namespace vncmail \
  --docker-server=registry.gitlab.vnc.biz \
  --docker-username=<deploy-token-name> \
  --docker-password='<deploy-token-secret>' \
  --docker-email=ci@vnc.biz

# b) App config secret — copy the template, set a real SESSION_SECRET, apply.
cp secret.example.yaml secret.yaml
#    edit secret.yaml:  SESSION_SECRET: "$(openssl rand -base64 32)"
kubectl apply -f secret.yaml

# c) Everything else (namespace, PVCs, Deployment, Service, Ingress)
kubectl apply -k .
```

> Alternative to (a): make the GitLab container registry public for this
> project, then delete the `imagePullSecrets:` block from `base/deployment.yaml`.

After this one-time setup, routine deploys to `dev` happen automatically via
CI on every push — see "Routine deploys go through CI now" above. This
section is for first-time bring-up (or `overlays/prod`, once it's real) and
troubleshooting, not the everyday path.

---

## 4. Verify

```bash
kubectl -n vncmail rollout status deploy/vncmail-plus     # -> successfully rolled out
kubectl -n vncmail get pods,pvc,ingress

# DNS: point vncmail.sandbox.vnc.de at the same ingress IP as bulwark.sandbox.vnc.de.
# cert-manager issues TLS once DNS resolves. Then:
curl -sI https://vncmail.sandbox.vnc.de/api/health         # -> HTTP/2 200
```

Open `https://vncmail.sandbox.vnc.de` and log in with a **full** email address
(e.g. `bernd.rodler@sandbox.vnc.de`) — Stalwart authenticates the full email, not
a bare username.

---

## 5. Update to a new build

Normally you don't — CI's `bump-dev` job + ArgoCD's automated sync do this
on every push to `dev`. To do it by hand (e.g. troubleshooting, before
automated sync is turned on):

```bash
kubectl -n vncmail set image deploy/vncmail-plus \
  vncmail-plus=registry.gitlab.vnc.biz/gitlab-instance-b9b5cf2f/vncmail-plus:sha-<sha>
```

ArgoCD will overwrite this on its next sync unless you also update
`deploy/k8s/overlays/dev/image-tag/kustomization.yaml` to match — that file
is CI-owned (see its header comment), so a by-hand `set image` is only ever
a temporary override, not a real fix.

Rollback (bypassing ArgoCD temporarily): `kubectl -n vncmail rollout undo deploy/vncmail-plus`.
The real rollback is reverting the commit that bumped the tag and letting
ArgoCD re-sync.

---

## 6. Troubleshooting

| Symptom | Cause / fix |
|---------|-------------|
| Pod `ImagePullBackOff` | `gitlab-registry` secret missing/expired, or token lacks `read_registry`. Recreate the secret (§3a) or make the registry public. |
| Pod `CrashLoopBackOff`, logs show `EACCES`/permission on `/app/data` | Volume not writable by uid 1001. `securityContext.fsGroup: 1001` is set in `base/deployment.yaml` — keep it; some storage drivers also need it on the PVC. |
| PVC stuck `Pending` | Wrong `storageClassName` in `base/pvc.yaml`. Set it to one from `kubectl get sc`. |
| Ingress has no address / no cert | Wrong `ingressClassName` or cert issuer. Match bulwark's (§2). Check `kubectl -n vncmail describe ingress vncmail-plus`. |
| Login shows "Ein Fehler ist aufgetreten" | Use the **full** email (`user@sandbox.vnc.de`), not a bare username. |
| Can't reach Stalwart | Check `JMAP_SERVER_URL` in the secret = `https://stalwart.sandbox.vnc.de`. |