vnctalk-prosody/AGENTS.md

# AGENTS.md — vnctalk-prosody

## What this repo is

Dockerized Prosody **13.0.6** XMPP server for VNCtalk, built against **Lua 5.4** on Alpine 3.23. The Dockerfile compiles Prosody from source, applies `.patch` unified diffs to upstream core files, and layers on custom Lua modules from `vnctalk/`. There is no Lua package manager, linter, or unit-test framework; verification is image-build + an optional pytest + slixmpp integration suite against a running server.

> **Version drift warning:** `patches/README.md`, `CLAUDE.md`, and `helm/prosody/Chart.yaml` (`appVersion: 0.11.6`) still reference older Prosody versions. The **Dockerfile is the source of truth** (downloads `prosody-13.0.6.tar.gz`, `--lua-version=5.4`). `argo/prosody.yaml` overrides the helm tag to `release-13.0.6-vnc`. Trust the Dockerfile when docs disagree.

## Build & verification

- **Build:** `docker build -t <tag> .` (multi-stage Alpine; builder compiles Prosody, final stage copies `/usr/local/`).
- **Build gotcha:** `./configure --idn-library=idn` is mandatory. Prosody 0.12+ defaults to ICU, but the runtime image ships no ICU data files (`U_FILE_ACCESS_ERROR`); libidn is used instead. The builder also passes `--lua-version=5.4`.
- **Docker-compose test harness** (`docker-compose.yml`): postgres + an aiohttp mock (stands in for auth/FCM/file-share/avatar backends) + prosody. Convenience wrappers in the `Makefile`:
  - `make up` — build + start postgres/mocks/prosody (run pytest from host against published ports).
  - `make down` / `make logs`.
  - `make test` runs the suite in the `tester` service, but `tester` is gated behind the `ci` profile (`profiles: ["ci"]`) — invoke as `docker compose --profile ci run --rm tester pytest . -v -c pytest.ini`. Easier: use `run-compose-tests.sh` (see below).
- **`run-compose-tests.sh`** is the recommended one-shot runner: creates `tests/venv`, brings up the stack, waits for prosody health, and runs the full suite from the host with all env vars set. Flags: `--no-down` (leave stack up), `--down` (tear down only). It auto-remaps telnet to 5583 if 5582 is taken (e.g. by a stale kubectl port-forward).
- **`run-tests.sh` / `run-all-tests.sh`** target an **external microlab deployment** (hardcoded hosts/creds), not the compose harness. They require an SSH tunnel to PostgreSQL (`ssh -p 60024 … -L 14322:127.0.0.1:5432`) and a `kubectl port-forward services/telnet 5582:5582 -n prosody`. Not for local image testing.
- `test.sh` is a **manual smoke test**: `./test.sh <image-tag>` runs the image with example env vars. Not automated.
- **Python integration suite** in `tests/` (pytest + slixmpp, `~86` tests across `test_01`–`test_12`). See `tests/README.md`, `tests/AUDIT.md`, `tests/MANUAL_TESTS.md`.
  - Set up: `python3 -m venv tests/venv && source tests/venv/bin/activate && pip install -r tests/requirements.txt`
  - **Always pass `-c tests/pytest.ini`** — it sets `asyncio_mode = auto`; without it collection fails with an asyncio error.
  - Dry run (no server, skips live-only tests): `pytest tests/ -v --skip-live -c tests/pytest.ini`
  - Single file: `pytest tests/test_05_patches.py -v -c tests/pytest.ini`
  - Collect-only sanity check: `pytest tests/ --collect-only -c tests/pytest.ini`
  - Against a server: set `XMPP_HOST`, `XMPP_PORT`, `XMPP_JID`, `XMPP_PASSWORD`, `XMPP_DOMAIN`, `MUC_DOMAIN`. For full coverage also `XMPP_JID2`/`XMPP_PASSWORD2` (distinct account), `REST_URL`, `BOSH_URL`, `WS_URL`, `ADMIN_TELNET_HOST`/`ADMIN_TELNET_PORT`, `PG_HOST`/`PG_PORT`/`PG_USER`/`PG_PASSWORD`/`PG_DB`, `MOCK_URL` (side-effect assertions), and `REST_USER`/`REST_PASSWORD` if `/rest` is behind HTTP Basic Auth. SSL verification is off by default (containers use self-signed certs); add `--verify-ssl` only with a real cert.

## Code layout

| Path | Purpose |
|------|---------|
| `Dockerfile` | Multi-stage build. Builder applies `patches/*.patch` to the extracted Prosody source, then `./configure --idn-library=idn --lua-version=5.4 && make install`. Final stage copies the built tree + `vnctalk/` modules + `config/`. |
| `patches/` | Unified diffs (`a/...`/`b/...` headers, paths relative to the Prosody source root) applied in the builder with `patch -p1 --fuzz=0` **before** configure. Applied via glob — **adding/removing a `.patch` needs no Dockerfile change.** `--fuzz=0` means upstream drift fails the build loudly. `patches/README.md` documents each patch's intent (but mislabels the base version — see drift warning above); `upgrade-plan.md` tracks per-version disposition. |
| `vnctalk/` | VNCtalk-original Prosody modules, copied wholesale to `/usr/local/lib/prosody/modules/`. |
| `config/` | Runtime templates and shell scripts (rendered/run at container start). |
| `tests/` | pytest suite + `mocks/` aiohttp mock service + `Dockerfile` for the tester image. |
| `helm/prosody/` | Helm chart for Kubernetes deployment (`Chart.yaml`, `values.yaml`, `templates/`). `appVersion` in `Chart.yaml` is stale; the actual image tag is set in `argo/prosody.yaml`. |
| `argo/prosody.yaml` | ArgoCD `Application` manifest — the canonical deploy config (image tag, ingress, env). Useful reference for production wiring. |
| `db-customization/` | PostgreSQL schema/migration SQL for the Prosody 13 DB (`prosody-13-migration-once.sql`, rules/triggers, no-owner variants). Applied out-of-band, not by the container. |
| `scan/` | CI artifact output (Trivy image-scan results), not source. |
| `upgrade-plan.md`, `PATCHES_AND_MODULES.md` | Reference docs tracking patch/module disposition across versions. |
| Root `*.log`, `testrun-*.md`, `m1-manual-tasks.md`, `db-change-analysis.md`, `vc-set.xml` | Working scratch / run-report artifacts from past test sessions, not source. |

### Patch vs. module — where a change belongs

- **`patches/`** for changes to *upstream* Prosody behavior (core or bundled modules: `mod_mam`, `mod_muc`, `mod_carbons`, `muc.lib.lua`, `register.lib.lua`, `hidden.lib.lua`, `mod_muc_unique`, …). Regenerate with `diff -u pristine/<path> patched/<path>` keeping the `a/<path>`/`b/<path>` header form. Document intent in `patches/README.md`.
- **`vnctalk/`** for VNCtalk-specific new functionality (`mod_vnc_*`, `mod_http_rest`, `mod_alias`, `mod_webpresence`, …).
- When adding a module of either kind, also enable it in `config/prosody.cfg.lua.template` (global `modules_enabled`, or the `modules_enabled` of the specific host/component), or it installs but stays inactive.

## Configuration

- The real config source is `config/prosody.cfg.lua.template`. At startup `startup.sh` renders it with `envsubst` to `/etc/prosody/prosody.cfg.lua`. **Edit the template, not a generated `.cfg.lua`.**
- Template variables use `${VAR}` syntax. The full set (verify here before assuming a knob doesn't exist): `prosodyDomain`, `prosodyDBhost`, `prosodyDBname`, `prosodyDBuser`, `prosodyDBpass`, `prosodyDBport`, `hybridaAuthUrl`, `fcmApiKey`, `fcm_api_url`, `del_api_url`, `fileShareBaseUrl`, `fileShareSecret`, `avatarUploadUrl`, `avatarUploadUser`, `avatarUploadPass`, `componentSecret`, `DEFAULT_JITSI_CONFERENCE`, `log_slow_events_threshold`, `SMACKS_HIBERNATION_TIME`, `C2S_STANZA_SIZE_LIMIT`, `S2S_STANZA_SIZE_LIMIT`, `PROSODY_LOG_LEVEL` (set by `startup.sh` from `PROSODY_DEBUG=true`).
- **Env-var naming gotcha:** the template uses snake_case `${fcm_api_url}` / `${del_api_url}`. `test.sh` historically passes camelCase `fcmApiUrl`/`fcmDelUrl`, which `envsubst` silently leaves unsubstituted. The compose harness uses the correct snake_case names — match the template.
- The template defines **two VirtualHosts + three Components**: `VirtualHost "${prosodyDomain}"` (HTTP-async auth), `VirtualHost "anon.${prosodyDomain}"` (anonymous auth), `Component "conference.${prosodyDomain}" "muc"` (MUC, own module list + MAM config), `Component "pubsub.${prosodyDomain}" "pubsub"`, and `Component "broadcast@${prosodyDomain}" "vnc_broadcast"`. Most module/config changes target one specific host/component, not the global list.
- `startup.sh` appends defaults to the rendered config when env vars are unset: `SMACKS_HIBERNATION_TIME` (300s; compose lowers to 10s so SMACKS-expiry tests don't wait 5 min), a random `componentSecret` (port 5347), `DEFAULT_JITSI_CONFERENCE`, `C2S/S2S_STANZA_SIZE_LIMIT` (5 MB), and `PROSODY_LOG_LEVEL`.
- The config sets `run_as_root = true` and `console_interfaces = { "*" }` / `http_interfaces = { "*", "::" }` — required because production and the compose harness run as root and expose the telnet/HTTP ports beyond loopback.

## Runtime quirks

- `startup.sh` (main `CMD`): writes TLS cert/key (from `$prosodySSLcert`/`$prosodySSLkey` env, else a baked-in dev default, overridden by `/etc/tls-update/tls.{crt,key}` if present), renders config, then runs `/usr/local/bin/prosody -F` (also tails `/var/log/prosody/prosody.log`).
- `startup-sidecar.sh` is a separate entrypoint for a static-file/redirect sidecar (`http-server` on :8080). **Not** the XMPP server; not invoked by the main `CMD`.
- `healthcheck.sh`: if `/etc/tls-update/tls.crt` differs from the loaded cert, exits **2** (not 1) to force a container restart so the new cert is picked up. Otherwise checks the admin telnet port 5582 via `check_tcp`.
- The image declares `USER prosody`, but `startup.sh` writes certs/config into root-owned `/etc/prosody`. Production and the compose harness run as root (`user: "0:0"`).

## CI / deploy

- Gitea Actions (`.gitea/workflows/deploy.yml`). `main` builds with kaniko (daemon-less) and pushes to the Gitea registry as `gitea.saas.vnc.biz/vnciac/vnctalk-prosody:sha-<short-sha>` and `:latest`. Auth via `REGISTRY_USER` / `REGISTRY_TOKEN` actions secrets.
- Kubernetes deploy is via the ArgoCD application in the `vnc-iac-env` GitOps repo (`dev/charts/vnctalk-prosody`). The in-repo `helm/prosody/` + `argo/prosody.yaml` are legacy references, not the live deploy path.

## Module conventions

- Lua 5.4. Modules follow Prosody conventions (`module:hook`, `module:open_store`, `module:get_option_string`, `module:provides`).
- Authentication is delegated to an external HTTP endpoint via `mod_auth_http_async` (`hybridaAuthUrl`) on the main VirtualHost; the global default is `internal_hashed`.
- Push notifications: `mod_vnc_fcm` (1:1) / `mod_vnc_muc_fcm` (MUC) over a configurable FCM proxy URL (`fcm_api_url`).
- `mod_http_rest` exposes `/rest` accepting `text/xml` bodies, injected as XMPP stanzas (fires `vnc-rest-message`, consumed by the `mod_carbons`/`mod_mam` patches).
- The `mod_mam` patch always stores (`shall_store → true`) and only archives stanzas with a `<body>`, because users live in the external HTTP auth backend so Prosody's `user_exists()` can't be trusted.