vnctalk-prosody/m1-manual-tasks.md

# M1 Manual Tasks — 0.11.6 → 0.12.6

All repo-level changes for M1 are complete. The tasks below require external
infrastructure (production/staging DB, the external telnet service, a running
dev deployment) and cannot be automated in the compose test harness.

Complete them in order before promoting the M1 image beyond `development-*`.

---

## §1 — Database schema migration (one-way, unrevertible)

The 0.11 → 0.12 Prosody archive schema changed. `prosodyctl mod_storage_sql
upgrade` is a **one-way migration** — after running it, the DB is no longer
compatible with a 0.11 image. Always rehearse on a copy first.

### 1.1 Snapshot the pre-upgrade schema (rollback point)

```bash
# On the production/staging Postgres:
pg_dump -U prosody -d prosody -F c -f /tmp/prosody-0.11-backup.dump
```

Store this dump safely — it is the only rollback path if M1 needs to be
reverted after the schema upgrade.

### 1.2 Rehearse the migration on a scratch copy

```bash
# Restore into a scratch DB
createdb -U postgres prosody_m1_rehearsal
pg_restore -U postgres -d prosody_m1_rehearsal /tmp/prosody-0.11-backup.dump

# Run the migration against the scratch DB using the M1 image.
# The command needs a TTY or piped "y" because it asks for confirmation.
docker run --rm -it --network host \
  -e prosodyDBhost=127.0.0.1 \
  -e prosodyDBname=prosody_m1_rehearsal \
  -e prosodyDBuser=postgres \
  -e prosodyDBpass=<password> \
  -e prosodyDomain=example.com \
  -e hybridaAuthUrl=http://localhost/auth \
  -e fcmApiKey=dummy \
  -e fcm_api_url=http://localhost/fcm \
  -e del_api_url=http://localhost/del \
  -e fileShareBaseUrl=http://localhost/share/ \
  -e fileShareSecret=dummy \
  -e avatarUploadUrl=http://localhost/avatar/ \
  -e avatarUploadUser=avatar \
  -e avatarUploadPass=dummy \
  <m1-image-tag> \
  sh -c 'echo y | prosodyctl mod_storage_sql upgrade'
```

Verify the rehearsal output shows tables altered/created with no errors.
Query the scratch DB to confirm archive data is intact:

```sql
SELECT count(*) FROM prosodyarchive;
```

### 1.3 Run the real migration

Only after the rehearsal succeeds and the external testsuite (§3) is green
against the M1 image on a dev deployment:

1. Take a **fresh** dump immediately before the real migration.
2. Stop the old (0.11) Prosody container.
3. Run `prosodyctl mod_storage_sql upgrade` against the production/staging DB
   (same command as §1.2 but with real DB credentials).
4. Start the M1 image.
5. Verify Prosody starts cleanly (empty error log, healthcheck passes).

### 1.4 Rollback

If the M1 image must be reverted after the schema upgrade:

1. Stop the M1 Prosody container.
2. Restore the pre-upgrade dump from §1.1 (or the fresh dump from §1.3 step 1):
   ```bash
   dropdb -U postgres prosody
   createdb -U postgres prosody
   pg_restore -U postgres -d prosody /tmp/prosody-0.11-backup.dump
   ```
3. Start the previous (0.11) image.

---

## §2 — Telnet console regression test

0.12 reimplemented `mod_admin_telnet` on top of `mod_admin_shell`. Command
syntax and output formatting changed. An external service telnets into port
5582 to run admin commands (upgrade-plan Decision 1); its exact command set
must be exercised against the M1 image before rollout.

### 2.1 Connect to the console

```bash
# Against the compose harness:
telnet localhost 5582

# Against a dev/staging deployment:
telnet <prosody-host> 5582
```

You should see a banner like:
```
Prosody 0.12.6 ~ <build info>
```

### 2.2 Exercise the external service's command set

Run every command the external telnet service uses. Common Prosody console
commands and their 0.12 status:

| Command | 0.11 behavior | 0.12 change to verify |
|---------|--------------|----------------------|
| `c2s:count()` | Returns session count | Verify same output format |
| `s2s:count()` | Returns session count | Verify same output format |
| `hosts` | Lists loaded virtual hosts | Verify output format |
| `module:list("conference.example.com")` | Lists MUC modules | Verify output format |
| `user:list("example.com")` | Lists users | May differ — verify |
| `server:shutdown()` | Shuts down | Verify still works |

**If the external service uses commands not listed above**, run them all
manually and compare output with the 0.11 server.

### 2.3 Verify non-loopback reachability

```bash
# From a different host/container (not 127.0.0.1):
telnet <prosody-container-ip> 5582
```

The connection must succeed — `console_interfaces = { "*" }` in the config
exposes the console on all interfaces (replaces the old `mod_admin_telnet`
patch and `portmanager` patch).

### 2.4 Automated smoke checks

The compose-harness testsuite already covers:
- `test_04_infra.py::TestAdminTelnet::test_telnet_port_open`
- `test_04_infra.py::TestAdminTelnet::test_telnet_banner`
- `test_11_smokes.py::TestAdminTelnetNonLoopback::test_telnet_reachable_on_non_loopback`

These verify the port is open, the banner contains "Prosody", and the console
is reachable on a non-loopback address. They do **not** verify specific
command syntax — that is the manual step above.

---

## §3 — External testsuite run

The compose-harness testsuite (76 passed, 5 skipped) verifies the image
against a mock backend. The **external testsuite** must also be run against a
real dev deployment with the migrated DB and real auth/FCM/file-share backends.

### 3.1 Build and deploy the M1 image

```bash
# Build locally for testing:
docker build -t vnctalk-prosody:m1 .

# Or use CI: merge to main, wait for the development-$SHA image, then deploy
# to the dev environment.
```

### 3.2 Run the compose-harness testsuite

```bash
# Start the compose stack (postgres + mocks + prosody):
make up

# Run the full suite from the host:
XMPP_HOST=localhost XMPP_PORT=5222 \
XMPP_DOMAIN=example.com MUC_DOMAIN=conference.example.com \
XMPP_JID=user1@example.com XMPP_PASSWORD=pass1 \
XMPP_JID2=user2@example.com XMPP_PASSWORD2=pass2 \
REST_URL=http://localhost:5280/rest \
BOSH_URL=http://localhost:5280/http-bind \
WS_URL=ws://localhost:5280/xmpp-websocket \
ADMIN_TELNET_HOST=localhost ADMIN_TELNET_PORT=5582 \
PG_HOST=localhost PG_PORT=5434 PG_DB=prosody \
PG_USER=prosody PG_PASSWORD=prosody \
MOCK_URL=http://localhost:8092 \
pytest tests/ -v -c tests/pytest.ini
```

Or run inside the tester container:
```bash
make test
```

### 3.3 Run the external testsuite against a dev deployment

Set the env vars to point at the dev server (adjust from `run-tests.sh`):

```bash
export XMPP_HOST=<dev-xmpp-host>
export XMPP_PORT=<dev-xmpp-port>
export XMPP_JID=<test-user1>@<domain>
export XMPP_PASSWORD=<password>
export XMPP_JID2=<test-user2>@<domain>
export XMPP_PASSWORD2=<password>
export XMPP_DOMAIN=<domain>
export MUC_DOMAIN=conference.<domain>
export REST_URL=https://<rest-host>/rest
export BOSH_URL=https://<xmpp-host>/http-bind
export WS_URL=wss://<xmpp-host>/xmpp-websocket
export ADMIN_TELNET_HOST=<telnet-host>
export ADMIN_TELNET_PORT=5582
export PG_HOST=<db-host>
export PG_PORT=<db-port>
export PG_DB=prosody
export PG_USER=prosody
export PG_PASSWORD=<db-password>
export MOCK_URL=<mock-url>          # if a mock is deployed for side-effect checks
export REST_USER=<rest-basic-auth-user>    # if /rest is behind HTTP Basic Auth
export REST_PASSWORD=<rest-basic-auth-pass>

pytest tests/ -v -c tests/pytest.ini
```

### 3.4 Key flows to verify

Pay particular attention to the patch-dependent flows:

1. **REST-injected message archiving + carbons** — `POST /rest` a message;
   verify it appears in both sender's and recipient's MAM archives and that
   carbons are delivered (`test_05_patches.py`, `test_09_http_sideeffects.py`).
2. **MUC posting by non-present members** — an owner/admin/member sends a
   groupchat message without joining the room (`test_02_muc.py`).
3. **Offline-member broadcast** — a remote affiliated member receives MUC
   messages without being in the room (requires federation; see
   `tests/MANUAL_TESTS.md` §1.3).
4. **MUC unregister/kick** — `iq-set` with `xmpp:vnctalk:unregister` removes
   affiliation and fires `vnc-muc-kick` (`test_02_muc.py`, `test_06_postgres.py`).
5. **MAM of HTTP-auth-only users** — `shall_store()` returns `true` for users
   that don't exist in the local DB (`test_05_patches.py`).
6. **`muc-config-sub-mitted` event** — room config submission fires the
   hyphenated event consumed by the external service (Decision 2). This is
   not covered by the automated testsuite — verify manually or with the
   external service's own monitoring.

### 3.5 Known test gaps

- `test_03_vnctalk.py::test_vcard_fallback` is **skipped** — `mod_vnc_vcard_fallback`
  exists in `vnctalk/` but is not enabled in the config template. The test was
  also observed to pass on external deployments by coincidence (test users had
  pre-existing vCards). Enable the module in the config first, then remove the
  `@pytest.mark.skip` decorator.
- `test_02_muc.py::test_hidden_lib_rejects_public_override` is skipped in the
  compose harness because it requires an admin JID. Run it manually against a
  deployment with an admin account.