vnctalk-prosody/m1-manual-tasks.md

M1 Manual Tasks — 0.11.6 → 0.12.6

All repo-level changes for M1 are complete. The tasks below require external infrastructure (production/staging DB, the external telnet service, a running dev deployment) and cannot be automated in the compose test harness.

Complete them in order before promoting the M1 image beyond development-*.

---

§1 — Database schema migration (one-way, unrevertible)

The 0.11 → 0.12 Prosody archive schema changed. prosodyctl mod_storage_sql upgrade is a one-way migration — after running it, the DB is no longer compatible with a 0.11 image. Always rehearse on a copy first.

1.1 Snapshot the pre-upgrade schema (rollback point)

# On the production/staging Postgres:
pg_dump -U prosody -d prosody -F c -f /tmp/prosody-0.11-backup.dump

Store this dump safely — it is the only rollback path if M1 needs to be reverted after the schema upgrade.

1.2 Rehearse the migration on a scratch copy

# Restore into a scratch DB
createdb -U postgres prosody_m1_rehearsal
pg_restore -U postgres -d prosody_m1_rehearsal /tmp/prosody-0.11-backup.dump

# Run the migration against the scratch DB using the M1 image.
# The command needs a TTY or piped "y" because it asks for confirmation.
docker run --rm -it --network host \
  -e prosodyDBhost=127.0.0.1 \
  -e prosodyDBname=prosody_m1_rehearsal \
  -e prosodyDBuser=postgres \
  -e prosodyDBpass=<password> \
  -e prosodyDomain=example.com \
  -e hybridaAuthUrl=http://localhost/auth \
  -e fcmApiKey=dummy \
  -e fcm_api_url=http://localhost/fcm \
  -e del_api_url=http://localhost/del \
  -e fileShareBaseUrl=http://localhost/share/ \
  -e fileShareSecret=dummy \
  -e avatarUploadUrl=http://localhost/avatar/ \
  -e avatarUploadUser=avatar \
  -e avatarUploadPass=dummy \
  <m1-image-tag> \
  sh -c 'echo y | prosodyctl mod_storage_sql upgrade'

Verify the rehearsal output shows tables altered/created with no errors. Query the scratch DB to confirm archive data is intact:

SELECT count(*) FROM prosodyarchive;

1.3 Run the real migration

Only after the rehearsal succeeds and the external testsuite (§3) is green against the M1 image on a dev deployment:

1. Take a fresh dump immediately before the real migration.
2. Stop the old (0.11) Prosody container.
3. Run prosodyctl mod_storage_sql upgrade against the production/staging DB (same command as §1.2 but with real DB credentials).
4. Start the M1 image.
5. Verify Prosody starts cleanly (empty error log, healthcheck passes).

1.4 Rollback

If the M1 image must be reverted after the schema upgrade:

1. Stop the M1 Prosody container.
2. Restore the pre-upgrade dump from §1.1 (or the fresh dump from §1.3 step 1):

   dropdb -U postgres prosody
   createdb -U postgres prosody
   pg_restore -U postgres -d prosody /tmp/prosody-0.11-backup.dump
   

3. Start the previous (0.11) image.

---

§2 — Telnet console regression test

0.12 reimplemented mod_admin_telnet on top of mod_admin_shell. Command syntax and output formatting changed. An external service telnets into port 5582 to run admin commands (upgrade-plan Decision 1); its exact command set must be exercised against the M1 image before rollout.

2.1 Connect to the console

# Against the compose harness:
telnet localhost 5582

# Against a dev/staging deployment:
telnet <prosody-host> 5582

You should see a banner like:

Prosody 0.12.6 ~ <build info>

2.2 Exercise the external service's command set

Run every command the external telnet service uses. Common Prosody console commands and their 0.12 status:

Command	0.11 behavior	0.12 change to verify
c2s:count()	Returns session count	Verify same output format
s2s:count()	Returns session count	Verify same output format
hosts	Lists loaded virtual hosts	Verify output format
module:list("conference.example.com")	Lists MUC modules	Verify output format
user:list("example.com")	Lists users	May differ — verify
server:shutdown()	Shuts down	Verify still works

If the external service uses commands not listed above, run them all manually and compare output with the 0.11 server.

2.3 Verify non-loopback reachability

# From a different host/container (not 127.0.0.1):
telnet <prosody-container-ip> 5582

The connection must succeed — console_interfaces = { "*" } in the config exposes the console on all interfaces (replaces the old mod_admin_telnet patch and portmanager patch).

2.4 Automated smoke checks

The compose-harness testsuite already covers:

• test_04_infra.py::TestAdminTelnet::test_telnet_port_open
• test_04_infra.py::TestAdminTelnet::test_telnet_banner
• test_11_smokes.py::TestAdminTelnetNonLoopback::test_telnet_reachable_on_non_loopback

These verify the port is open, the banner contains "Prosody", and the console is reachable on a non-loopback address. They do not verify specific command syntax — that is the manual step above.

---

§3 — External testsuite run

The compose-harness testsuite (76 passed, 5 skipped) verifies the image against a mock backend. The external testsuite must also be run against a real dev deployment with the migrated DB and real auth/FCM/file-share backends.

3.1 Build and deploy the M1 image

# Build locally for testing:
docker build -t vnctalk-prosody:m1 .

# Or use CI: merge to main, wait for the development-$SHA image, then deploy
# to the dev environment.

3.2 Run the compose-harness testsuite

# Start the compose stack (postgres + mocks + prosody):
make up

# Run the full suite from the host:
XMPP_HOST=localhost XMPP_PORT=5222 \
XMPP_DOMAIN=example.com MUC_DOMAIN=conference.example.com \
XMPP_JID=user1@example.com XMPP_PASSWORD=pass1 \
XMPP_JID2=user2@example.com XMPP_PASSWORD2=pass2 \
REST_URL=http://localhost:5280/rest \
BOSH_URL=http://localhost:5280/http-bind \
WS_URL=ws://localhost:5280/xmpp-websocket \
ADMIN_TELNET_HOST=localhost ADMIN_TELNET_PORT=5582 \
PG_HOST=localhost PG_PORT=5434 PG_DB=prosody \
PG_USER=prosody PG_PASSWORD=prosody \
MOCK_URL=http://localhost:8092 \
pytest tests/ -v -c tests/pytest.ini

Or run inside the tester container:

make test

3.3 Run the external testsuite against a dev deployment

Set the env vars to point at the dev server (adjust from run-tests.sh):

export XMPP_HOST=<dev-xmpp-host>
export XMPP_PORT=<dev-xmpp-port>
export XMPP_JID=<test-user1>@<domain>
export XMPP_PASSWORD=<password>
export XMPP_JID2=<test-user2>@<domain>
export XMPP_PASSWORD2=<password>
export XMPP_DOMAIN=<domain>
export MUC_DOMAIN=conference.<domain>
export REST_URL=https://<rest-host>/rest
export BOSH_URL=https://<xmpp-host>/http-bind
export WS_URL=wss://<xmpp-host>/xmpp-websocket
export ADMIN_TELNET_HOST=<telnet-host>
export ADMIN_TELNET_PORT=5582
export PG_HOST=<db-host>
export PG_PORT=<db-port>
export PG_DB=prosody
export PG_USER=prosody
export PG_PASSWORD=<db-password>
export MOCK_URL=<mock-url>          # if a mock is deployed for side-effect checks
export REST_USER=<rest-basic-auth-user>    # if /rest is behind HTTP Basic Auth
export REST_PASSWORD=<rest-basic-auth-pass>

pytest tests/ -v -c tests/pytest.ini

3.4 Key flows to verify

Pay particular attention to the patch-dependent flows:

1. REST-injected message archiving + carbons — POST /rest a message; verify it appears in both sender's and recipient's MAM archives and that carbons are delivered (test_05_patches.py, test_09_http_sideeffects.py).
2. MUC posting by non-present members — an owner/admin/member sends a groupchat message without joining the room (test_02_muc.py).
3. Offline-member broadcast — a remote affiliated member receives MUC messages without being in the room (requires federation; see tests/MANUAL_TESTS.md §1.3).
4. MUC unregister/kick — iq-set with xmpp:vnctalk:unregister removes affiliation and fires vnc-muc-kick (test_02_muc.py, test_06_postgres.py).
5. MAM of HTTP-auth-only users — shall_store() returns true for users that don't exist in the local DB (test_05_patches.py).
6. muc-config-sub-mitted event — room config submission fires the hyphenated event consumed by the external service (Decision 2). This is not covered by the automated testsuite — verify manually or with the external service's own monitoring.

3.5 Known test gaps

• test_03_vnctalk.py::test_vcard_fallback is skipped — mod_vnc_vcard_fallback exists in vnctalk/ but is not enabled in the config template. The test was also observed to pass on external deployments by coincidence (test users had pre-existing vCards). Enable the module in the config first, then remove the @pytest.mark.skip decorator.
• test_02_muc.py::test_hidden_lib_rejects_public_override is skipped in the compose harness because it requires an admin JID. Run it manually against a deployment with an admin account.