dexcord/docs/test-plans/2026-07-05-api-surface.md
Luna ca5b3cf599
Some checks failed
ci / test (push) Has been cancelled
ci / test (pull_request) Has been cancelled
docs: add test plan for full API surface
Claude-Session: https://claude.ai/code/session_01T4vgXtQLs2wfNwn6B1m4hm
2026-07-05 08:49:41 -03:00

84 lines
5.7 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Human Test Plan — dexcord Full API Surface (2026-07-05-api-surface)
Coverage validation: **PASS** — 44/44 acceptance criteria covered by automated tests
(633 tests, 0 failures, 1 excluded `:flaky` with a deterministic default-suite twin).
This plan covers the residue automation cannot prove: hermeticity, CI enablement,
and migration-guide editorial quality, plus an optional live end-to-end.
## Prerequisites
- Elixir 1.18 / OTP 27, `mix deps.get` clean.
- `mix test` green from the repo root (baseline: 633 tests, 0 failures, 1 excluded).
- A Gitea repo with push access for the CI checks.
- Optional: a real Discord bot token + a throwaway test guild for the live end-to-end.
## Phase A: Suite Hermeticity (HV-1 — AC4.1)
Automated coverage proves the suite is green; this proves green also means *no external network*.
| Step | Action | Expected |
|------|--------|----------|
| A1 | Add `127.0.0.1 discord.com` (and `docs.discord.com`) to `/etc/hosts`, or run `sudo unshare -n mix test` from repo root | Full suite still exits 0 with the same pass count (633/0/1) |
| A2 | `rg -n 'discord\.com|githubusercontent' test/` | Hits only fixtures, doc-link strings, and the deliberately network-only `dexcord.coverage.refresh` task — never a live call from a test |
| A3 | `rg -n 'coverage\.refresh' test/ .gitea/` | No match — the network-only refresh task is never invoked by tests or CI |
| A4 | Revert `/etc/hosts` | Clean state restored |
## Phase B: CI Gate Actually Runs & Bites (HV-2 — AC4.3)
| Step | Action | Expected |
|------|--------|----------|
| B1 | Confirm `.gitea/workflows/ci.yml` lists `- run: mix test` and `- run: mix dexcord.coverage` | Both steps present |
| B2 | In Gitea repo settings, enable **Actions** | Actions enabled (owner infra step; no CI configured today) |
| B3 | Push a normal commit (or open a PR) | Pipeline triggers; both `mix test` and `mix dexcord.coverage` run green |
| B4 | On a throwaway branch, delete one in-scope `endpoint` declaration line (e.g. `get_channel` in `lib/dexcord/api/channels.ex`), push | The `mix dexcord.coverage` step turns the pipeline **red** (non-zero exit, route reported missing) |
| B5 | Delete the throwaway branch | Gate proven to bite; no residue |
## Phase C: Migration Guide Completeness & Prose (HV-3 — AC5.1)
The sample-validity half is automated (`migration_guide_samples_test.exs`); this covers
editorial completeness against the real old app.
| Step | Action | Expected |
|------|--------|----------|
| C1 | Read the old `alamedya-migration.md` (untracked, repo root) and extract its Discord touchpoint checklist | A concrete list, esp. the old string↔int snowflake dance and thread/cache reads specific to alamedya |
| C2 | Open `docs/alamedya-migration-v2.md` and confirm all 8 mandated sections exist | §1 Boot/config, §2 READY backfill, §3 MESSAGE_CREATE, §4 Thread cache reads, §5 Integer snowflakes, §6 Slash commands, §7 REST calls, §8 Hydration |
| C3 | For each old touchpoint from C1, find its before→after mapping in v2 | Nothing dropped; each maps to a v2 section |
| C4 | Confirm the `KEEP IN SYNC` comment sits in both the guide §3 handler and `test/dexcord/migration_guide_samples_test.exs` | Present in both |
| C5 | Read for prose quality | Accurate, adoptable, no stale map-shaped examples |
## End-to-End: Live Bot Round-Trip (optional; spans Phases 2/6/7)
Validates decode → dispatch → cache → send against real Discord, which the fakes only approximate.
1. Configure a real bot token + `allowed_mentions` default per guide §1; boot the app against a test guild.
2. On READY, call `Dexcord.Cache.guilds()` → expect the test guild present; after `GUILD_CREATE`, `Dexcord.Cache.channels(guild_id)` populated with typed channel structs (`%Dexcord.TextChannel{}` etc.).
3. From another account, post `ping!` in a visible channel → the bot replies as a threaded reply (`message_reference` set to your message).
4. `@mention` the bot → it replies.
5. Post as a webhook (or another bot) → the bot sends nothing (webhook/bot authors skipped).
6. `Dexcord.Api.message_history(channel_id) |> Enum.take(5)` in `iex` → exactly 5 `%Dexcord.Message{}` in descending id order, one page fetched.
7. Register a slash command, invoke it → handler receives a typed `%Dexcord.Interaction{}`; `Dexcord.Slash.respond/2` posts a visible reply.
## Human Verification Required
| Criterion | Why Manual | Steps |
|-----------|------------|-------|
| AC4.1 (hermeticity) | "green suite" ≠ "no egress" — a mis-stubbed fake could pass while calling out | Phase A |
| AC4.3 (CI runs) | A checked-in YAML can't assert the server executes it; Gitea Actions enablement is owner infra | Phase B |
| AC5.1 (guide completeness/prose) | Coverage-completeness and readability are editorial judgments | Phase C |
## Traceability
| Acceptance Criterion | Automated Test | Manual Step |
|----------------------|----------------|-------------|
| AC1.11.7 | endpoint_macro_test, endpoint_test, api_facade_test | E2E steps 2, 6 |
| AC1.81.10 | coverage_test, ratelimit_test | Phase B (B4 exercises the gate) |
| AC2.12.9 | struct_test, enum_test, flags_test, endpoint_test | — (pure) |
| AC2.102.17 | model_guild/message/channel/interaction/role_test, events_test | E2E step 2 |
| AC2.182.19 | gateway_integration_test | E2E steps 35 |
| AC2.202.22, AC3.8 | cache_test, cache_fill_test, cache_cascade_test | E2E step 2 |
| AC3.13.6 | send_test, messageable_test, ergonomics_helpers_test, model_message_parts_test | E2E steps 34 |
| AC3.7 | pagination_test | E2E step 6 |
| AC4.1 | full `mix test` | Phase A |
| AC4.2 | struct/enum/flags/endpoint test files (async categorization) | — |
| AC4.3 | coverage_test + `.gitea/workflows/ci.yml` | Phase B |
| AC5.1 | migration_guide_samples_test | Phase C |