api-surface #1
1 changed files with 84 additions and 0 deletions
docs: add test plan for full API surface
Claude-Session: https://claude.ai/code/session_01T4vgXtQLs2wfNwn6B1m4hm
commit
ca5b3cf599
84
docs/test-plans/2026-07-05-api-surface.md
Normal file
84
docs/test-plans/2026-07-05-api-surface.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
# Human Test Plan — dexcord Full API Surface (2026-07-05-api-surface)
|
||||
|
||||
Coverage validation: **PASS** — 44/44 acceptance criteria covered by automated tests
|
||||
(633 tests, 0 failures, 1 excluded `:flaky` with a deterministic default-suite twin).
|
||||
This plan covers the residue automation cannot prove: hermeticity, CI enablement,
|
||||
and migration-guide editorial quality, plus an optional live end-to-end.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Elixir 1.18 / OTP 27, `mix deps.get` clean.
|
||||
- `mix test` green from the repo root (baseline: 633 tests, 0 failures, 1 excluded).
|
||||
- A Gitea repo with push access for the CI checks.
|
||||
- Optional: a real Discord bot token + a throwaway test guild for the live end-to-end.
|
||||
|
||||
## Phase A: Suite Hermeticity (HV-1 — AC4.1)
|
||||
|
||||
Automated coverage proves the suite is green; this proves green also means *no external network*.
|
||||
|
||||
| Step | Action | Expected |
|
||||
|------|--------|----------|
|
||||
| A1 | Add `127.0.0.1 discord.com` (and `docs.discord.com`) to `/etc/hosts`, or run `sudo unshare -n mix test` from repo root | Full suite still exits 0 with the same pass count (633/0/1) |
|
||||
| A2 | `rg -n 'discord\.com|githubusercontent' test/` | Hits only fixtures, doc-link strings, and the deliberately network-only `dexcord.coverage.refresh` task — never a live call from a test |
|
||||
| A3 | `rg -n 'coverage\.refresh' test/ .gitea/` | No match — the network-only refresh task is never invoked by tests or CI |
|
||||
| A4 | Revert `/etc/hosts` | Clean state restored |
|
||||
|
||||
## Phase B: CI Gate Actually Runs & Bites (HV-2 — AC4.3)
|
||||
|
||||
| Step | Action | Expected |
|
||||
|------|--------|----------|
|
||||
| B1 | Confirm `.gitea/workflows/ci.yml` lists `- run: mix test` and `- run: mix dexcord.coverage` | Both steps present |
|
||||
| B2 | In Gitea repo settings, enable **Actions** | Actions enabled (owner infra step; no CI configured today) |
|
||||
| B3 | Push a normal commit (or open a PR) | Pipeline triggers; both `mix test` and `mix dexcord.coverage` run green |
|
||||
| B4 | On a throwaway branch, delete one in-scope `endpoint` declaration line (e.g. `get_channel` in `lib/dexcord/api/channels.ex`), push | The `mix dexcord.coverage` step turns the pipeline **red** (non-zero exit, route reported missing) |
|
||||
| B5 | Delete the throwaway branch | Gate proven to bite; no residue |
|
||||
|
||||
## Phase C: Migration Guide Completeness & Prose (HV-3 — AC5.1)
|
||||
|
||||
The sample-validity half is automated (`migration_guide_samples_test.exs`); this covers
|
||||
editorial completeness against the real old app.
|
||||
|
||||
| Step | Action | Expected |
|
||||
|------|--------|----------|
|
||||
| C1 | Read the old `alamedya-migration.md` (untracked, repo root) and extract its Discord touchpoint checklist | A concrete list, esp. the old string↔int snowflake dance and thread/cache reads specific to alamedya |
|
||||
| C2 | Open `docs/alamedya-migration-v2.md` and confirm all 8 mandated sections exist | §1 Boot/config, §2 READY backfill, §3 MESSAGE_CREATE, §4 Thread cache reads, §5 Integer snowflakes, §6 Slash commands, §7 REST calls, §8 Hydration |
|
||||
| C3 | For each old touchpoint from C1, find its before→after mapping in v2 | Nothing dropped; each maps to a v2 section |
|
||||
| C4 | Confirm the `KEEP IN SYNC` comment sits in both the guide §3 handler and `test/dexcord/migration_guide_samples_test.exs` | Present in both |
|
||||
| C5 | Read for prose quality | Accurate, adoptable, no stale map-shaped examples |
|
||||
|
||||
## End-to-End: Live Bot Round-Trip (optional; spans Phases 2/6/7)
|
||||
|
||||
Validates decode → dispatch → cache → send against real Discord, which the fakes only approximate.
|
||||
|
||||
1. Configure a real bot token + `allowed_mentions` default per guide §1; boot the app against a test guild.
|
||||
2. On READY, call `Dexcord.Cache.guilds()` → expect the test guild present; after `GUILD_CREATE`, `Dexcord.Cache.channels(guild_id)` populated with typed channel structs (`%Dexcord.TextChannel{}` etc.).
|
||||
3. From another account, post `ping!` in a visible channel → the bot replies as a threaded reply (`message_reference` set to your message).
|
||||
4. `@mention` the bot → it replies.
|
||||
5. Post as a webhook (or another bot) → the bot sends nothing (webhook/bot authors skipped).
|
||||
6. `Dexcord.Api.message_history(channel_id) |> Enum.take(5)` in `iex` → exactly 5 `%Dexcord.Message{}` in descending id order, one page fetched.
|
||||
7. Register a slash command, invoke it → handler receives a typed `%Dexcord.Interaction{}`; `Dexcord.Slash.respond/2` posts a visible reply.
|
||||
|
||||
## Human Verification Required
|
||||
|
||||
| Criterion | Why Manual | Steps |
|
||||
|-----------|------------|-------|
|
||||
| AC4.1 (hermeticity) | "green suite" ≠ "no egress" — a mis-stubbed fake could pass while calling out | Phase A |
|
||||
| AC4.3 (CI runs) | A checked-in YAML can't assert the server executes it; Gitea Actions enablement is owner infra | Phase B |
|
||||
| AC5.1 (guide completeness/prose) | Coverage-completeness and readability are editorial judgments | Phase C |
|
||||
|
||||
## Traceability
|
||||
|
||||
| Acceptance Criterion | Automated Test | Manual Step |
|
||||
|----------------------|----------------|-------------|
|
||||
| AC1.1–1.7 | endpoint_macro_test, endpoint_test, api_facade_test | E2E steps 2, 6 |
|
||||
| AC1.8–1.10 | coverage_test, ratelimit_test | Phase B (B4 exercises the gate) |
|
||||
| AC2.1–2.9 | struct_test, enum_test, flags_test, endpoint_test | — (pure) |
|
||||
| AC2.10–2.17 | model_guild/message/channel/interaction/role_test, events_test | E2E step 2 |
|
||||
| AC2.18–2.19 | gateway_integration_test | E2E steps 3–5 |
|
||||
| AC2.20–2.22, AC3.8 | cache_test, cache_fill_test, cache_cascade_test | E2E step 2 |
|
||||
| AC3.1–3.6 | send_test, messageable_test, ergonomics_helpers_test, model_message_parts_test | E2E steps 3–4 |
|
||||
| AC3.7 | pagination_test | E2E step 6 |
|
||||
| AC4.1 | full `mix test` | Phase A |
|
||||
| AC4.2 | struct/enum/flags/endpoint test files (async categorization) | — |
|
||||
| AC4.3 | coverage_test + `.gitea/workflows/ci.yml` | Phase B |
|
||||
| AC5.1 | migration_guide_samples_test | Phase C |
|
||||
Loading…
Add table
Add a link
Reference in a new issue