api-surface #1
1 changed files with 388 additions and 0 deletions
docs: add full API surface design plan (api-surface)
Completed brainstorming session. Design includes: - Three compile-time DSLs (Struct/Enum/Flags) generating decode/encode/merge from single declarations, string-key matching with zero wire atomization - Integer Snowflake rework, distinct channel structs + factory, first-class partials with Cache.fill hydration - Endpoint DSL covering ~180 v10 REST endpoints + mix dexcord.coverage spec-diff gate - 7 implementation phases
commit
3d5517d39e
388
docs/design-plans/2026-07-04-api-surface.md
Normal file
388
docs/design-plans/2026-07-04-api-surface.md
Normal file
|
|
@ -0,0 +1,388 @@
|
|||
# Dexcord Full API Surface Design
|
||||
|
||||
## Summary
|
||||
|
||||
Dexcord's current API surface is a thin, map-based wrapper: REST calls return raw string-keyed maps, gateway events arrive as maps, and callers do their own field access and type coercion. This design replaces that contract end-to-end with generated structs. Three compile-time DSLs (`Dexcord.Struct` for object shapes, `Dexcord.Enum` for open integer enums, `Dexcord.Flags` for bitfields) let each of the ~60 Discord object types and ~180 REST endpoints be declared once — as a field list or a route line — and have the struct definition, type spec, decoder, encoder, and (for endpoints) the callable function itself generated from that single declaration. This keeps the "one source of truth per shape" property that hand-written triple declarations (struct + type + decoder, written separately) tend to lose over time, which is the specific failure mode being designed away from Nostrum's approach.
|
||||
|
||||
The rest of the design follows from making that decode step happen exactly once, in one place: gateway JSON is turned into typed structs in the dispatcher before it ever reaches a handler or the cache, and REST responses are decoded the same way before being returned to the caller. Decoding is built to degrade rather than crash — unrecognized fields, enum values, and events fall back to sensible defaults or raw passthrough instead of raising, so a single malformed payload can't take down the dispatch loop. On top of the typed core sits a discord.py-flavored ergonomics layer (protocol-based message sending, reply helpers, mention rendering, an embed builder) intended to make the typed API pleasant to use rather than just type-safe. Work is sequenced across seven phases — DSLs first, then models, then events, then the REST layer and its coverage tooling, then wiring it all into the live gateway/cache/slash-command paths, finishing with the ergonomics layer and a migration guide.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
1. **Typed REST surface**: `Dexcord.Api` covers all mainstream v10 bot endpoint groups (channels, messages, guilds, members, roles, reactions, threads, webhooks, invites, interactions, app commands, emojis, stickers, polls, auto-mod, scheduled events, audit logs) via a minimal-boilerplate endpoint-definition layer (macros or similar). Monetization, soundboard, stage instances, and templates stay on `request/4`, which remains public as the escape hatch.
|
||||
2. **Deep structs**: real structs with `Dexcord.Snowflake` IDs everywhere — API returns, gateway event payloads handed to `handle_event`, and cache reads. No back-compat with the map contract.
|
||||
3. **discord.py-flavored ergonomics**: struct-centric helper functions on model modules (`Dexcord.Message.reply/2`-style), accepting structs or snowflakes where sensible.
|
||||
4. **Tests**: machinery tested hard (endpoint macro layer, decode engine, Snowflake, partial/null/unknown-field edge cases) + representative per-group fake-server round-trips; suite stays green and hermetic.
|
||||
5. **New alamedya migration guide**: a fresh guide (not an update of the existing internal `alamedya-migration.md`, which remains uncommitted) written against the new struct contract.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### api-surface.AC1: Typed REST surface
|
||||
- **api-surface.AC1.1 Success:** A DSL-declared endpoint generates a function that issues the correct method + interpolated path (FakeRest wire assertion) and returns `{:ok, decoded_struct}`
|
||||
- **api-surface.AC1.2 Success:** `query:` params are URL-encoded when provided, omitted entirely when not
|
||||
- **api-surface.AC1.3 Success:** `reason: true` endpoints send `X-Audit-Log-Reason` (URL-encoded) when `:reason` is passed, and no header otherwise
|
||||
- **api-surface.AC1.4 Success:** `files: true` endpoints encode multipart with `payload_json` + `files[n]` parts referenced by attachment `id`
|
||||
- **api-surface.AC1.5 Success:** Snowflake arguments accept integer, decimal string, or struct-with-`.id`, all producing the identical request path
|
||||
- **api-surface.AC1.6 Failure:** A 4xx/5xx response decodes to `{:error, %Dexcord.Api.Error{}}` with status/code/message intact
|
||||
- **api-surface.AC1.7 Success:** `Dexcord.Api.request/4` remains public with its current raw-map contract
|
||||
- **api-surface.AC1.8 Success:** `mix dexcord.coverage` reports zero missing in-scope endpoints against the pinned spec
|
||||
- **api-surface.AC1.9 Failure:** Removing a route-table entry makes `mix dexcord.coverage` exit non-zero
|
||||
- **api-surface.AC1.10 Success:** New route shapes get bounded bucket keys (invite codes collapse; no raw tokens in ETS)
|
||||
|
||||
### api-surface.AC2: Deep structs & decode engine
|
||||
- **api-surface.AC2.1 Success:** `from_map/1` decodes declared fields including nested structs and lists
|
||||
- **api-surface.AC2.2 Success:** Unknown JSON keys are ignored; decoding mints zero atoms from wire data
|
||||
- **api-surface.AC2.3 Failure:** Wrong-shaped field values (list where map expected, junk scalars) decode to `nil`/default/raw-value — never raise
|
||||
- **api-surface.AC2.4 Success:** Unknown enum int → `{:unknown, n}`; `encode(decode(n))` is lossless for known and unknown values
|
||||
- **api-surface.AC2.5 Success:** Flags fields keep the raw integer; `has?/2`, `to_list/1`, `from_list/1` correct; unknown bits survive round-trip in the int
|
||||
- **api-surface.AC2.6 Success:** `tristate:` fields decode to `:absent | nil | value`
|
||||
- **api-surface.AC2.7 Success:** `:presence` fields decode key-presence to boolean (present-with-null → `true`)
|
||||
- **api-surface.AC2.8 Success:** `merge_map/2` updates only keys present in the partial map
|
||||
- **api-surface.AC2.9 Success:** `to_map/1`/body encoding: snowflakes → strings, enums → ints, `DateTime` → ISO8601; absent keyword key = omitted field, `nil` = JSON null
|
||||
- **api-surface.AC2.10 Success:** Golden GUILD_CREATE decodes with typed channels/roles/members/threads
|
||||
- **api-surface.AC2.11 Success:** MESSAGE_CREATE's member-without-user → `%PartialMember{}`, author a full `%User{}`
|
||||
- **api-surface.AC2.12 Success:** Webhook-authored message decodes (synthetic author, no member)
|
||||
- **api-surface.AC2.13 Success:** `referenced_message`: absent → `:absent`, null → `nil`, present → `%Message{}`
|
||||
- **api-surface.AC2.14 Success:** Channel factory maps all 15 documented types to their structs; unknown type → `%UnknownChannel{raw: map}`
|
||||
- **api-surface.AC2.15 Success:** INTERACTION_CREATE decodes in both guild (member + permissions) and DM (user) variants
|
||||
- **api-surface.AC2.16 Success:** Resolved interaction data decodes to partial structs (member w/o user, 4-field channel)
|
||||
- **api-surface.AC2.17 Success:** Every atom in `EventNames.all/0` has decode coverage — full object, envelope struct, or documented passthrough (test iterates the list)
|
||||
- **api-surface.AC2.18 Success:** Handler receives `{atom, typed_payload}`; unknown events arrive `{{:unknown_event, name}, raw_map}`
|
||||
- **api-surface.AC2.19 Failure:** A malformed event payload logs, falls back to raw delivery, and dispatch of subsequent events continues
|
||||
- **api-surface.AC2.20 Success:** Cache stores and returns structs keyed by integer snowflakes
|
||||
- **api-surface.AC2.21 Success:** GUILD_UPDATE merge preserves gateway-only fields (`joined_at` survives)
|
||||
- **api-surface.AC2.22 Success:** GUILD_DELETE cascade purge works against struct-valued tables
|
||||
|
||||
### api-surface.AC3: Ergonomics
|
||||
- **api-surface.AC3.1 Success:** `Api.send/2` accepts channel structs, threads, and bare snowflakes via `Messageable`, hitting the right channel route
|
||||
- **api-surface.AC3.2 Success:** `Api.send/2` to `%User{}`/`%Member{}` creates the DM once, then reuses the cached DM channel (second send makes no create-DM call)
|
||||
- **api-surface.AC3.3 Success:** `Message.reply/2` sets `message_reference` to the source message; `mention_author:` flows into allowed_mentions
|
||||
- **api-surface.AC3.4 Success:** `mention/1` + `String.Chars` render `<@id>`/`<#id>`/`<@&id>`; `PartialEmoji.to_string/1` renders send format
|
||||
- **api-surface.AC3.5 Success:** Config-level `allowed_mentions` default applies to sends; per-send values merge field-wise over it
|
||||
- **api-surface.AC3.6 Success:** `Embed` builder chain produces a valid wire map
|
||||
- **api-surface.AC3.7 Success:** Pagination streams lazily page across multiple FakeRest hits
|
||||
- **api-surface.AC3.8 Success:** `Cache.fill/1` hydrates declared slots from cache; cache miss leaves slot `nil`; never issues HTTP
|
||||
|
||||
### api-surface.AC4: Suite health
|
||||
- **api-surface.AC4.1 Success:** Full suite green and hermetic (no external network) throughout
|
||||
- **api-surface.AC4.2 Success:** DSL machinery (Struct/Enum/Flags/endpoint) each has a dedicated exhaustive test file
|
||||
- **api-surface.AC4.3 Success:** `mix dexcord.coverage` runs in CI and gates regressions
|
||||
|
||||
### api-surface.AC5: Migration guide
|
||||
- **api-surface.AC5.1 Success:** `docs/alamedya-migration-v2.md` exists, covers every Discord touchpoint in alamedya (READY backfill, MESSAGE_CREATE flow, thread cache reads, integer snowflakes vs the old string dance), and its code samples are valid against the final API
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Discord Gateway**: Discord's persistent WebSocket connection that pushes real-time events (message created, member joined, etc.) to a bot, as opposed to the REST API which is request/response.
|
||||
- **Snowflake**: Discord's ID format — a 64-bit integer that encodes a creation timestamp, making IDs sortable by time. This design narrows Dexcord's representation from "integer or string" to integer-only.
|
||||
- **Struct** (Elixir): A compile-time-checked, fixed-field data record built on top of Elixir maps. `Dexcord.Struct` here is a DSL that generates these (plus their type spec and (de)serialization functions) from one field declaration.
|
||||
- **DSL (domain-specific language)**: A small, purpose-built syntax (implemented via Elixir macros) for declaring something — here, object shapes (`Dexcord.Struct`), enums (`Dexcord.Enum`), bitfields (`Dexcord.Flags`), and REST routes (the endpoint DSL) — that expands into ordinary code at compile time.
|
||||
- **Atom / atomization**: Elixir atoms are fixed constants stored in a global, never-garbage-collected table; converting untrusted external strings into atoms can exhaust that table and crash the VM. "Zero atoms minted from wire data" means decoding matches on string keys directly rather than converting them to atoms.
|
||||
- **Open integer enum**: An enum that must tolerate values Discord adds later. `Dexcord.Enum` decodes recognized ints to atoms and unrecognized ones to `{:unknown, n}`, and can re-encode either losslessly.
|
||||
- **Bitfield / Flags**: An integer where each bit is an independent boolean (e.g., Discord permissions). `Dexcord.Flags` keeps the raw integer (so unknown bits aren't lost) while offering helpers to test/list/build named flags.
|
||||
- **Tristate field**: A field with three meaningfully different states — absent from the payload, explicitly `null`, or a real value — rather than the usual two (present or `nil`). Used where "not present" and "present but null" mean different things (e.g., a message that isn't a reply vs. a reply whose original was deleted).
|
||||
- **Presence field**: A field where only *whether the key exists* in the JSON matters, not its value — key present (even if null) decodes to `true`.
|
||||
- **Hydration slot / `hydrate`**: A declared field on a gateway "envelope" struct (e.g., just a `user_id`) that can be optionally filled in later from the cache (e.g., the full `User`) via `Cache.fill/1`, without ever making a network call.
|
||||
- **Envelope struct**: A per-event struct used for gateway events that don't carry a full Discord object (e.g., `MESSAGE_DELETE` only carries IDs), as opposed to "full-object" events like `MESSAGE_CREATE` that decode directly to the model struct.
|
||||
- **Partial struct**: A distinct struct for a shape Discord documents as genuinely different from (usually a subset of) the full object — e.g., a guild member payload that lacks the nested user — used only where the shape truly differs, not merely where a field is sometimes missing.
|
||||
- **Messageable (protocol)**: An Elixir protocol implemented by every "thing you can send a message to" (channels, threads, users, messages, raw IDs) so `Api.send/2` can resolve any of them to the right destination and route.
|
||||
- **discord.py**: A popular Python Discord bot library whose API taste (naming, `get`/`fetch` conventions, ergonomics) this design explicitly imitates.
|
||||
- **Nostrum**: An existing Elixir Discord library whose design choices (integer snowflakes, lenient casting, typed lists) are partly kept and partly deliberately rejected (its atom-minting and field-drift problems).
|
||||
- **ETS (Erlang Term Storage)**: The in-memory key-value store the cache is built on; tables here are `:public` with `read_concurrency: true` so many processes can read concurrently while one process writes.
|
||||
- **Single-writer cache**: An architecture where only the dispatcher process ever writes to the cache (in gateway event order), while any process can read — avoiding write races without needing locks.
|
||||
- **Dispatcher**: The process that receives decoded gateway events, writes them to cache, and hands them off to handler code; the design's decode step happens here, once, before anything else sees the event.
|
||||
- **Ratelimit bucket / `route_key`**: Discord rate-limits are tracked per "bucket," derived from a request's route with variable segments (IDs, tokens) collapsed so buckets don't multiply unboundedly or leak sensitive values (like invite codes) into ETS keys.
|
||||
- **Multipart / `payload_json`**: The HTTP encoding Discord requires for endpoints that accept file uploads alongside JSON — the JSON body goes in a `payload_json` part, files in separate numbered parts.
|
||||
- **`X-Audit-Log-Reason` header**: An HTTP header Discord uses to attach a human-readable reason to moderation actions in the guild's audit log.
|
||||
- **OpenAPI spec (`discord-api-spec`)**: Discord's official, auto-generated machine-readable description of its REST API (published as a "public preview"), used here only as a checklist to verify endpoint coverage, not as codegen input.
|
||||
- **`mix` task**: A command-line task in Elixir's build tool (`mix`); `mix dexcord.coverage` is a custom one that checks the declared routes against the pinned spec.
|
||||
- **Golden fixture**: A checked-in, real (or realistic) sample payload used as a stable, exact-match test input — e.g., a captured `GUILD_CREATE` payload used to verify decoding.
|
||||
- **FakeRest / FakeGateway**: This codebase's existing test doubles that simulate Discord's REST API and gateway connection so tests run hermetically, without real network calls.
|
||||
- **`persistent_term`**: An Erlang/OTP storage mechanism for rarely-changing global data with very fast reads, used here to hold validated bot configuration.
|
||||
- **Behaviour + `@before_compile`**: An Elixir pattern where a module declares a contract (`behaviour`) that implementers must satisfy, combined with a compile-time hook (`@before_compile`) that injects fallback/catch-all code into the implementing module.
|
||||
- **Keyword list**: An Elixir list of `{atom, value}` pairs (written `key: value, ...`), Elixir's idiomatic way of passing optional named arguments — used here for endpoint bodies, where an omitted key means "don't send this field" versus an explicit `nil`.
|
||||
- **Components V2**: A newer generation of Discord's interactive message-component system (buttons, selects, etc.) alongside the original component set.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Overview
|
||||
|
||||
Three compile-time DSLs generate everything from single declarations; a decode pipeline converts raw gateway/REST JSON (string-keyed maps from the built-in `JSON` module) into structs exactly once per event; the REST surface is declared as a route table that expands into typed functions. Discord's official OpenAPI spec (`discord/discord-api-spec`) is **not** codegen input — it is a coverage checklist consumed by a dev-only mix task.
|
||||
|
||||
Prior art: discord.py sets the taste (model taxonomy, tolerance posture, get/fetch semantics, Messageable); Nostrum's casting vocabulary is kept (`{:struct, M}`/`{:list, T}` type specs, nil passthrough, lenient scalar fallback, integer snowflakes) while its regretted mechanics are dropped (global payload atomization, triple-declared fields, raw-int enums).
|
||||
|
||||
### The three DSLs
|
||||
|
||||
**`Dexcord.Struct`** (`lib/dexcord/struct.ex`) — a `field` declaration DSL. One declaration per field generates, at compile time:
|
||||
|
||||
- `defstruct` with defaults
|
||||
- `@type t` (generated from the same field table — cannot drift)
|
||||
- `from_map/1` — decodes a string-keyed map; **matches string keys directly** (no atomization pass, zero atoms minted from wire data)
|
||||
- `to_map/1` — encodes back to a JSON-ready map (snowflakes → strings, enums → ints, flags → int, `DateTime` → ISO8601)
|
||||
- `merge_map/2` — `(struct, raw_partial_map) → struct`, updating **only keys present in the map** (correct partial-update semantics for the cache)
|
||||
- `fill/2` hydration support for declared `hydrate` slots (see Cache integration)
|
||||
|
||||
Field types: `:snowflake`, `:string`, `:integer`, `:boolean`, `:datetime` (ISO8601 → `DateTime.t()`), `{:struct, Module}`, `{:list, type}`, `{:enum, Module}`, `{:flags, Module}`, plus modifiers `default:`, `tristate: true` (decodes to `:absent | nil | value`; for `Message.referenced_message` absent = not a reply, null = deleted referent), and `:presence` (key-presence boolean; for `RoleTags.premium_subscriber` where present-null means true).
|
||||
|
||||
```elixir
|
||||
defmodule Dexcord.Message do
|
||||
use Dexcord.Struct
|
||||
|
||||
discord_struct do
|
||||
field :id, :snowflake
|
||||
field :channel_id, :snowflake
|
||||
field :author, {:struct, Dexcord.User}
|
||||
field :content, :string
|
||||
field :timestamp, :datetime
|
||||
field :embeds, {:list, {:struct, Dexcord.Embed}}, default: []
|
||||
field :type, {:enum, Dexcord.MessageType}
|
||||
field :flags, {:flags, Dexcord.MessageFlags}
|
||||
field :referenced_message, {:struct, __MODULE__}, tristate: true
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**`Dexcord.Enum`** (`lib/dexcord/enum.ex`) — open integer enums. One table generates `@type t :: :atom | ... | {:unknown, integer()}`, `decode/1` (unknown int → `{:unknown, n}`), `encode/1` (lossless round-trip, `{:unknown, n}` → `n`).
|
||||
|
||||
**`Dexcord.Flags`** (`lib/dexcord/flags.ex`) — bitfields. Struct fields keep the **raw integer** (lossless, forward-compatible); the module generates `has?/2`, `to_list/1` (unknown bits omitted from the list only, never from the int), `from_list/1`, `all/0`. Used for `Dexcord.Permissions`, `Dexcord.MessageFlags`, and friends; `Dexcord.Intents` remains as-is.
|
||||
|
||||
### Decode semantics (the tolerance contract)
|
||||
|
||||
Decode must never crash the dispatcher:
|
||||
|
||||
- Generated `from_map/1` guards shapes (`when is_map/is_list`); surprises fall through to `nil`/default; failed scalar casts keep the raw value.
|
||||
- Unknown JSON keys are never inspected — forward compatible by construction.
|
||||
- Unknown enum int → `{:unknown, n}`. Unknown channel `type` → `%Dexcord.UnknownChannel{type: {:unknown, n}, raw: map}`. Unknown gateway event → `{{:unknown_event, name}, raw_map}` (unchanged from today).
|
||||
- Absent and JSON-null both decode to `nil`, except fields explicitly declared `tristate:` or `:presence`.
|
||||
- Encode direction: request bodies come from keyword opts; **absent key = omit field, `nil` value = JSON null** — the PATCH tri-state maps onto keyword-list key presence with no sentinel.
|
||||
|
||||
### Snowflake (breaking rework)
|
||||
|
||||
`Dexcord.Snowflake.t()` becomes `0..0xFFFF_FFFF_FFFF_FFFF` — a plain integer (Nostrum-proven, discord.py-identical). Module: `cast/1`, `cast!/1`, `dump/1` (→ string for wire), `defguard is_snowflake/1`, existing `from_datetime/1`, `to_datetime/1`, `to_unix/1` (the missing `:error` path in `to_unix/1` gets fixed). Wire strings normalize to int once, at decode. Ids sort chronologically, interpolate, and pattern-match naturally.
|
||||
|
||||
### Models
|
||||
|
||||
~60 model modules under `lib/dexcord/model/`, **flat module names** (`Dexcord.Message`, `Dexcord.User`, `Dexcord.Guild`, `Dexcord.TextChannel` — directory ≠ namespace, deliberately).
|
||||
|
||||
**Channels are distinct structs** chosen by a factory (`lib/dexcord/model/channel.ex`, `Dexcord.Channel.from_map/1` dispatching on `"type"`): `TextChannel`, `VoiceChannel`, `CategoryChannel`, `AnnouncementChannel`, `ForumChannel`, `MediaChannel`, `StageChannel`, `DMChannel`, `GroupDMChannel`, `Thread` (types 10/11/12), `UnknownChannel` (fallback). Shared fields come from a common declaration helper; capability is protocol membership (`CategoryChannel` is not `Messageable` — wrong sends fail at match time).
|
||||
|
||||
**Partials are first-class** where Discord documents a genuinely distinct shape (the rule: different provenance → different struct; same shape with an occasionally-missing field → `nil`):
|
||||
|
||||
- `Dexcord.PartialEmoji` (reactions/components/poll media; `to_string/1` gives send format)
|
||||
- `Dexcord.PartialMember` (gateway member-without-user; interaction resolved members)
|
||||
- `Dexcord.PartialChannel` (interaction resolved channels)
|
||||
- `Dexcord.PartialGuild` (user-guilds list), `Dexcord.UnavailableGuild` (READY stubs)
|
||||
- `Dexcord.MessageSnapshot` (forwards)
|
||||
|
||||
No `PartialMessage`/`Object` equivalents — the id-accepting API layer makes them unnecessary.
|
||||
|
||||
### Gateway events
|
||||
|
||||
`Dexcord.Events` (`lib/dexcord/events.ex` + `lib/dexcord/events/*.ex`): a generated table maps event atom → payload type. Full-object events decode to the model struct (`{:MESSAGE_CREATE, %Dexcord.Message{}}`); envelope events get per-event structs (`{:MESSAGE_DELETE, %Dexcord.Events.MessageDelete{}}`, `{:MESSAGE_REACTION_ADD, %Dexcord.Events.ReactionAdd{}}`, …) so the handler always receives exactly what the gateway sent, typed. Handler contract: `{event_atom, typed_payload}` or `{{:unknown_event, name}, raw_map}` — the 2-tuple shape is unchanged.
|
||||
|
||||
Envelope structs declare **hydration slots**:
|
||||
|
||||
```elixir
|
||||
discord_struct do
|
||||
field :user_id, :snowflake
|
||||
field :channel_id, :snowflake
|
||||
field :guild_id, :snowflake
|
||||
field :emoji, {:struct, Dexcord.PartialEmoji}
|
||||
hydrate :user, from: :user_id, type: Dexcord.User
|
||||
hydrate :channel, from: :channel_id, type: :channel
|
||||
hydrate :guild, from: :guild_id, type: Dexcord.Guild
|
||||
end
|
||||
```
|
||||
|
||||
`Dexcord.Cache.fill/1` populates the slots from cache — best-effort (miss → slot stays `nil`), never blocks, never issues HTTP. Hydration is invoked only by user handler code, never by the dispatcher.
|
||||
|
||||
### REST surface
|
||||
|
||||
**Endpoint DSL** (`lib/dexcord/api/endpoint.ex`): ~12 group modules (`lib/dexcord/api/channels.ex`, `messages.ex`, `guilds.ex`, `members.ex`, `roles.ex`, `webhooks.ex`, `invites.ex`, `interactions.ex`, `commands.ex`, `emojis_stickers.ex`, `moderation.ex`, `misc.ex` — exact grouping settled in implementation planning) declare routes:
|
||||
|
||||
```elixir
|
||||
endpoint :create_message, :post, "/channels/:channel_id/messages",
|
||||
body: true, files: true, returns: Dexcord.Message
|
||||
|
||||
endpoint :get_guild_audit_log, :get, "/guilds/:guild_id/audit-logs",
|
||||
query: [:user_id, :action_type, :before, :after, :limit],
|
||||
returns: Dexcord.AuditLog
|
||||
|
||||
endpoint :delete_role, :delete, "/guilds/:guild_id/roles/:role_id",
|
||||
reason: true, returns: nil
|
||||
```
|
||||
|
||||
Each expands to a function with `@spec`, `@doc` (method + path + docs link), path interpolation (URL-encoded segments), query building, body encoding, response decoding into `returns:`. Conventions:
|
||||
|
||||
- Coverage target: **~180 endpoints** across the in-scope groups (channels, messages/reactions/pins, guilds/members/roles/bans, users, webhooks, invites, interaction responses/followups, application commands, emojis, stickers, polls, auto-moderation, scheduled events, audit logs, gateway/application info). Excluded (escape hatch): monetization, soundboard, stage instances, guild templates, deprecated pin routes, Bearer-only endpoints.
|
||||
- Return contract: `{:ok, struct | [struct] | nil}` | `{:error, %Dexcord.Api.Error{}}`. `request/4` stays public and map-based (the break-glass).
|
||||
- `reason: true` → `:reason` opt → `X-Audit-Log-Reason` header (hand-flagged on the ~60 mutating endpoints; the OpenAPI spec doesn't model reason headers).
|
||||
- `files: true` → multipart `payload_json` + `files[n]` encoding — new machinery alongside `request/4` in `lib/dexcord/api.ex`.
|
||||
- Snowflake arguments accept int, decimal string, or a struct with an `.id` field. Bodies accept keyword lists, maps, or structs (`to_map/1`).
|
||||
- Naming: Discord docs names, except "Modify X" → `edit_x`.
|
||||
- Sugar arities (e.g. `create_message(channel_id, "text")`) are hand-written over the generated base.
|
||||
- `Dexcord.Api` (`lib/dexcord/api.ex`) remains the flat front door: generated `defdelegate`s to the group modules + `request/4`.
|
||||
|
||||
**Ratelimit**: `Dexcord.Api.Ratelimit.route_key/2` (`lib/dexcord/api/ratelimit.ex`) gains `templatize` clauses for new route shapes — invite codes (non-snowflake segment must collapse), `sticker-packs`, application-emoji routes — so buckets neither over-share nor grow unbounded ETS keys. Message-delete's separate bucket family and webhook/interaction token digests already exist and are kept.
|
||||
|
||||
**Coverage tooling**: `mix dexcord.coverage` diffs the compiled route table against a checked-in snapshot of `discord-api-spec`'s `openapi.json` (`priv/discord_openapi.json` + a refresh task), reporting endpoints in the spec but missing from Dexcord (excluded groups allowlisted). Runs in CI; fails on regression.
|
||||
|
||||
### Ergonomics layer
|
||||
|
||||
- `Dexcord.Messageable` protocol (`lib/dexcord/messageable.ex`): `resolve/1 → {:channel, id} | {:dm_user, id}`; implemented for TextChannel, VoiceChannel, Thread, DMChannel, User, Member, Message, Interaction, and bare snowflakes. One `Dexcord.Api.send/2` funnel; `{:dm_user, id}` performs the create-DM dance with the DM channel id cached.
|
||||
- `Dexcord.Message.reply/2` (sets `message_reference`; `mention_author:` opt).
|
||||
- `mention/1` on User/Member/Role/channel structs; `String.Chars` impls; `created_at/1` wherever there's an id; `Dexcord.Util.format_dt/2` (`<t:…:R>` markup).
|
||||
- `Dexcord.Embed` pipe-friendly builder; `Dexcord.Permissions` (`has?/2`, `to_list/1`, `from_list/1`) plus `Dexcord.Guild.member_permissions/2` computing role + overwrite resolution.
|
||||
- `allowed_mentions`: app-level default in `Dexcord.Config`, per-send field-wise merge, `mention_author:` sugar.
|
||||
- Pagination: `Dexcord.Api.message_history/2` and friends (members, bans, audit log) return lazy `Stream`s that page transparently.
|
||||
|
||||
### Data flow
|
||||
|
||||
```
|
||||
Gateway (JSON.decode!) [unchanged]
|
||||
→ Dispatcher: Events.decode(name, raw) [decode ONCE]
|
||||
→ Cache.handle_dispatch(name, decoded, raw) [structs stored; merge_map for partial updates]
|
||||
→ handler Task: handle_event({name, decoded})
|
||||
→ Slash.dispatch(%Interaction{}, module) [typed interactions]
|
||||
```
|
||||
|
||||
Cache stores structs keyed by **integer snowflakes** (tables and single-writer discipline unchanged, `lib/dexcord/cache.ex`); GUILD_CREATE explodes typed children into the child tables; UPDATE events use `merge_map/2` so gateway-only fields (e.g. `joined_at`) survive partial updates. Reads return structs. A decode failure for one event logs, falls back to delivering the raw map, and never kills dispatch.
|
||||
|
||||
## Existing Patterns
|
||||
|
||||
Followed unchanged:
|
||||
|
||||
- **Single-writer cache via Dispatcher** (`lib/dexcord/dispatcher.ex:35-51`, `lib/dexcord/cache.ex`): cache writes stay inline in the dispatcher process, in gateway order; tables stay `:public`, `read_concurrency: true`, reads straight from ETS. Only the stored values (structs, not maps) and key types (integers, not strings) change.
|
||||
- **Behaviour + `@before_compile` catch-all injection** (`Dexcord.Handler`, `Dexcord.Slash`, `Dexcord.Prefix.Router`): kept as-is; the new DSLs follow the same macro hygiene conventions.
|
||||
- **Pure logic in standalone testable modules** (`Gateway.Payload`, `EventNames`, `Snowflake`, `Intents`): the DSLs, `Events` table, and endpoint expansion follow this.
|
||||
- **Config**: validated bot config in `:persistent_term` via `Dexcord.Config`; tuning knobs via `Application.get_env` (preserves the EnvSandbox test pattern). `allowed_mentions` default joins the validated config.
|
||||
- **Test harness** (`test/support/`): EnvSandbox + FakeRest + FakeGateway patterns are the template for all new tests; `api_integration_test.exs` is the model for endpoint round-trips.
|
||||
- **`Dexcord.Api.Error`** stays the error struct; `request/4` keeps its exact contract.
|
||||
- **Moduledoc style**: substantial `@moduledoc` with design rationale, `@spec` on all public functions, `@doc false` for internals.
|
||||
|
||||
Deliberate divergences:
|
||||
|
||||
- **The map contract is removed** at all four seams (handler events, cache values, Api returns, Slash interactions). This is the point of the design; no compatibility layer. v0.1's "minimal typed REST" locked decision is consciously revisited.
|
||||
- **Macro DSLs are a new pattern** in this codebase. Justification: ~130 shapes × 3 hand-written declarations each is the alternative, and it is the approach Nostrum demonstrably regrets (field drift, atomization hedges).
|
||||
- **`Dexcord.Snowflake.t()` narrows** from `integer() | String.t()` to integer-only. Justification: one canonical representation ends the string/int comparison bugs (alamedya's `String.to_integer` dance).
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
<!-- START_PHASE_1 -->
|
||||
### Phase 1: Foundations — DSLs and Snowflake
|
||||
**Goal:** The three code-generating DSLs and the integer Snowflake exist, fully tested; everything later is declarations.
|
||||
|
||||
**Components:**
|
||||
- `Dexcord.Struct` DSL in `lib/dexcord/struct.ex` — `discord_struct`/`field`/`hydrate` macros generating `defstruct`, `@type t`, `from_map/1`, `to_map/1`, `merge_map/2`; field types `:snowflake`, `:string`, `:integer`, `:boolean`, `:datetime`, `{:struct, M}`, `{:list, t}`, `{:enum, M}`, `{:flags, M}`; modifiers `default:`, `tristate:`, `:presence`
|
||||
- `Dexcord.Enum` DSL in `lib/dexcord/enum.ex` — `decode/1`/`encode/1` with `{:unknown, n}` round-trip
|
||||
- `Dexcord.Flags` DSL in `lib/dexcord/flags.ex` — raw-int storage, `has?/2`, `to_list/1`, `from_list/1`, `all/0`
|
||||
- Reworked `Dexcord.Snowflake` in `lib/dexcord/snowflake.ex` — integer `t()`, `cast/1`, `cast!/1`, `dump/1`, `is_snowflake/1` guard, fixed `to_unix/1`
|
||||
- Fixture modules + exhaustive DSL tests in `test/dexcord/struct_test.exs`, `enum_test.exs`, `flags_test.exs`, updated `snowflake_test.exs`
|
||||
|
||||
**Dependencies:** None.
|
||||
|
||||
**Done when:** DSL fixture modules compile with correct `defstruct`/`@type`; decode/encode/merge/tristate/presence/lenient-fallback behaviors all pass (`api-surface.AC2.1–AC2.9`); suite green.
|
||||
<!-- END_PHASE_1 -->
|
||||
|
||||
<!-- START_PHASE_2 -->
|
||||
### Phase 2: Core models
|
||||
**Goal:** The message-path object graph decodes from real payloads.
|
||||
|
||||
**Components:**
|
||||
- Models in `lib/dexcord/model/`: `User`, `Member`, `PartialMember`, `Message`, `Attachment`, `Embed` (+ sub-objects), `Reaction`, `Emoji`, `PartialEmoji`, `Role` (+ `RoleTags` with presence-booleans), `Guild`, `Channel` factory + `TextChannel`/`VoiceChannel`/`CategoryChannel`/`AnnouncementChannel`/`ForumChannel`/`MediaChannel`/`StageChannel`/`DMChannel`/`GroupDMChannel`/`Thread`/`UnknownChannel`, `UnavailableGuild`, `PartialGuild`, `VoiceState`, `Presence`, `Sticker`, `StickerItem`, `MessageSnapshot`
|
||||
- Enum/flag tables: `ChannelType`, `MessageType`, `MessageFlags`, `Permissions`, `UserFlags`, and companions the above need
|
||||
- Golden payload fixtures in `test/fixtures/` (GUILD_CREATE, MESSAGE_CREATE with member+mentions, DM message, forum/thread channels, webhook-authored message, reply with deleted referent) + field-level decode tests
|
||||
|
||||
**Dependencies:** Phase 1.
|
||||
|
||||
**Done when:** Golden fixtures decode field-correct, including the notorious corners (`api-surface.AC2.10–AC2.14`); channel factory dispatches all 15 documented types + unknown fallback.
|
||||
<!-- END_PHASE_2 -->
|
||||
|
||||
<!-- START_PHASE_3 -->
|
||||
### Phase 3: Remaining models and event envelopes
|
||||
**Goal:** Every in-scope object and gateway event has a typed home.
|
||||
|
||||
**Components:**
|
||||
- Remaining models in `lib/dexcord/model/`: `Interaction` (+ data variants, `ResolvedData` with `PartialChannel`/resolved partials), `ApplicationCommand` (+ options tree), component structs (ActionRow/Button/selects/TextInput + Components V2 set), `Webhook`, `Invite` (+ metadata), `AuditLog` (+ entries/changes), `AutoModerationRule` (+ triggers/actions), `GuildScheduledEvent` (+ recurrence), `Poll` (+ media/answers/results), `Application`, `Team`, `WelcomeScreen`, `Onboarding`, `Integration`, `Ban`, `Widget` objects, `ThreadMember`, `FollowedChannel`, `VoiceRegion`, `Connection`
|
||||
- `Dexcord.Events` decode table in `lib/dexcord/events.ex`; envelope structs with `hydrate` slots in `lib/dexcord/events/*.ex` for every non-full-object event in `Dexcord.EventNames` (reactions, deletes, bulk delete, member add/remove/update, role events, typing, thread list sync, poll votes, …)
|
||||
- Golden payload fixtures: INTERACTION_CREATE (slash + component + modal, guild and DM), reaction add, audit log entry, scheduled event
|
||||
|
||||
**Dependencies:** Phase 2.
|
||||
|
||||
**Done when:** `Events.decode/2` covers every atom in `Dexcord.EventNames.all/0` (full-object, envelope, or documented raw passthrough); interaction fixtures decode including resolved partials (`api-surface.AC2.15–AC2.17`).
|
||||
<!-- END_PHASE_3 -->
|
||||
|
||||
<!-- START_PHASE_4 -->
|
||||
### Phase 4: Endpoint DSL and Api core
|
||||
**Goal:** The endpoint machinery works end-to-end; the existing 16 endpoints are reborn as declarations.
|
||||
|
||||
**Components:**
|
||||
- Endpoint DSL in `lib/dexcord/api/endpoint.ex` — route declaration → function generation (`@spec`, docs, path interpolation with URL-encoded segments, `query:` building, `body:` encoding from keyword/map/struct, `reason:` header, `returns:` decoding)
|
||||
- Multipart machinery in `lib/dexcord/api.ex` — `payload_json` + `files[n]` encoding for `files: true` endpoints
|
||||
- `Ratelimit.route_key/2` extensions in `lib/dexcord/api/ratelimit.ex` — invite-code collapse, `sticker-packs`, application-emoji clauses
|
||||
- Existing 16 endpoints redeclared via the DSL in group modules; `Dexcord.Api` facade with generated delegates + `request/4` unchanged
|
||||
- Endpoint-machinery tests against FakeRest (path/query/body/reason/multipart on the wire; struct decode on return; error passthrough)
|
||||
|
||||
**Dependencies:** Phases 1–2 (Phase 3 models only needed for later groups).
|
||||
|
||||
**Done when:** All 16 existing endpoints pass FakeRest round-trips returning structs (`api-surface.AC1.1–AC1.7`); multipart and reason-header wire format verified; ratelimit templatizer tests cover new clauses.
|
||||
<!-- END_PHASE_4 -->
|
||||
|
||||
<!-- START_PHASE_5 -->
|
||||
### Phase 5: Full endpoint rollout and coverage gate
|
||||
**Goal:** ~180 endpoints declared; coverage is mechanically enforced.
|
||||
|
||||
**Components:**
|
||||
- All in-scope endpoint declarations across the group modules in `lib/dexcord/api/`
|
||||
- Sugar arities (`create_message/2` with binary, `get_channel_messages/3` locator form, etc.)
|
||||
- `mix dexcord.coverage` task in `lib/mix/tasks/dexcord.coverage.ex` + pinned `priv/discord_openapi.json` + refresh task; CI wiring
|
||||
- Representative per-group FakeRest tests (one round-trip per group minimum, plus every special shape: top-level-array bodies, `wait=true` webhooks, `with_response` callbacks, binary widget.png, no-auth routes)
|
||||
|
||||
**Dependencies:** Phases 3–4.
|
||||
|
||||
**Done when:** `mix dexcord.coverage` reports zero missing in-scope endpoints and runs in CI (`api-surface.AC1.8–AC1.10`); representative tests green.
|
||||
<!-- END_PHASE_5 -->
|
||||
|
||||
<!-- START_PHASE_6 -->
|
||||
### Phase 6: Gateway, cache, and slash integration
|
||||
**Goal:** The typed contract is live end-to-end; the map contract is gone.
|
||||
|
||||
**Components:**
|
||||
- `Dexcord.Dispatcher` decodes once via `Events.decode/2`; delivers `{name, typed}` to handler Tasks; decode-failure fallback to raw with logging
|
||||
- `Dexcord.Cache` stores structs keyed by integer snowflakes; typed GUILD_CREATE explosion; `merge_map/2` on UPDATE events; reads return structs; `Dexcord.Cache.fill/1` hydration
|
||||
- `Dexcord.Slash` typed: `%Interaction{}` into callbacks, struct-or-map `commands/0`, response helpers accepting keyword/map/struct bodies
|
||||
- `Dexcord.Prefix` internals matching on `%Message{}`
|
||||
- Updated integration tests: `cache_integration_test.exs`, `cache_cascade_test.exs`, `gateway_integration_test.exs`, `slash_test.exs`, `ergonomics_integration_test.exs`, cache-merge scenarios, `Cache.fill` round-trips
|
||||
|
||||
**Dependencies:** Phase 3 (models/events); independent of Phases 4–5.
|
||||
|
||||
**Done when:** FakeGateway end-to-end flows deliver typed events, cache returns structs surviving partial updates, hydration fills from cache (`api-surface.AC2.18–AC2.22`, `api-surface.AC3.8`); no map-shaped reads remain.
|
||||
<!-- END_PHASE_6 -->
|
||||
|
||||
<!-- START_PHASE_7 -->
|
||||
### Phase 7: Ergonomics, docs, and the alamedya guide
|
||||
**Goal:** The discord.py taste layer; the library is adoptable.
|
||||
|
||||
**Components:**
|
||||
- `Dexcord.Messageable` protocol in `lib/dexcord/messageable.ex` + `Dexcord.Api.send/2` funnel with create-DM handling
|
||||
- `Dexcord.Message.reply/2`; `mention/1` + `String.Chars` + `created_at/1` across models; `Dexcord.Util.format_dt/2`
|
||||
- `Dexcord.Embed` builder; `Dexcord.Guild.member_permissions/2`; `allowed_mentions` config default + merge
|
||||
- Pagination streams (`message_history/2`, member/ban/audit-log streams)
|
||||
- New migration guide `docs/alamedya-migration-v2.md` written against the struct contract; README + moduledoc refresh
|
||||
- Ergonomics tests (protocol dispatch incl. DM dance, reply reference wiring, permission computation, allowed-mentions merge, stream pagination against FakeRest)
|
||||
|
||||
**Dependencies:** Phases 4–6.
|
||||
|
||||
**Done when:** Ergonomics ACs pass (`api-surface.AC3.1–AC3.7`); guide exists and compiles against the final API (`api-surface.AC5.1`); full suite green, `mix dexcord.coverage` green (`api-surface.AC4.1–AC4.3`).
|
||||
<!-- END_PHASE_7 -->
|
||||
|
||||
## Additional Considerations
|
||||
|
||||
**Decode failure blast radius:** one malformed event must never take down dispatch — `Events.decode/2` rescues per-event, logs at warning with the event name (never the full payload at warn level), and falls back to `{{name}, raw_map}` delivery so a handler can still act. Cache skips writes for events whose decode failed rather than storing half-decoded structs.
|
||||
|
||||
**GUILD_CREATE size:** decode of a large guild (channels + members + presences) happens once, inline in the dispatcher, replacing today's raw-map writes. The generated `from_map/1` is a single traversal with no atomization pass; if profiling during Phase 6 shows dispatcher lag on very large guilds, the escape valve is decoding guild children lazily per-table (design permits; not built preemptively).
|
||||
|
||||
**Spec drift:** `mix dexcord.coverage` compares against a *pinned* `openapi.json`; refreshing the pin is a deliberate act (`mix dexcord.coverage.refresh`), so upstream spec churn never breaks CI spontaneously. The spec is "public preview" — docs win on conflict, and reason-header support is hand-flagged because the spec omits it entirely.
|
||||
|
||||
**Registrar/global-wipe guards:** the Registrar's empty-command-list guard and retry containment (`lib/dexcord/slash/registrar.ex`) are untouched; `commands/0` returning structs is normalized to maps before the existing logic.
|
||||
|
||||
**Out of scope, still reachable:** monetization, soundboard, stage instances, guild templates, and Bearer-only endpoints remain on `request/4`. New Discord surface lands first as an `{:unknown, n}` enum value, an unmatched map key (silently ignored), or a missing endpoint flagged by the coverage task — all non-breaking.
|
||||
Loading…
Add table
Add a link
Reference in a new issue