out-of-your-element/readme.md

133 lines
6.3 KiB
Markdown
Raw Normal View History

2023-09-06 00:00:39 +00:00
# Out Of Your Element
Modern Matrix-to-Discord appservice bridge.
## Why a new bridge?
* Modern: Supports new Discord features like replies, threads and stickers, and new Matrix features like edits, spaces and space membership.
* Reliable: Any errors on either side are notified on Matrix and can be retried.
* Tested: A test suite and code coverage make sure all the core logic works.
* Simple development: No build step (it's JavaScript, not TypeScript), minimal/lightweight dependencies, and abstraction only where necessary so that less background knowledge is required. No need to learn about Intents or library functions.
2023-09-11 00:26:53 +00:00
* No locking algorithm: Other bridges use a locking algorithm which is a source of frequent bugs. This bridge avoids the need for one.
* Latest API: Being on the latest Discord API version lets it access all features, without the risk of deprecated API versions being removed.
2023-09-06 00:00:39 +00:00
## What works?
2023-09-06 00:04:15 +00:00
Most features you'd expect in both directions, plus a little extra spice:
2023-09-06 00:00:39 +00:00
* Messages
* Edits
* Deletions
2023-09-06 00:04:15 +00:00
* Text formatting, including spoilers
2023-09-06 00:00:39 +00:00
* Reactions
2023-09-06 00:04:15 +00:00
* Mentions
2023-09-06 00:00:39 +00:00
* Replies
2023-09-06 01:07:05 +00:00
* Threads
2023-09-10 09:35:51 +00:00
* Stickers (all formats: PNG, APNG, GIF, and Lottie)
2023-09-06 00:04:15 +00:00
* Attachments
* Spoiler attachments
* Guild-Space details syncing
* Channel-Room details syncing
* Custom room names/avatars can be applied on Matrix-side
* Larger files from Discord are linked instead of reuploaded to Matrix
2023-09-06 00:00:39 +00:00
## Caveats
* Custom emojis don't fully work yet.
* Embeds don't work yet.
2023-09-06 01:07:05 +00:00
* This bridge is not designed for puppetting.
2023-09-06 00:00:39 +00:00
* Some aspects of this bridge are customised for my homeserver. I'm working over time to make it more general. Please please reach out to @cadence:cadence.moe if you would like to run this, and I'll work with you to get it running!
# Development information
## You will need
2023-09-11 00:26:53 +00:00
* Administrative access to a homeserver
2023-09-06 00:00:39 +00:00
* Discord bot
* (For now) Help and support from @cadence:cadence.moe. Message me and tell me you're interested in OOYE!
* The L1 and L2 emojis
## Initial setup
Node.js version 18 or later is required: https://nodejs.org/en/download/releases (the matrix-appservice dependency demands 18)
2023-09-12 08:43:56 +00:00
Install dependencies: `npm install --save-dev` (only need --save-dev if you will run the automated tests)
2023-09-06 00:00:39 +00:00
Copy `config.example.js` to `config.js` and fill in Discord token.
2023-09-12 08:43:56 +00:00
Copy `registration.example.yaml` to `registration.yaml` and fill in bracketed values. You could generate each hex string with `dd if=/dev/urandom bs=32 count=1 2> /dev/null | basenc --base16 | dd conv=lcase 2> /dev/null`. Register the registration in Synapse's `homeserver.yaml` through the usual appservice installation process, then restart Synapse.
2023-09-06 00:00:39 +00:00
2023-09-12 08:43:56 +00:00
If developing on a different computer to the one running the homeserver, use SSH port forwarding so that Synapse can connect on its `localhost:6693` to reach the running bridge on your computer. Example: `ssh -T -v -R 6693:localhost:6693 me@matrix.cadence.moe`
2023-09-06 00:00:39 +00:00
2023-09-12 08:43:56 +00:00
Run `node scripts/seed.js` to check your setup, create the database and server state (only need to run this once ever)
2023-09-06 01:07:05 +00:00
2023-09-06 00:00:39 +00:00
Make sure the tests work: `npm t`
2023-09-06 01:07:05 +00:00
Start the bridge: `node start.js`
Any files you change will automatically be reloaded, except for `stdin.js` and `d2m/discord-*.js`
I recommend developing in Visual Studio Code so that the JSDoc x TypeScript annotation comments work. I don't know which other editors or language servers support annotations and type inference.
2023-09-06 00:00:39 +00:00
## Repository structure
.
2023-09-06 01:07:05 +00:00
* Run this to start the bridge:
├── start.js
2023-09-06 00:00:39 +00:00
* Runtime configuration, like tokens and user info:
├── config.js
├── registration.yaml
* The bridge's SQLite database is stored here:
├── db
│   └── *.sql, *.db
* Discord-to-Matrix bridging:
├── d2m
│   * Execute actions through the whole flow, like sending a Discord message to Matrix:
│   ├── actions
│   │   └── *.js
│   * Convert data from one form to another without depending on bridge state. Called by actions:
│   ├── converters
│   │   └── *.js
│   * Making Discord work:
│   ├── discord-*.js
│   * Listening to events from Discord and dispatching them to the correct `action`:
│   └── event-dispatcher.js
* Matrix-to-Discord bridging:
├── m2d
│   * Execute actions through the whole flow, like sending a Matrix message to Discord:
│   ├── actions
│   │   ├── *.js
│   ├── converters
│   │   └── *.js
│   └── event-dispatcher.js
2023-09-06 01:07:05 +00:00
* We aren't using the matrix-js-sdk, so here's all the stuff we need to call the Matrix C-S API:
2023-09-06 00:00:39 +00:00
├── matrix
│   └── *.js
* Various files you can run once if you need them. Hopefully you won't need them.
├── scripts
2023-09-06 01:07:05 +00:00
│   ├── *.js
│   * First time running a new bridge? Run this file to plant a seed, which will flourish into state for the bridge:
│   └── seed.js
* You are here! :)
2023-09-06 00:00:39 +00:00
└── readme.md
## Dependency justification
2023-09-06 01:07:05 +00:00
(deduped transitive dependency count) dependency name: explanation
2023-09-06 00:00:39 +00:00
2023-09-06 01:07:05 +00:00
* (0) @chriscdn/promise-semaphore: It does what I want! I like it!
* (42) better-sqlite3: SQLite3 is the best database, and this is the best library for it. Really! I love it.
2023-09-06 00:00:39 +00:00
* (1) chunk-text: It does what I want.
* (0) cloudstorm: Discord gateway library with bring-your-own-caching that I trust.
* (8) snowtransfer: Discord API library with bring-your-own-caching that I trust.
* (1) discord-markdown: This is my fork! I make sure it does what I want.
* (1) heatsync: Module hot-reloader that I trust.
* (1) js-yaml: It seems to do what I want, and it's already pulled in by matrix-appservice.
2023-09-06 01:07:05 +00:00
* (70) matrix-appservice: I wish it didn't pull in express :(
2023-09-06 00:00:39 +00:00
* (0) mixin-deep: This is my fork! It fixes a bug in regular mixin-deep.
* (3) node-fetch@2: I like it and it does what I want.
2023-09-10 09:35:51 +00:00
* (0) pngjs: Lottie stickers are converted to bitmaps with the vendored Rlottie WASM build, then the bitmaps are converted to PNG with pngjs.
2023-09-06 00:00:39 +00:00
* (0) prettier-bytes: It does what I want and has no dependencies.
* (0) try-to-catch: Not strictly necessary, but it does what I want and has no dependencies.
* (1) turndown: I need an HTML-to-Markdown converter and this one looked suitable enough. It has some bugs that I've worked around, so I might switch away from it later.