d2m username colors #107

Open
mschae wants to merge 8 commits from mschae/out-of-your-element:role-colors into main
33 changed files with 411 additions and 84 deletions
Showing only changes of commit 68799aebbe - Show all commits

Merge branch 'main' into role-colors

mschae23 2026-11-11 19:11:05 +01:00
Signed by untrusted user: mschae
GPG key ID: A87721D300DE88AB

49
.github/workflows/main.yml vendored Normal file
View file

@ -0,0 +1,49 @@
# Copied from: https://github.com/actions/starter-workflows/blob/e3c451d60f119b71caebf13c98ac45da6e15b4b7/pages/jekyll.yml#L4
name: Publish MkDocs to Pages
on:
push:
branches:
- main
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
build:
# Specify runner, build static files as an artifact
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v1
with:
ref: main
- name: Build static files
id: build
run: |
pip install mkdocs-material
mkdocs build
- name: Upload static files as artifact
id: deployment
uses: actions/upload-pages-artifact@v3
with:
path: site/
# Deploy built files
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View file

@ -1,11 +1,13 @@
# Development setup
# Developer orientation
## Development setup
* Install development dependencies with `npm install --save-dev` so you can run the tests.
* Most files you change, such as actions, converters, and web, will automatically be reloaded.
* If developing on a different computer to the one running the homeserver, use SSH port forwarding so that Synapse can connect on its `localhost:6693` to reach the running bridge on your computer. Example: `ssh -T -v -R 6693:localhost:6693 me@matrix.cadence.moe`
* I recommend developing in Visual Studio Code so that the JSDoc x TypeScript annotation comments just work automatically. I don't know which other editors or language servers support annotations and type inference.
# Efficiency details
## Efficiency details
Using WeatherStack as a thin layer between the bridge application and the Discord API lets us control exactly what data is cached in memory. Only necessary information is cached. For example, member data, user data, message content, and past edits are never stored in memory. This keeps the memory usage low and also prevents it ballooning in size over the bridge's runtime.
@ -18,7 +20,7 @@ File uploads (like avatars from bridged members) are checked locally and dedupli
Switching to [WAL mode](https://www.sqlite.org/wal.html) could improve your database access speed even more. Run `node scripts/wal.js` if you want to switch to WAL mode. (This will also enable `synchronous = NORMAL`.)
# Repository structure
## Repository structure
.
* Runtime configuration, like tokens and user info:
@ -73,13 +75,13 @@ Switching to [WAL mode](https://www.sqlite.org/wal.html) could improve your data
* Hopefully you won't need the rest of these. Code quality varies wildly.
└── *.js
# Read next
## Read next
If you haven't set up Out Of Your Element yet, you might find [Simplified homeserver setup](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/simplified-homeserver-setup.md) helpful.
If you haven't set up Out Of Your Element yet, you might find [Simplified homeserver setup](./simplified-homeserver-setup.md) helpful.
If you don't know what the Matrix event JSON generally looks like, turn on developer tools in your client (Element has pretty good ones). Right click a couple of messages and see what they look like on the inside.
I recommend first reading [How to add a new event type](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/how-to-add-a-new-event-type.md) as this will fill you in on key information in how the codebase is organised, which data structures are important, and what level of abstraction we're working on.
I recommend first reading [How to add a new event type](./how-to-add-a-new-event-type.md) as this will fill you in on key information in how the codebase is organised, which data structures are important, and what level of abstraction we're working on.
If you haven't seen the [Discord API documentation](https://discord.com/developers/docs/) before, have a quick look at one of the pages on there. Same with the [Matrix Client-Server APIs](https://spec.matrix.org/latest/client-server-api/). You don't need to know these inside out, they're primarily references, not stories. But it is useful to have an idea of what a couple of the API endpoints look like, the kind of data they tend to accept, and the kind of data they tend to return.
@ -87,7 +89,7 @@ Then you might like to peruse the other files in the docs folder. Most of these
Whether you read those or not, I'm more than happy to help you 1-on-1 with coding your dream feature. Join the chatroom [#out-of-your-element:cadence.moe](https://matrix.to/#/#out-of-your-element:cadence.moe) or PM me [@cadence:cadence.moe](https://matrix.to/#/@cadence:cadence.moe) and ask away.
# Dependency justification
## Dependency justification
Total transitive production dependencies: 114

View file

@ -1,4 +1,4 @@
# Foreign keys in the Out Of Your Element database
# Database foreign keys
Historically, Out Of Your Element did not use foreign keys in the database, but since I found a need for them, I have decided to add them. Referential integrity is probably valuable as well.

View file

@ -7,6 +7,7 @@ It's much easier to understand code with examples. So let's go through it togeth
First, we need to know what pinned events are supposed to look like. The Matrix C-S spec gives this example:
> **pinned** ... [string] ... Required: An ordered list of event IDs to pin.
>
> ```json
> {
> "content": {
@ -26,8 +27,8 @@ This is part of the persistent room state. Simple enough. To update this, the st
What does it look like on Discord-side?
> **Get Pinned Messages** \
> `GET` `/channels/{channel.id}/pins` \
> **Get Pinned Messages**<br>
> `GET` `/channels/{channel.id}/pins`<br>
> Returns all pinned messages in the channel as an array of message objects.
This is an API request to get the pinned messages. To update this, an API request will pin or unpin any specific message, adding or removing it from the list.
@ -289,10 +290,10 @@ As long as the database is consistent, this new `select` will return at most 1 e
Actions call converters to do the thing, but actions have to receive their input event from somewhere. Let's wire it up so we receive a "pins changed" event from Discord and do the whole flow from there. Checking the documentation again, Discord will trigger this gateway event when the pins change:
> **Channel Pins Update** \
> Sent when a message is pinned or unpinned in a text channel. This is not sent when a pinned message is deleted. \
> **guild_id?** ... snowflake ... ID of the guild \
> **channel_id** ... snowflake ... ID of the channel \
> **Channel Pins Update**<br>
> Sent when a message is pinned or unpinned in a text channel. This is not sent when a pinned message is deleted.<br>
> **guild_id?** ... snowflake ... ID of the guild<br>
> **channel_id** ... snowflake ... ID of the channel<br>
> **last_pin_timestamp?** ... ?ISO8601 timestamp ... Time at which the most recent pinned message was pinned
Notably, the event doesn't deliver the actual list of pinned messages to us. We'll have to listen for this event, then trigger an API request to `GET` the pins list. Alright, enough preparation, time to code.

View file

@ -1,3 +1,5 @@
# PluralKit notetaking
## What is PluralKit
PluralKit is a Discord bot. After a Discord user registers with PK, PK will delete and repost their messages. The reposted messages will be sent by a webhook with a custom display name and avatar. This effectively lets a person assume a custom display name and avatar at will on a per-message basis. People use this for roleplaying and/or dissociative-identity-disorder things. PK is extremely popular.

View file

@ -1,6 +1,10 @@
# d2m
---
title: Algorithms notetaking
---
## Known issues
## d2m
### Known issues
- m->d code blocks have slightly too much spacing
- rooms will be set up even if the bridge does not have permission for the channels, which breaks when it restarts and tries to fetch messages
@ -12,7 +16,7 @@
- pluralkit considerations for artemis
- consider whether to use nested spaces for channel categories and threads
## Mapping
### Mapping
Remember that a discord message may be transformed to multiple matrix messages.
@ -23,14 +27,14 @@ A database will be used to store the discord id to matrix event id mapping. Tabl
There needs to be a way to easily manually trigger something later. For example, it should be easy to manually retry sending a message, or check all members for changes, etc.
## Current manual process for setting up a server
### Current manual process for setting up a server
1. Call createSpace.createSpace(discord.guilds.get(GUILD_ID))
2. Call createRoom.createAllForGuild(GUILD_ID) // TODO: Only create rooms that the bridge bot has read permissions in!
3. Edit source code of event-dispatcher.js isGuildAllowed() and add the guild ID to the list
4. If developing, make sure SSH port forward is activated, then wait for events to sync over!
## Transforming content
### Transforming content
1. Upload attachments to mxc if they are small enough.
2. Convert discord message text and embeds to matrix event.
@ -43,9 +47,9 @@ There needs to be a way to easily manually trigger something later. For example,
5. Send attachments.
6. Store in database.
## Discord's permissions in spaces
### Discord's permissions in spaces
### The space itself
#### The space itself
Discord guilds are invite only, so the corresponding **space** should initially be set to:
@ -64,14 +68,14 @@ Step by step process:
2. Create channel rooms for the channels. Store the channel-room ID relationship in the database. (Probably no need to store parent-child relationships in the database?)
3. Send state events to put the channel rooms in the space.
### Private channels
#### Private channels
Discord **channels** that disallow view permission to @everyone should instead have the following **room** settings in Matrix:
- Find & join access: Private (so space members cannot join without an additional invitation)
- Who can read history: Anyone (XXX: is this safe??? is this a fishbowl situation? https://github.com/matrix-org/synapse/issues/9202)
### Discord experience: /invite command
#### Discord experience: /invite command
The context-sensitive /invite command will invite Matrix users to the corresponding spaces or rooms.
@ -79,9 +83,9 @@ The context-sensitive /invite command will invite Matrix users to the correspond
- In a **private channel**, /invite will invite a distinct username or MXID to the **room**, provided the user has Manage Server.
- In a **public or private thread**, /invite will invite a distinct username or MXID to the **thread room**.
# d2m events
## d2m events
## Login - backfill
### Login - backfill
Need to backfill any messages that were missed while offline.
@ -97,31 +101,31 @@ For channels that have missed messages, use the getChannelMessages function, and
Can use custom transaction ID (?) to send the original timestamps to Matrix. See appservice docs for details.
## Message sent
### Message sent
1. Transform content.
2. Send to matrix.
## Webhook message sent
### Webhook message sent
- Consider using the _ooye_bot account to send all webhook messages to prevent extraneous joins?
- Downside: the profile information from the most recently sent message would stick around in the member list. This is tolerable.
- Otherwise, could use an account per webhook ID, but if webhook IDs are often deleted and re-created, this could still end up leaving too many accounts in the room.
- The original bridge uses an account per webhook display name, which makes the most sense in terms of canonical accounts, but leaves too many accounts in the room.
## Message deleted
### Message deleted
1. Look up equivalents on matrix.
2. Delete on matrix.
## Message edited / embeds added
### Message edited / embeds added
1. Look up equivalents on matrix.
2. Transform content.
3. Build replacement event with fallbacks.
4. Send to matrix.
## Reaction added/removed/emoji removed/all removed
### Reaction added/removed/emoji removed/all removed
m->d reactions will have to be sent as the bridge bot since webhooks cannot add reactions. This also means Discord users can't tell who reacted. We will have to tolerate this.
@ -147,13 +151,13 @@ Need to know the discord ID of the message that was reacted to. If we know the e
So we do need a database table. It will only hold reactions that were sent by Matrix users and were successfully bridged. It will associate the reaction event ID with the Discord message ID it was reacted on (skipping the middleman).
## Member data changed
### Member data changed
1. Compare current member against cached version in database.
2. Update member on matrix.
3. Update cached version in database.
## Channel created / updated
### Channel created / updated
(but I should be able to manually call this function at any time to run the same code on any given channel)
@ -165,7 +169,7 @@ So we do need a database table. It will only hold reactions that were sent by Ma
4. Make sure the permissions are correct according to the rules above!
5. Add to space.
## Emojis updated
### Emojis updated
1. Upload any newly added images to msc.
2. Create or replace state event for the bridged pack. (Can just use key "ooye" and display name "Discord", or something, for this pack.)
@ -177,23 +181,23 @@ insert into emoji select replace(substr(discord_url, 35), ".gif", "") as id, 1 a
insert into emoji select replace(substr(discord_url, 35), ".png", "") as id, 0 as animated, mxc_url from file where discord_url like 'https://cdn.discordapp.com/emojis/%.png';
```
# Various considerations
## Various considerations
## Issues if the bridge database is rolled back
### Issues if the bridge database is rolled back
### channel_room table
#### channel_room table
- Duplicate rooms will be created on matrix.
### sim table
#### sim table
- Sims will already be registered, registration will fail, all events from those sims will fail.
### guild_space table
#### guild_space table
- channelToKState will fail, so channel data differences won't be calculated, so channel/thread creation and sync will fail.
### event_message table
#### event_message table
- Events referenced by other events will be dropped, for example
- edits will be ignored
@ -201,19 +205,19 @@ insert into emoji select replace(substr(discord_url, 35), ".png", "") as id, 0 a
- reactions will be ignored
- replies won't generate a reply
### file
#### file
- Some files like avatars may be re-uploaded to the matrix content repository, secretly taking more storage space on the server.
### webhook
#### webhook
- Some duplicate webhooks may be created.
### sim_member table
#### sim_member table
- Some sims will try to re-join the room, which is slow the first time.
## Creating and notifying about new threads:
### Creating and notifying about new threads:
Discord's gateway events when a thread is created off a message:
@ -225,27 +229,27 @@ Discord's gateway events when a thread is created off a message:
When viewing this thread, it shows the message branched from at the top, and then the first "real" message right underneath, as separate groups.
### Problem 1
#### Problem 1
If THREAD_CREATE creates the matrix room, this will still be in-flight when MESSAGE_CREATE ensures the room exists and creates a room too. There will be two rooms created and the bridge falls over.
#### Possible solution: Ignore THREAD_CREATE
##### Possible solution: Ignore THREAD_CREATE
Then the room will be implicitly created by the two MESSAGE_CREATEs, which are in series.
#### Possible solution: Store in-flight room creations - ✔️ this solution is implemented
##### Possible solution: Store in-flight room creations - ✔️ this solution is implemented
Then the room will definitely only be created once, and we can still handle both events if we want to do special things for THREAD_CREATE.
#### Possible solution: Don't implicitly create rooms
##### Possible solution: Don't implicitly create rooms
But then old and current threads would never have their messages bridged unless I manually intervene. Don't like that.
### Problem 2
#### Problem 2
MESSAGE_UPDATE with flags=32 is telling that message to become an announcement of the new thread's creation, but this happens before THREAD_CREATE. The matrix room won't actually exist when we see MESSAGE_UPDATE, therefore we cannot make the MESSAGE_UPDATE link to the new thread.
#### Possible solution: Ignore MESSAGE_UPDATE and bridge THREAD_CREATE as the announcement - ✔️ this solution is implemented
##### Possible solution: Ignore MESSAGE_UPDATE and bridge THREAD_CREATE as the announcement - ✔️ this solution is implemented
When seeing THREAD_CREATE (if we use solution B above) we could react to it by creating the thread announcement message in the parent channel. This is possible because THREAD_CREATE gives a thread object and that includes the parent channel ID to send the announcement message to.

View file

@ -0,0 +1,89 @@
# Private messages
## Invite flow (m->d)
1. Matrix user invites OOYE sim to chat
2. OOYE verifies that they share a room that they've both(?) spoken in (by member cache).
- If so, says "enter a message and it will send the user a message request"
- If not, says "you have to share a room first, or the discord user has to start chat"
3. Wait for first message from Matrix user
- Continue accepting further messages from Matrix user but put them in backlog
- Backlog is just stored in the room on Matrix. Use an event ID to point into it (?)
4. Announce to Discord user that they have a message request
5. Discord user acts on message request
- Allowed: Add Matrix user to allow-list and unspool backlog
- Denied: Add Matrix to deny-list and dismiss message request
## Invite flow (d->m)
1. Discord user invites Matrix user to chat (e.g. with /dm command)
2. OOYE informs Discord user that the Matrix user has been invited and messages will show up here
3. Add Matrix user to allow-list (clear deny-list if applicable)
4. Send the message to the Matrix user
## Unspool backlog
1. If not the most recent chatter (store in database), post user's card image, use components or something to stand out
2. Use pagination API to start at marked event and send subsequent events in order
## Database structures
- Allow-list pairing a Discord user and Matrix user
- Deny-list pairing a Discord user and Matrix user
- Conversation location pairing a Discord user and Matrix user with a room ID and a DM channel ID
- Backlog pairing a Discord user and Matrix user with a timeline event in the Matrix room
```
BEGIN TRANSACTION;
CREATE TABLE "pm" (
"user_id" TEXT NOT NULL,
"mxid" TEXT NOT NULL,
"status" INTEGER NOT NULL,
"channel_id" TEXT NOT NULL,
"room_id" TEXT NOT NULL,
"spool_event" TEXT,
PRIMARY KEY("user_id","mxid")
) WITHOUT ROWID;
COMMIT;
```
### Statuses
- pending
- allowed
- denied
-
## What hell am I putting the database through
Ideally I would like to be able to reuse all existing code that handles looking up existing messages: reactions, polls, editing, deleting, all that. It should be abstract enough to work in either DM channels or guild channels.
So what database structures does that rely on?
- `event_message`
- `message_room`
- `historical_channel_room`
- `channel_room`
- `guild_active` (does not require `guild_space`)
Much of this makes sense - it would be good to be able to seamlessly upgrade DM rooms for example - but I have to break the link at either `channel_room` or `guild_active`.
The `channel_room` table is a convenient place to look up the link. However, it also includes concerns (like the name and speedbump) that are not applicable to DMs. Also, its `channel_id` is the primary key, but on Discord, multiple Matrix user DMs will be in the same DM channel ID, so this primary key either needs to be changed or fabricated keys need to be used, neither of which seems pretty.
So I'm leaning towards not using `channel_room` for DM channels, but the other option is using it and disconnecting it from `guild_active`.
I'm also concerned about whether it's a problem for multiple _active_ rooms in `historical_channel_room` to share a `reference_channel_id`, because isn't that used to specify whether the rooms are part of the same history?
## Brainwave while out walking (not sure if it's correct yet)
I think I figured out the data structures for doing dms on the bridge!
the trick is to not store room_id in channel_room (just make it nullable) and to rely on historical_channel_room for pairing everyone simultaneously
so when I get home if I have dopamine I'll look into how much it would hurt if that column was nullable
and how much it would hurt if historical_channel_room could hold multiple active rooms
if all that seems possible then I'll run a schema change at the same time to move speedbump associations to another table
## Outstanding issues
- What do we do if the Matrix user leaves the room but wants to continue the conversation later?

View file

@ -1,4 +1,4 @@
# Simplified Homeserver Setup
# Simplified homeserver setup
Full instructions are located around here: https://matrix-org.github.io/synapse/v1.37/setup/installation.html

View file

@ -6,7 +6,7 @@ This also goes for Podman, Nix, and other similar technology that upends a progr
## What I recommend
I recommend [following the official setup guide,](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md) which does not use Docker.
I recommend [following the official setup guide,](./install.md) which does not use Docker.
Ultimately, though, do what makes you happy. I can't stop you from doing what you want. As long as you read this page and understand my perspective, that's good enough for me.
@ -44,13 +44,13 @@ OOYE was designed to be simple and courteous:
### If you care about ease of setup
In my opinion, the [official setup process](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md) is straightforward. After installing prerequisites (Node.js and the repo clone), the rest of the process interactively guides you through providing necessary information. Your input is checked for correctness so the bridge will definitely work when you run it.
In my opinion, the [official setup process](./install.md) is straightforward. After installing prerequisites (Node.js and the repo clone), the rest of the process interactively guides you through providing necessary information. Your input is checked for correctness so the bridge will definitely work when you run it.
I find this easier than the usual Docker workflow of pasting values into a YAML and rolling the dice on whether it will start up or not.
### If you care about security in the case of compromise/RCE
There are no known vulnerabilities in dependencies. I [carefully selected simple, light dependencies](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/developer-orientation.md#dependency-justification) to reduce attack surface area.
There are no known vulnerabilities in dependencies. I [carefully selected simple, light dependencies](../for-developers/developer-orientation.md#dependency-justification) to reduce attack surface area.
For defense in depth, I suggest running OOYE as a different user.

View file

@ -1,6 +1,10 @@
# Setup
---
title: Install
---
If you want Docker, [please read this first.](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/docker.md)
## Setup
If you want Docker, [please read this first.](./docker.md)
If you get stuck, you're welcome to message [#out-of-your-element:cadence.moe](https://matrix.to/#/#out-of-your-element:cadence.moe) or [@cadence:cadence.moe](https://matrix.to/#/@cadence:cadence.moe) to ask for help setting up OOYE!
@ -8,7 +12,7 @@ You'll need:
* Administrative access to a homeserver
* Discord bot
* Domain name for the bridge's website - [more info](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/why-does-the-bridge-have-a-website.md)
* Domain name for the bridge's website - [more info](./why-does-the-bridge-have-a-website.md)
* Reverse proxy for that domain - an interactive process will help you set this up in step 5!
Follow these steps:
@ -18,7 +22,7 @@ Follow these steps:
1. Switch to a normal user account. (i.e. do not run any of the following commands as root or sudo.)
1. Clone this repo and checkout a specific tag. (Development happens on main. Stable versions are tagged.)
* The latest release tag is ![](https://img.shields.io/gitea/v/release/cadence/out-of-your-element?gitea_url=https%3A%2F%2Fgitdab.com&style=flat-square&label=%20&color=black).
* The latest release tag is ![](https://img.shields.io/gitea/v/release/cadence/out-of-your-element?gitea_url=https%3A%2F%2Fgitdab.com&style=flat-square&label=%20&color=white).
1. Install dependencies: `npm install`
@ -45,7 +49,7 @@ New versions are announced in [#updates](https://matrix.to/#/#ooye-updates:caden
1. Restart the bridge: Stop the currently running process, and then start the new one with `npm run start`
# Get Started
## Bridge something
Visit the website on the domain name you set up, and click the button to add the bot to your Discord server.
@ -57,9 +61,9 @@ After that, to get into the rooms on your Matrix account, use the invite form on
I hope you enjoy Out Of Your Element!
# Extras
## Extras
## /plu/ral support
### /plu/ral support
To support /plu/ral webhook proxying, OOYE needs to call the /plu/ral API authenticated. I can't provide an API key for everyone to share, so you have to make your own. Here's how:
@ -68,9 +72,9 @@ To support /plu/ral webhook proxying, OOYE needs to call the /plu/ral API authen
3. Create a new application named Out Of Your Element. Copy the displayed token.
3. Run `node scripts/plu-ral-api.js` and paste the token.
# Appendix
## Appendix
## Example reverse proxy for nginx, dedicated domain name
### Example reverse proxy for nginx, dedicated domain name
Replace `bridge.cadence.moe` with the hostname you're using.
@ -98,14 +102,14 @@ server {
}
```
## Example reverse proxy for nginx, sharing a domain name
### Example reverse proxy for nginx, sharing a domain name
Same as above, but change the following:
- `location / {` -> `location /ooye/ {` (any sub-path you want; you MUST use a trailing slash or it won't work)
- `proxy_pass http://127.0.0.1:6693;` -> `proxy_pass http://127.0.0.1:6693/;` (you MUST use a trailing slash on this too or it won't work)
## Example reverse proxy for Caddy, dedicated domain name
### Example reverse proxy for Caddy, dedicated domain name
```nix
bridge.cadence.moe {
@ -118,8 +122,8 @@ bridge.cadence.moe {
}
```
## Example reverse proxy for traefik
### Example reverse proxy for traefik
Note: Out Of Your Element has no official Docker support. This guide is for using traefik when OOYE is ***not*** in a container.
See [third-party/reverse-proxy-traefik.md](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/third-party/reverse-proxy-traefik.md)
See [third-party/reverse-proxy-traefik.md](./reverse-proxy-traefik.md)

View file

@ -28,9 +28,9 @@ Now that you've done that, the privacy policy is linked at the bottom of every w
This [application by Amber of kathund.dev](https://bridge.cadence.moe/download/matrix/snowball.kathund.dev/kDSCaRYXXrXpfZQiifZDnhNW) for only Message Content and Server Members was successful. You may wish to take notes from it when writing your own application, though I suggest _not_ copying it word for word.
I recorded a video demonstrating the Guild Members intent, you can use this: https://youtu.be/i9_sx0fYQWs
I recorded a video demonstrating the Guild Members intent, you can use this: <https://youtu.be/i9_sx0fYQWs>
I also recorded a video demonstrating the Message Content intent, you can use this too: https://youtu.be/LsPho8gtGRw
I also recorded a video demonstrating the Message Content intent, you can use this too: <https://youtu.be/LsPho8gtGRw>
## Difference between privileged intents and verification

View file

@ -1,3 +1,9 @@
---
title: Reverse proxy with Traefik
authors:
- "@bgtlover:stealthy.club"
---
> This guide was written by @bgtlover:stealthy.club, a community contributor. The author of Out Of Your Element hopes it will be useful, but cannot say whether the information is accurate or complete.
## Example reverse proxy configuration with traefik

View file

@ -0,0 +1,61 @@
# Commands
## For Discord users
### `/invite`
Invite a Matrix user to this Discord server.
On Matrix, people can choose which channels they want to join. If you use the command a second time, you can additionally invite them to the specific channel.
### `/ping`
Notify a Matrix user.
You can do this without the command just by typing `@their username`. It won't look like anything to you, but it works on the other side! You only need to use `/ping` if you really want to be sure.
### Right click -> `Matrix info`
See details about who sent a message.
### Right click -> `Responses`
See the specific users who reacted to a message or voted on a poll.
## For Discord moderators
### `/privacy`
Set how public or private the Matrix rooms are.
### Right click -> `Permissions`
Give Matrix users additional permissions:
- `Default`
- `Moderator` - can delete messages, kick or ban
- `Admin` - can moderate moderators and break the room
## For Matrix users
### `/thread`
Create a new bridged Discord thread. You must write a `name` for the thread after the command.
This is necessary since standard Matrix threads are not bridged as Discord threads, they appear as reply chains on Discord instead.
### `/invite`
Create a Discord invite link. You can use this to invite people who only use Discord.
### `/time`
Similar to Discord's @time feature. Set your time zone in your client, then write a `human-readable time` after the command, and it'll be converted to everyone else's local time.
## For Matrix moderators
### `/emoji`
For clients that don't support editing emojis natively, add a new emoji to the bridged Discord server.
Use this while replying to an image that you want to add as an emoji. You can set a custom name by writing `:name:` after the command.

12
docs/index.md Normal file
View file

@ -0,0 +1,12 @@
# Overview
Useful links:
- [Use a public instance](https://ooye.org/)
- [Source code](https://gitdab.com/cadence/out-of-your-element)
- [Discuss on Matrix](https://matrix.to/#/%23out-of-your-element:cadence.moe)
Documentation:
- [Install](./for-hosters/install.md)
- Browse the rest of the documentation in the sidebar

12
mkdocs.yml Normal file
View file

@ -0,0 +1,12 @@
site_name: Docs for Out Of Your Element
site_url: https://docs.ooye.org
theme:
name: material
markdown_extensions:
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences

View file

@ -36,7 +36,7 @@ I've also added some interesting features that I haven't seen in any other bridg
* Matrix custom emojis from private rooms are still visible on Discord as a sprite sheet
* To save space, larger files from Discord are linked instead of reuploaded to Matrix (links don't expire)
For more information about features, [see the user guide.](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/user-guide.md)
For more information about features, [see the user guide.](https://docs.ooye.org/for-users/user-guide/)
## Caveats
@ -46,4 +46,4 @@ For more information about features, [see the user guide.](https://gitdab.com/ca
## Get started!
[Read the installation instructions →](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md)
[Read the installation instructions →](https://docs.ooye.org/for-hosters/install/)

View file

@ -138,9 +138,10 @@ async function channelToKState(channel, guild, di) {
const globalAdmins = select("member_power", ["mxid", "power_level"], {room_id: "*"}).all()
const globalAdminPower = globalAdmins.reduce((a, c) => (a[c.mxid] = c.power_level, a), {})
const additionalCreators = select("member_power", "mxid", {room_id: "*"}, "AND power_level > 100").pluck().all().concat(spaceCreatorsAndFounders)
const additionalCreatorsList = select("member_power", "mxid", {room_id: "*"}, "AND power_level > 100").pluck().all().concat(spaceCreatorsAndFounders)
const additionalCreatorsUnique = [...new Set(additionalCreatorsList)].filter(mxid => mxid !== mUtils.bot)
const creationContent = {}
creationContent.additional_creators = additionalCreators
creationContent.additional_creators = additionalCreatorsUnique
if (channel.type === DiscordTypes.ChannelType.GuildForum) creationContent.type = "m.space"

View file

@ -20,6 +20,8 @@ const createRoom = sync.require("./create-room")
const expression = sync.require("./expression")
/** @type {import("../../matrix/kstate")} */
const ks = sync.require("../../matrix/kstate")
/** @type {import("../../matrix/utils")} */
const mxUtils = sync.require("../../matrix/utils")
/** @type {Map<string, Promise<string>>} guild ID -> Promise<space ID> */
const inflightSpaceCreate = sync.remember(() => new Map())
@ -67,12 +69,13 @@ async function createSpace(guild, kstate) {
async function guildToKState(guild, privacyLevel) {
assert.equal(typeof privacyLevel, "number")
const globalAdmins = select("member_power", ["mxid", "power_level"], {room_id: "*"}).all()
const additionalCreators = select("member_power", "mxid", {room_id: "*"}, "AND power_level > 100").pluck().all()
const additionalCreatorsList = select("member_power", "mxid", {room_id: "*"}, "AND power_level > 100").pluck().all()
const additionalCreatorsUnique = [...new Set(additionalCreatorsList)].filter(mxid => mxid !== mxUtils.bot)
const guildKState = {
"m.room.create/": {
type: "m.space",
additional_creators: additionalCreators
additional_creators: additionalCreatorsUnique
},
"m.room.name/": {name: guild.name},
"m.room.avatar/": {

View file

@ -106,6 +106,7 @@ async function ensureSimJoined(pkMessage, roomID) {
* Generate profile data based on webhook displayname and configured avatar.
* @param {Ty.PkMessage} pkMessage
* @param {Ty.WebhookAuthor} author
* @returns {Promise<Ty.Event.M_Room_Member>}
*/
async function memberToStateContent(pkMessage, author) {
// We prefer to use the member's avatar URL data since the image upload can be cached across channels,

View file

@ -111,6 +111,7 @@ async function ensureSimJoined(pluRalMessage, roomID) {
* Generate profile data based on webhook displayname and configured avatar.
* @param {Ty.PluRalWebhookMessage} pluRalMessage
* @param {Ty.WebhookAuthor} author
* @returns {Promise<Ty.Event.M_Room_Member>}
*/
async function memberToStateContent(pluRalMessage, author) {
// We prefer to use the member's avatar URL data since the image upload can be cached across channels,

View file

@ -100,8 +100,8 @@ async function userToGlobalProfile(user) {
globalProfile.displayname = user.username
if (user.global_name) globalProfile.displayname = user.global_name
if (user.avatar) {
const avatarPath = file.userAvatar(user) // the user avatar only
if (avatarPath) {
globalProfile.avatar_url = await file.uploadDiscordFileToMxc(avatarPath)
}
@ -120,6 +120,7 @@ async function userToGlobalProfile(user) {
* @param {DiscordTypes.APIUser} user
* @param {Omit<DiscordTypes.APIGuildMember, "user"> | undefined} member
* @param {DiscordTypes.APIGuild} guild
* @returns {Promise<Ty.Event.M_Room_Member & {"eu.she-a.color"?: {on_light: string, on_dark: string}}>}
*/
async function memberToStateContent(user, member, guild) {
let displayname = user.username
@ -139,9 +140,9 @@ async function memberToStateContent(user, member, guild) {
}
}
if (member?.avatar || user.avatar) {
// const avatarPath = file.userAvatar(user) // the user avatar only
const avatarPath = file.memberAvatar(guild.id, user, member) // the member avatar or the user avatar
if (avatarPath) {
content["moe.cadence.ooye.member"].avatar = avatarPath
content.avatar_url = await file.uploadDiscordFileToMxc(avatarPath)
}
@ -255,7 +256,7 @@ async function syncUser(user, member, channel, guild, roomID, interactionMetadat
* @param {string} roomID
* @param {string} mxid
* @param {DiscordTypes.APIUser | null} user
* @param {{displayname: string, avatar_url?: string, "eu.she-a.color"?: {on_light: string, on_dark: string}}} content
* @param {Awaited<ReturnType<memberToStateContent>>} content
* @param {number | null} powerLevel
* @param {{allowOverwrite?: boolean, updateGlobalProfile?: boolean}} [options]
*/

View file

@ -87,6 +87,7 @@ async function ensureSimJoined(fakeUserID, author, roomID) {
/**
* Generate profile data based on webhook displayname and configured avatar.
* @param {Ty.WebhookAuthor} author
* @returns {Promise<Ty.Event.M_Room_Member>}
*/
async function authorToStateContent(author) {
const avatar = file.userAvatar(author)

View file

@ -1,16 +1,28 @@
// @ts-check
const {discord, db, select} = require("../../passthrough")
const {discord, db, select, sync} = require("../../passthrough")
const guildNicknamedCache = sync.remember(() => new Set())
/** @param {string} guildID */
function markGuildNicknamed(guildID) {
db.prepare("UPDATE guild_space SET nicknamed = 1 WHERE guild_id = ?").run(guildID)
guildNicknamedCache.add(guildID)
}
/** @param {string} guildID */
async function setNicknameFromGuildID(guildID) {
// Check if nickname is already set via cache
if (guildNicknamedCache.has(guildID)) return
// Check if nickname is already set via DB (and update cache if so)
const nicknamed = select("guild_space", "nicknamed", {guild_id: guildID}).pluck().get()
if (nicknamed !== 0) return
if (nicknamed !== 0) {
guildNicknamedCache.add(guildID)
return
}
// Hasn't been done yet. Apply nickname now
try {
await discord.snow.guild.editGuildMember(guildID, "@me", {nick: "a Matrix user"})
markGuildNicknamed(guildID)

View file

@ -1,5 +1,6 @@
// @ts-check
const DiscordTypes = require("discord-api-types/v10")
const passthrough = require("../passthrough")
const {reg, writeRegistration} = require("./read-registration.js")
const Ty = require("../types")
@ -138,14 +139,30 @@ async function _actuallyUploadDiscordFileToMxc(url) {
}
}
/**
* @param {DiscordTypes.APIGuild} guild
* @returns {string | null}
*/
function guildIcon(guild) {
if (!guild.icon) return null
return `/icons/${guild.id}/${guild.icon}.png?size=${IMAGE_SIZE}`
}
/**
* @param {DiscordTypes.APIUser | Ty.WebhookAuthor} user
* @returns {string | null}
*/
function userAvatar(user) {
if (!user.avatar) return null
return `/avatars/${user.id}/${user.avatar}.png?size=${IMAGE_SIZE}`
}
/**
* @param {string} guildID
* @param {DiscordTypes.APIUser} user
* @param {Omit<DiscordTypes.APIGuildMember, "user"> | undefined} member
* @returns {string}
*/
function memberAvatar(guildID, user, member) {
if (!member?.avatar) return userAvatar(user)
return `/guilds/${guildID}/users/${user.id}/avatars/${member?.avatar}.png?size=${IMAGE_SIZE}`

View file

@ -26,3 +26,33 @@ test("removeExpiryParams: works on media proxy and keeps quality", t => {
const result = file._removeExpiryParams(url)
t.equal(result, "https://media.discordapp.net/attachments/1160894080998461480/1548665030898094240/855064e9-bdae-487a-a497-c662bd510487.png?format=webp")
})
test("guildIcon: returns icon path", t => {
t.equal(
// @ts-ignore
file.guildIcon({id: "112760669178241024", icon: "a_f83622e09ead74f0c5c527fe241f8f8c"}),
"/icons/112760669178241024/a_f83622e09ead74f0c5c527fe241f8f8c.png?size=1024"
)
})
test("guildIcon: returns null if guild has no icon", t => {
t.equal(
// @ts-ignore
file.guildIcon({id: "112760669178241024", icon: null}),
null
)
})
test("userAvatar: returns avatar path", t => {
t.equal(
file.userAvatar({id: "113340068197859328", username: "kumaccino", avatar: "b48302623a12bc7c59a71328f72ccb39"}),
"/avatars/113340068197859328/b48302623a12bc7c59a71328f72ccb39.png?size=1024"
)
})
test("userAvatar: returns null if user has no avatar", t => {
t.equal(
file.userAvatar({id: "113340068197859328", username: "kumaccino", avatar: null}),
null
)
})

View file

@ -32,7 +32,11 @@ async function kstateUploadMxc(obj) {
for (const [k, v] of Object.entries(obj)) {
if (v == null || typeof v !== "object") continue
if (v.$url) {
if ("$url" in v) {
if (!v.$url) {
delete obj[k]
continue
}
promises.push(
file.uploadDiscordFileToMxc(v.$url)
.then(mxc => obj[k] = mxc)

View file

@ -21,6 +21,18 @@ test("kstate strip: keeps true conditions while removing $if", t => {
})
})
test("kstateUploadMxc: removes keys where $url is null", async t => {
const input = {
"m.room.avatar/": {
url: {$url: null}
}
}
await kstateUploadMxc(input)
t.deepEqual(input, {
"m.room.avatar/": {}
})
})
test("kstateUploadMxc: sets the mxc", async t => {
const input = {
"m.room.avatar/": {

View file

@ -22,6 +22,8 @@ function getStats() {
["hour", 1 * 60 * 60 * 1000]
]
const oldestMessageID = db.prepare("select min(message_id) from event_message where source = 0").pluck().get()
// console.time("get stats")
let temp = {
guilds: db.prepare("select count(*) from guild_space").pluck().get(),
@ -29,7 +31,7 @@ function getStats() {
messages: db.prepare("select count(*) from message_room").pluck().get(),
...durations.reduce((a, c) => (a[`messages_last_${c[0]}`] = getMessageCountLastDuration(c[1]), a), {}),
message_sources: db.prepare("select count(*) from event_message where part = 0 group by source order by source").pluck().all(),
oldest_message: new Date(dUtils.snowflakeToTimestampExact(db.prepare("select min(message_id) from event_message where source = 0").pluck().get())), // good until 2090
oldest_message: new Date(oldestMessageID ? dUtils.snowflakeToTimestampExact(oldestMessageID) : 0), // good until 2090
discord_users: db.prepare("select count(*) from sim").pluck().get(),
matrix_users: db.prepare("select count(distinct mxid) from member_cache where mxid not like ?").pluck().get(reg.namespaces.users[0].regex.replace(/\.\*.*/, "%")),
}