Document encryption as unsupported

- Autojoined child spaces are recorded as invited
- Update entry when reinvited
- Delete entry when uninvited or removed from room
- Allow linking with spaces you moderate, even if you didn't invite
- Store power levels immediately for new invited rooms
  - Mark members as missing profile in this case
- Only delete from invite table if it left the space

.gitignore

@@ -3,6 +3,8 @@ config.js
 registration.yaml
 ooye.db*
 events.db*
+backfill.db*
+custom-webroot
 
 # Automatically generated
 node_modules

docs/developer-orientation.md

@@ -89,7 +89,7 @@ Whether you read those or not, I'm more than happy to help you 1-on-1 with codin
 
 # Dependency justification
 
-Total transitive production dependencies: 137
+Total transitive production dependencies: 134
 
 ### <font size="+2">🦕</font>
 
@@ -119,8 +119,8 @@ Total transitive production dependencies: 137
 * (0) entities: Looks fine. No dependencies.
 * (0) get-relative-path: Looks fine. No dependencies.
 * (1) heatsync: Module hot-reloader that I trust.
-* (1) js-yaml: Will be removed in the future after registration.yaml is converted to JSON.
 * (0) lru-cache: For holding unused nonce in memory and letting them be overwritten later if never used.
+* (0) mime-type: File extension to mime type mapping that's already pulled in by stream-mime-type.
 * (0) prettier-bytes: It does what I want and has no dependencies.
 * (0) snowtransfer: Discord API library with bring-your-own-caching that I trust.
 * (0) try-to-catch: Not strictly necessary, but it's already pulled in by supertape, so I may as well.

docs/docker.md

@@ -0,0 +1,76 @@
+# Docker policy
+
+**Out Of Your Element has no official support for Docker. There are no official files or images. If you choose to run Out Of Your Element in Docker, you must disclose this when asking for support. I may refuse to provide support/advice at any time. I may refuse to acknowledge issue reports.**
+
+This also goes for Podman, Nix, and other similar technology that upends a program's understanding of what it's running on.
+
+## What I recommend
+
+I recommend [following the official setup guide,](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md) which does not use Docker.
+
+Ultimately, though, do what makes you happy. I can't stop you from doing what you want. As long as you read this page and understand my perspective, that's good enough for me.
+
+## Why I advise against Docker
+
+When misconfigured, Docker has terrible impacts. It can cause messages to go missing or even permanent data loss. These have happened to people.
+
+Docker also makes it much harder for me to advise on debugging because it puts barriers between you and useful debugging tools, such as stdin, the database file, a shell, and the inspector. It's also not clear which version of the source code is running in the container, as there are many pieces of Docker (builder, container, image) that can cache old data, often making it so you didn't actually update when you thought you did. This has happened to people.
+
+## Why I don't provide a good configuration myself
+
+It is not possible for Docker to be correctly configured by default. The defaults are broken and will cause data loss.
+
+It is also not possible for me to provide a correct configuration for everyone. Even if I provided a correct image, the YAMLs and command-line arguments must be written by individual end users. Incorrect YAMLs and command-line arguments may cause connection issues or permanent data loss.
+
+## Why I don't provide assistance if you run OOYE in Docker
+
+Problems you encounter, especially with the initial setup, are much more likely to be caused by nuances in your Docker setup than problems in my code. Therefore, my code is not responsible for the problem. The cause of the problem is different code that I can't advise on.
+
+Also, if you reported an issue and I asked for additional information to help find the cause, you might be unable to provide it because of the debugging barriers discussed above.
+
+## Why I don't provide Docker resources
+
+I create OOYE unpaid in my spare time because I enjoy the process. I find great enjoyment in creating code and none at all in creating infrastructure.
+
+## Why you're probably fine without Docker
+
+### If you care about system footprint
+
+OOYE was designed to be simple and courteous:
+
+* It only creates files in its working directory
+* It does not require any other processes to be running (e.g., no dependency on a Postgres process)
+* It only requires node/npm executables in PATH, which you can store in any folder if you don't want to use your package manager
+
+### If you care about ease of setup
+
+In my opinion, the [official setup process](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md) is straightforward. After installing prerequisites (Node.js and the repo clone), the rest of the process interactively guides you through providing necessary information. Your input is checked for correctness so the bridge will definitely work when you run it.
+
+I find this easier than the usual Docker workflow of pasting values into a YAML and rolling the dice on whether it will start up or not.
+
+### If you care about security in the case of compromise/RCE
+
+There are no known vulnerabilities in dependencies. I [carefully selected simple, light dependencies](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/developer-orientation.md#dependency-justification) to reduce attack surface area.
+
+For defense in depth, I suggest running OOYE as a different user.
+
+### If you want to see all the processes when you run docker ps
+
+Well, you got me there.
+
+## Unofficial, independent, community-provided container setups
+
+I acknowledge the demand for using OOYE in a container, so I will still point you in the right direction.
+
+I had no hand in creating these and have not used or tested them whatsoever. I make no assurance that these will work reliably, or even at all. If you use these, you must do so with the understanding that if you run into any problems, **you must ask for support from the author of that setup, not from me, because you're running their code, not mine.**
+
+***The following list is distributed for your information, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.***
+
+- by melody: https://git.shork.ch/docker-images/out-of-your-element
+- by sim2kid: https://github.com/sim2kid/ooye-docker
+- by Katharos Technology: https://github.com/katharostech/docker_ooye
+- by Emma: https://cgit.rory.gay/nix/OOYE-module.git/tree
+
+## Making your own Docker setup
+
+If you decide to make your own, I may provide advice or indicate problems at my discretion. You acknowledge that I am not required to provide evidence of problems I indicate, nor solutions to them. You acknowledge that it is not possible for me to exhaustively indicate every problem, so I cannot indicate correctness. Even if I have provided advice to an unofficial, independent, community-provided setup, I do not endorse it.

docs/get-started.md

@@ -1,5 +1,7 @@
 # Setup
 
+If you want Docker, [please read this first.](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/docker.md)
+
 If you get stuck, you're welcome to message [#out-of-your-element:cadence.moe](https://matrix.to/#/#out-of-your-element:cadence.moe) or [@cadence:cadence.moe](https://matrix.to/#/@cadence:cadence.moe) to ask for help setting up OOYE!
 
 You'll need:

jsconfig.json

@@ -2,6 +2,7 @@
 	"compilerOptions": {
 		"target": "es2024",
 		"module": "nodenext",
+		"lib": ["ESNext"],
 		"strict": true,
 		"noImplicitAny": false,
 		"useUnknownInCatchVariables": false

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "out-of-your-element",
-  "version": "3.2.0",
+  "version": "3.4.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "out-of-your-element",
-      "version": "3.2.0",
+      "version": "3.4.0",
       "license": "AGPL-3.0-or-later",
       "dependencies": {
         "@chriscdn/promise-semaphore": "^3.0.1",
@@ -24,7 +24,7 @@
         "better-sqlite3": "^12.2.0",
         "chunk-text": "^2.0.1",
         "cloudstorm": "^0.15.2",
-        "discord-api-types": "^0.38.36",
+        "discord-api-types": "^0.38.38",
         "domino": "^2.1.6",
         "enquirer": "^2.4.1",
         "entities": "^5.0.0",
@@ -33,6 +33,7 @@
         "heatsync": "^2.7.2",
         "htmx.org": "^2.0.4",
         "lru-cache": "^11.0.2",
+        "mime-types": "^2.1.35",
         "prettier-bytes": "^1.0.4",
         "sharp": "^0.34.5",
         "snowtransfer": "^0.17.1",
@@ -50,7 +51,7 @@
         "supertape": "^12.0.12"
       },
       "engines": {
-        "node": ">=20"
+        "node": ">=22"
       }
     },
     "../extended-errors/enhance-errors": {
@@ -764,101 +765,14 @@
         "url": "https://opencollective.com/libvips"
       }
     },
-    "node_modules/@isaacs/balanced-match": {
-      "version": "4.0.1",
-      "resolved": "https://registry.npmjs.org/@isaacs/balanced-match/-/balanced-match-4.0.1.tgz",
-      "integrity": "sha512-yzMTt9lEb8Gv7zRioUilSglI0c0smZ9k5D65677DLWLtWJaXIS3CqcGyUFByYKlnUj6TkjLVs54fBl6+TiGQDQ==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": "20 || >=22"
-      }
-    },
-    "node_modules/@isaacs/brace-expansion": {
-      "version": "5.0.0",
-      "resolved": "https://registry.npmjs.org/@isaacs/brace-expansion/-/brace-expansion-5.0.0.tgz",
-      "integrity": "sha512-ZT55BDLV0yv0RBm2czMiZ+SqCGO7AvmOM3G/w2xhVPH+te0aKgFjmBvGlL1dH+ql2tgGO3MVrbb3jCKyvpgnxA==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "@isaacs/balanced-match": "^4.0.1"
-      },
-      "engines": {
-        "node": "20 || >=22"
-      }
-    },
     "node_modules/@isaacs/cliui": {
-      "version": "8.0.2",
-      "resolved": "https://registry.npmjs.org/@isaacs/cliui/-/cliui-8.0.2.tgz",
-      "integrity": "sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==",
+      "version": "9.0.0",
+      "resolved": "https://registry.npmjs.org/@isaacs/cliui/-/cliui-9.0.0.tgz",
+      "integrity": "sha512-AokJm4tuBHillT+FpMtxQ60n8ObyXBatq7jD2/JA9dxbDDokKQm8KMht5ibGzLVU9IJDIKK4TPKgMHEYMn3lMg==",
       "dev": true,
-      "license": "ISC",
-      "dependencies": {
-        "string-width": "^5.1.2",
-        "string-width-cjs": "npm:string-width@^4.2.0",
-        "strip-ansi": "^7.0.1",
-        "strip-ansi-cjs": "npm:strip-ansi@^6.0.1",
-        "wrap-ansi": "^8.1.0",
-        "wrap-ansi-cjs": "npm:wrap-ansi@^7.0.0"
-      },
+      "license": "BlueOak-1.0.0",
       "engines": {
-        "node": ">=12"
-      }
-    },
-    "node_modules/@isaacs/cliui/node_modules/ansi-styles": {
-      "version": "6.2.3",
-      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-6.2.3.tgz",
-      "integrity": "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">=12"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/ansi-styles?sponsor=1"
-      }
-    },
-    "node_modules/@isaacs/cliui/node_modules/emoji-regex": {
-      "version": "9.2.2",
-      "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-9.2.2.tgz",
-      "integrity": "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg==",
-      "dev": true,
-      "license": "MIT"
-    },
-    "node_modules/@isaacs/cliui/node_modules/string-width": {
-      "version": "5.1.2",
-      "resolved": "https://registry.npmjs.org/string-width/-/string-width-5.1.2.tgz",
-      "integrity": "sha512-HnLOCR3vjcY8beoNLtcjZ5/nxn2afmME6lhrDrebokqMap+XbeW8n9TXpPDOqdGK5qcI3oT0GKTW6wC7EMiVqA==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "eastasianwidth": "^0.2.0",
-        "emoji-regex": "^9.2.2",
-        "strip-ansi": "^7.0.1"
-      },
-      "engines": {
-        "node": ">=12"
-      },
-      "funding": {
-        "url": "https://github.com/sponsors/sindresorhus"
-      }
-    },
-    "node_modules/@isaacs/cliui/node_modules/wrap-ansi": {
-      "version": "8.1.0",
-      "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-8.1.0.tgz",
-      "integrity": "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-styles": "^6.1.0",
-        "string-width": "^5.0.1",
-        "strip-ansi": "^7.0.1"
-      },
-      "engines": {
-        "node": ">=12"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
+        "node": ">=18"
       }
     },
     "node_modules/@istanbuljs/schema": {
@@ -1171,19 +1085,6 @@
         "node": ">=6"
       }
     },
-    "node_modules/ansi-regex": {
-      "version": "6.2.2",
-      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-6.2.2.tgz",
-      "integrity": "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">=12"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/ansi-regex?sponsor=1"
-      }
-    },
     "node_modules/ansi-styles": {
       "version": "4.3.0",
       "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
@@ -1616,9 +1517,9 @@
       }
     },
     "node_modules/discord-api-types": {
-      "version": "0.38.37",
-      "resolved": "https://registry.npmjs.org/discord-api-types/-/discord-api-types-0.38.37.tgz",
-      "integrity": "sha512-Cv47jzY1jkGkh5sv0bfHYqGgKOWO1peOrGMkDFM4UmaGMOTgOW8QSexhvixa9sVOiz8MnVOBryWYyw/CEVhj7w==",
+      "version": "0.38.38",
+      "resolved": "https://registry.npmjs.org/discord-api-types/-/discord-api-types-0.38.38.tgz",
+      "integrity": "sha512-7qcM5IeZrfb+LXW07HvoI5L+j4PQeMZXEkSm1htHAHh4Y9JSMXBWjy/r7zmUCOj4F7zNjMcm7IMWr131MT2h0Q==",
       "license": "MIT",
       "workspaces": [
         "scripts/actions/documentation"
@@ -1634,13 +1535,6 @@
       "resolved": "https://registry.npmjs.org/domino/-/domino-2.1.6.tgz",
       "integrity": "sha512-3VdM/SXBZX2omc9JF9nOPCtDaYQ67BGp5CoLpIQlO2KCAPETs8TcDHacF26jXadGbvUteZzRTeos2fhID5+ucQ=="
     },
-    "node_modules/eastasianwidth": {
-      "version": "0.2.0",
-      "resolved": "https://registry.npmjs.org/eastasianwidth/-/eastasianwidth-0.2.0.tgz",
-      "integrity": "sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==",
-      "dev": true,
-      "license": "MIT"
-    },
     "node_modules/emoji-regex": {
       "version": "8.0.0",
       "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
@@ -1860,14 +1754,40 @@
         "url": "https://github.com/sponsors/isaacs"
       }
     },
+    "node_modules/glob/node_modules/balanced-match": {
+      "version": "4.0.2",
+      "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.2.tgz",
+      "integrity": "sha512-x0K50QvKQ97fdEz2kPehIerj+YTeptKF9hyYkKf6egnwmMWAkADiO0QCzSp0R5xN8FTZgYaBfSaue46Ej62nMg==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "jackspeak": "^4.2.3"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      }
+    },
+    "node_modules/glob/node_modules/brace-expansion": {
+      "version": "5.0.2",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.2.tgz",
+      "integrity": "sha512-Pdk8c9poy+YhOgVWw1JNN22/HcivgKWwpxKq04M/jTmHyCZn12WPJebZxdjSa5TmBqISrUSgNYU3eRORljfCCw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "balanced-match": "^4.0.2"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      }
+    },
     "node_modules/glob/node_modules/minimatch": {
-      "version": "10.1.1",
-      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.1.1.tgz",
-      "integrity": "sha512-enIvLvRAFZYXJzkCYG5RKmPfrFArdLv+R+lbQ53BmIMLIry74bjKzX6iHAm8WYamJkhSSEabrWN5D97XnKObjQ==",
+      "version": "10.2.0",
+      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.0.tgz",
+      "integrity": "sha512-ugkC31VaVg9cF0DFVoADH12k6061zNZkZON+aX8AWsR9GhPcErkcMBceb6znR8wLERM2AkkOxy2nWRLpT9Jq5w==",
       "dev": true,
       "license": "BlueOak-1.0.0",
       "dependencies": {
-        "@isaacs/brace-expansion": "^5.0.0"
+        "brace-expansion": "^5.0.2"
       },
       "engines": {
         "node": "20 || >=22"
@@ -2054,13 +1974,13 @@
       }
     },
     "node_modules/jackspeak": {
-      "version": "4.1.1",
-      "resolved": "https://registry.npmjs.org/jackspeak/-/jackspeak-4.1.1.tgz",
-      "integrity": "sha512-zptv57P3GpL+O0I7VdMJNBZCu+BPHVQUk55Ft8/QCJjTVxrnJHuVuX/0Bl2A6/+2oyR/ZMEuFKwmzqqZ/U5nPQ==",
+      "version": "4.2.3",
+      "resolved": "https://registry.npmjs.org/jackspeak/-/jackspeak-4.2.3.tgz",
+      "integrity": "sha512-ykkVRwrYvFm1nb2AJfKKYPr0emF6IiXDYUaFx4Zn9ZuIH7MrzEZ3sD5RlqGXNRpHtvUHJyOnCEFxOlNDtGo7wg==",
       "dev": true,
       "license": "BlueOak-1.0.0",
       "dependencies": {
-        "@isaacs/cliui": "^8.0.2"
+        "@isaacs/cliui": "^9.0.0"
       },
       "engines": {
         "node": "20 || >=22"
@@ -2154,6 +2074,7 @@
       "version": "2.1.35",
       "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
       "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+      "license": "MIT",
       "dependencies": {
         "mime-db": "1.52.0"
       },
@@ -2813,45 +2734,6 @@
         "node": ">=8"
       }
     },
-    "node_modules/string-width-cjs": {
-      "name": "string-width",
-      "version": "4.2.3",
-      "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
-      "integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "emoji-regex": "^8.0.0",
-        "is-fullwidth-code-point": "^3.0.0",
-        "strip-ansi": "^6.0.1"
-      },
-      "engines": {
-        "node": ">=8"
-      }
-    },
-    "node_modules/string-width-cjs/node_modules/ansi-regex": {
-      "version": "5.0.1",
-      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
-      "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">=8"
-      }
-    },
-    "node_modules/string-width-cjs/node_modules/strip-ansi": {
-      "version": "6.0.1",
-      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
-      "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-regex": "^5.0.1"
-      },
-      "engines": {
-        "node": ">=8"
-      }
-    },
     "node_modules/string-width/node_modules/ansi-regex": {
       "version": "5.0.1",
       "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
@@ -2873,46 +2755,6 @@
         "node": ">=8"
       }
     },
-    "node_modules/strip-ansi": {
-      "version": "7.1.2",
-      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-7.1.2.tgz",
-      "integrity": "sha512-gmBGslpoQJtgnMAvOVqGZpEz9dyoKTCzy2nfz/n8aIFhN/jCE/rCmcxabB6jOOHV+0WNnylOxaxBQPSvcWklhA==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-regex": "^6.0.1"
-      },
-      "engines": {
-        "node": ">=12"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/strip-ansi?sponsor=1"
-      }
-    },
-    "node_modules/strip-ansi-cjs": {
-      "name": "strip-ansi",
-      "version": "6.0.1",
-      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
-      "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-regex": "^5.0.1"
-      },
-      "engines": {
-        "node": ">=8"
-      }
-    },
-    "node_modules/strip-ansi-cjs/node_modules/ansi-regex": {
-      "version": "5.0.1",
-      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
-      "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">=8"
-      }
-    },
     "node_modules/strip-json-comments": {
       "version": "2.0.1",
       "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
@@ -3257,48 +3099,6 @@
         "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
       }
     },
-    "node_modules/wrap-ansi-cjs": {
-      "name": "wrap-ansi",
-      "version": "7.0.0",
-      "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
-      "integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-styles": "^4.0.0",
-        "string-width": "^4.1.0",
-        "strip-ansi": "^6.0.0"
-      },
-      "engines": {
-        "node": ">=10"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
-      }
-    },
-    "node_modules/wrap-ansi-cjs/node_modules/ansi-regex": {
-      "version": "5.0.1",
-      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
-      "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">=8"
-      }
-    },
-    "node_modules/wrap-ansi-cjs/node_modules/strip-ansi": {
-      "version": "6.0.1",
-      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
-      "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
-      "dev": true,
-      "license": "MIT",
-      "dependencies": {
-        "ansi-regex": "^5.0.1"
-      },
-      "engines": {
-        "node": ">=8"
-      }
-    },
     "node_modules/wrap-ansi/node_modules/ansi-regex": {
       "version": "5.0.1",
       "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "out-of-your-element",
-  "version": "3.2.0",
+  "version": "3.4.0",
   "description": "A bridge between Matrix and Discord",
   "main": "index.js",
   "repository": {
@@ -12,10 +12,10 @@
     "discord",
     "bridge"
   ],
-  "author": "Cadence, PapiOphidian",
+  "author": "Cadence",
   "license": "AGPL-3.0-or-later",
   "engines": {
-    "node": ">=20"
+    "node": ">=22"
   },
   "dependencies": {
     "@chriscdn/promise-semaphore": "^3.0.1",
@@ -33,7 +33,7 @@
     "better-sqlite3": "^12.2.0",
     "chunk-text": "^2.0.1",
     "cloudstorm": "^0.15.2",
-    "discord-api-types": "^0.38.36",
+    "discord-api-types": "^0.38.38",
     "domino": "^2.1.6",
     "enquirer": "^2.4.1",
     "entities": "^5.0.0",
@@ -42,6 +42,7 @@
     "heatsync": "^2.7.2",
     "htmx.org": "^2.0.4",
     "lru-cache": "^11.0.2",
+    "mime-types": "^2.1.35",
     "prettier-bytes": "^1.0.4",
     "sharp": "^0.34.5",
     "snowtransfer": "^0.17.1",
@@ -66,7 +67,6 @@
     "setup": "node --enable-source-maps scripts/setup.js",
     "addbot": "node addbot.js",
     "test": "cross-env FORCE_COLOR=true supertape --no-check-assertions-count --format tap --no-worker test/test.js | tap-dot",
-    "test-slow": "cross-env FORCE_COLOR=true supertape --no-check-assertions-count --format tap --no-worker test/test.js -- --slow | tap-dot",
     "cover": "c8 -o test/coverage --skip-full -x db/migrations -x src/m2d/event-dispatcher.js -x src/matrix/file.js -x src/matrix/api.js -x src/d2m/converters/rlottie-wasm.js -r html -r text supertape --no-check-assertions-count --format fail --no-worker test/test.js -- --slow"
   }
 }

readme.md

@@ -38,6 +38,7 @@ For more information about features, [see the user guide.](https://gitdab.com/ca
 
 * This bridge is not designed for puppetting.
 * Direct Messaging is not supported until I figure out a good way of doing it.
+* Encrypted messages are not supported. Decryption is often unreliable on Matrix, and your messages end up in plaintext on Discord anyway, so there's not much advantage.
 
 ## Get started!
 

scripts/backfill.js

@@ -29,6 +29,9 @@ const DiscordClient = require("../src/d2m/discord-client")
 const discord = new DiscordClient(reg.ooye.discord_token, "half")
 passthrough.discord = discord
 
+const {as} = require("../src/matrix/appservice")
+passthrough.as = as
+
 const orm = sync.require("../src/db/orm")
 passthrough.from = orm.from
 passthrough.select = orm.select
@@ -69,7 +72,7 @@ async function event(event) {
 				backfill: true,
 				...message
 			}
-			await eventDispatcher.onMessageCreate(discord, simulatedGatewayDispatchData)
+			await eventDispatcher.MESSAGE_CREATE(discord, simulatedGatewayDispatchData)
 			preparedInsert.run(channelID, message.id)
 		}
 		last = messages.at(-1)?.id

scripts/estimate-size.js

@@ -0,0 +1,65 @@
+// @ts-check
+
+const pb = require("prettier-bytes")
+const sqlite = require("better-sqlite3")
+const HeatSync = require("heatsync")
+
+const {reg} = require("../src/matrix/read-registration")
+const passthrough = require("../src/passthrough")
+
+const sync = new HeatSync({watchFS: false})
+Object.assign(passthrough, {reg, sync})
+
+const DiscordClient = require("../src/d2m/discord-client")
+
+const discord = new DiscordClient(reg.ooye.discord_token, "no")
+passthrough.discord = discord
+
+const db = new sqlite("ooye.db")
+passthrough.db = db
+
+const api = require("../src/matrix/api")
+
+const {room: roomID} = require("minimist")(process.argv.slice(2), {string: ["room"]})
+if (!roomID) {
+	console.error("Usage: ./scripts/estimate-size.js --room=<!room id here>")
+	process.exit(1)
+}
+
+const {channel_id, guild_id} = db.prepare("SELECT channel_id, guild_id FROM channel_room WHERE room_id = ?").get(roomID)
+
+const max = 1000
+
+;(async () => {
+	let total = 0
+	let size = 0
+	let from
+
+	while (total < max) {
+		const events = await api.getEvents(roomID, "b", {limit: 1000, from})
+		total += events.chunk.length
+		from = events.end
+		console.log(`Fetched ${total} events so far`)
+
+		for (const e of events.chunk) {
+			if (e.content?.info?.size) {
+				size += e.content.info.size
+			}
+		}
+
+		if (events.chunk.length === 0 || !events.end) break
+	}
+
+	console.log(`Total size of uploads: ${pb(size)}`)
+
+	const searchResults = await discord.snow.requestHandler.request(`/guilds/${guild_id}/messages/search`, {
+		channel_id,
+		offset: "0",
+		limit: "1"
+	}, "get", "json")
+
+	const totalAllTime = searchResults.total_results
+	const fractionCounted = total / totalAllTime
+	console.log(`That counts for ${(fractionCounted*100).toFixed(2)}% of the history on Discord (${totalAllTime.toLocaleString()} messages)`)
+	console.log(`The size of uploads for the whole history would be approx: ${pb(Math.floor(size/total*totalAllTime))}`)
+})()

scripts/reset-web-password.js

@@ -0,0 +1,17 @@
+// @ts-check
+
+const {reg, writeRegistration, registrationFilePath} = require("../src/matrix/read-registration")
+const {prompt} = require("enquirer")
+
+;(async () => {
+	/** @type {{web_password: string}} */
+	const passwordResponse = await prompt({
+		type: "text",
+		name: "web_password",
+		message: "Choose a simple password (optional)"
+	})
+
+	reg.ooye.web_password = passwordResponse.web_password
+	writeRegistration(reg)
+	console.log("Saved. Restart Out Of Your Element to apply this change.")
+})()

scripts/setup.js

@@ -1,6 +1,7 @@
 #!/usr/bin/env node
 // @ts-check
 
+const Ty = require("../src/types")
 const assert = require("assert").strict
 const fs = require("fs")
 const sqlite = require("better-sqlite3")
@@ -17,22 +18,6 @@ const {SnowTransfer} = require("snowtransfer")
 const DiscordTypes = require("discord-api-types/v10")
 const {createApp, defineEventHandler, toNodeListener} = require("h3")
 
-// Move database file if it's still in the old location
-if (fs.existsSync("db")) {
-	if (fs.existsSync("db/ooye.db")) {
-		fs.renameSync("db/ooye.db", "ooye.db")
-	}
-	const files = fs.readdirSync("db")
-	if (files.length) {
-		console.error("The db folder is deprecated and must be removed. Your ooye.db database file has already been moved to the root of the repo. You must manually move or delete the remaining files:")
-		for (const file of files) {
-			console.error(file)
-		}
-		process.exit(1)
-	}
-	fs.rmSync("db", {recursive: true})
-}
-
 const passthrough = require("../src/passthrough")
 const db = new sqlite("ooye.db")
 const migrate = require("../src/db/migrate")
@@ -102,7 +87,7 @@ function defineEchoHandler() {
 			type: "input",
 			name: "server_name",
 			message: "Homeserver name",
-			validate: serverName => !!serverName.match(/[a-z][a-z.]+[a-z]/)
+			validate: serverName => !!serverName.match(/[a-z0-9][.a-z0-9-]+[a-z]/)
 		})
 
 		console.log("What is the URL of your homeserver?")
@@ -184,17 +169,21 @@ function defineEchoHandler() {
 			}
 		})
 
-		const mandatoryIntentFlags = DiscordTypes.ApplicationFlags.GatewayMessageContent | DiscordTypes.ApplicationFlags.GatewayMessageContentLimited
-		if (!(client.flags & mandatoryIntentFlags)) {
+		const intentFlagPossibilities = [
+			DiscordTypes.ApplicationFlags.GatewayMessageContent | DiscordTypes.ApplicationFlags.GatewayPresence | DiscordTypes.ApplicationFlags.GatewayGuildMembers,
+			DiscordTypes.ApplicationFlags.GatewayMessageContentLimited | DiscordTypes.ApplicationFlags.GatewayPresenceLimited | DiscordTypes.ApplicationFlags.GatewayGuildMembersLimited
+		]
+		const intentFlagMask = intentFlagPossibilities.reduce((a, c) => a | c, 0)
+		if (!intentFlagPossibilities.includes(client.flags & intentFlagMask)) {
 			console.log(`On that same page, scroll down to Privileged Gateway Intents and enable all switches.`)
 			await prompt({
 				type: "invisible",
 				name: "intents",
 				message: "Press Enter when you've enabled them",
-				validate: async token => {
+				validate: async () => {
 					process.stdout.write(magenta("checking, please wait..."))
 					client = await snow.requestHandler.request(`/applications/@me`, {}, "get", "json")
-					if (client.flags & mandatoryIntentFlags) {
+					if (intentFlagPossibilities.includes(client.flags & intentFlagMask)) {
 						return true
 					} else {
 						return "Switches have not been enabled yet"
@@ -220,7 +209,7 @@ function defineEchoHandler() {
 			type: "invisible",
 			name: "description",
 			message: "Press Enter to acknowledge",
-			validate: async token => {
+			validate: async () => {
 				process.stdout.write(magenta("checking, please wait..."))
 				client = await snow.requestHandler.request(`/applications/@me`, {}, "get", "json")
 				if (client.description?.match(/out.of.your.element/i)) {
@@ -237,7 +226,8 @@ function defineEchoHandler() {
 		const clientSecretResponse = await prompt({
 			type: "input",
 			name: "discord_client_secret",
-			message: "Client secret"
+			message: "Client secret",
+			validate: secret => !!secret
 		})
 
 		const expectedUri = `${bridgeOriginResponse.bridge_origin}/oauth`
@@ -247,7 +237,7 @@ function defineEchoHandler() {
 				type: "invisible",
 				name: "redirect_uri",
 				message: "Press Enter when you've added it",
-				validate: async token => {
+				validate: async () => {
 					process.stdout.write(magenta("checking, please wait..."))
 					client = await snow.requestHandler.request(`/applications/@me`, {}, "get", "json")
 					if (client.redirect_uris.includes(expectedUri)) {
@@ -296,14 +286,15 @@ function defineEchoHandler() {
 	console.log()
 
 	// Done with user prompts, reg is now guaranteed to be valid
+	const mreq = require("../src/matrix/mreq")
 	const api = require("../src/matrix/api")
-	const file = require("../src/matrix/file")
 	const DiscordClient = require("../src/d2m/discord-client")
 	const discord = new DiscordClient(reg.ooye.discord_token, "no")
 	passthrough.discord = discord
 
 	const {as} = require("../src/matrix/appservice")
 	as.router.use("/**", defineEchoHandler())
+	await as.listen()
 
 	console.log("⏳ Waiting for you to register the file with your homeserver... (Ctrl+C to cancel)")
 	process.once("SIGINT", () => {
@@ -353,7 +344,13 @@ function defineEchoHandler() {
 	await api.register(reg.sender_localpart)
 
 	// upload initial images...
-	const avatarUrl = await file.uploadDiscordFileToMxc("https://cadence.moe/friends/out_of_your_element.png")
+	const avatarBuffer = await fs.promises.readFile(join(__dirname, "..", "docs", "img", "icon.png"), null)
+	/** @type {Ty.R.FileUploaded} */
+	const root = await mreq.mreq("POST", "/media/v3/upload", avatarBuffer, {
+		headers: {"Content-Type": "image/png"}
+	})
+	const avatarUrl = root.content_uri
+	assert(avatarUrl)
 
 	console.log("✅ Matrix appservice login works...")
 
@@ -362,8 +359,7 @@ function defineEchoHandler() {
 	console.log("✅ Emojis are ready...")
 
 	// set profile data on discord...
-	const avatarImageBuffer = await fetch("https://cadence.moe/friends/out_of_your_element.png").then(res => res.arrayBuffer())
-	await discord.snow.user.updateSelf({avatar: "data:image/png;base64," + Buffer.from(avatarImageBuffer).toString("base64")})
+	await discord.snow.user.updateSelf({avatar: "data:image/png;base64," + avatarBuffer.toString("base64")})
 	console.log("✅ Discord profile updated...")
 
 	// set profile data on homeserver...

scripts/start-server.js

@@ -34,5 +34,9 @@ passthrough.select = orm.select
 	console.log("Discord gateway started")
 	sync.require("../src/web/server")
 
+	discord.cloud.once("ready", () => {
+		as.listen()
+	})
+
 	require("../src/stdin")
 })()

src/d2m/actions/create-room.js

@@ -124,7 +124,11 @@ async function channelToKState(channel, guild, di) {
 
 	const everyonePermissions = dUtils.getPermissions(guild.id, [], guild.roles, undefined, channel.permission_overwrites)
 	const everyoneCanSend = dUtils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.SendMessages)
-	const everyoneCanMentionEveryone = dUtils.hasAllPermissions(everyonePermissions, ["MentionEveryone"])
+	const everyoneCanMentionEveryone = dUtils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.MentionEveryone)
+
+	const pollStartPowerLevel = {}
+	const everyoneCanCreatePolls = dUtils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.SendPolls)
+	if (everyoneCanSend && !everyoneCanCreatePolls) pollStartPowerLevel["org.matrix.msc3381.poll.start"] = 10
 
 	const spacePowerDetails = await mUtils.getEffectivePower(guildSpaceID, [], di.api)
 	spacePowerDetails.powerLevels.users ??= {}
@@ -159,7 +163,8 @@ async function channelToKState(channel, guild, di) {
 			events_default: everyoneCanSend ? 0 : READ_ONLY_ROOM_EVENTS_DEFAULT_POWER,
 			events: {
 				"m.reaction": 0,
-				"m.room.redaction": 0 // only affects redactions of own events, required to be able to un-react
+				"m.room.redaction": 0, // only affects redactions of own events, required to be able to un-react
+				...pollStartPowerLevel
 			},
 			notifications: {
 				room: everyoneCanMentionEveryone ? 0 : 20
@@ -434,19 +439,11 @@ function syncRoom(channelID) {
 	return _syncRoom(channelID, true)
 }
 
-async function unbridgeChannel(channelID) {
-	/** @ts-ignore @type {DiscordTypes.APIGuildChannel} */
-	const channel = discord.channels.get(channelID)
-	assert.ok(channel)
-	assert.ok(channel.guild_id)
-	return unbridgeDeletedChannel(channel, channel.guild_id)
-}
-
 /**
  * @param {{id: string, topic?: string?}} channel channel-ish (just needs an id, topic is optional)
  * @param {string} guildID
  */
-async function unbridgeDeletedChannel(channel, guildID) {
+async function unbridgeChannel(channel, guildID) {
 	const roomID = select("channel_room", "room_id", {channel_id: channel.id}).pluck().get()
 	assert.ok(roomID)
 	const row = from("guild_space").join("guild_active", "guild_id").select("space_id", "autocreate").where({guild_id: guildID}).get()
@@ -483,14 +480,13 @@ async function unbridgeDeletedChannel(channel, guildID) {
 
 	if (!botInRoom) return
 
-	// demote admins in room
-	/** @type {Ty.Event.M_Power_Levels} */
-	const powerLevelContent = await api.getStateEvent(roomID, "m.room.power_levels", "")
-	powerLevelContent.users ??= {}
-	for (const mxid of Object.keys(powerLevelContent.users)) {
-		if (powerLevelContent.users[mxid] >= 100 && mUtils.eventSenderIsFromDiscord(mxid) && mxid !== mUtils.bot) {
-			delete powerLevelContent.users[mxid]
-			await api.sendState(roomID, "m.room.power_levels", "", powerLevelContent, mxid)
+	// demote discord sim admins in room
+	const {powerLevels, allCreators} = await mUtils.getEffectivePower(roomID, [], api)
+	const powerLevelsUsers = (powerLevels.users ||= {})
+	for (const mxid of Object.keys(powerLevelsUsers)) {
+		if (powerLevelsUsers[mxid] >= (powerLevels.state_default ?? 50) && !allCreators.includes(mxid) && mUtils.eventSenderIsFromDiscord(mxid) && mxid !== mUtils.bot) {
+			delete powerLevelsUsers[mxid]
+			await api.sendState(roomID, "m.room.power_levels", "", powerLevels, mxid) // done individually because each user must demote themselves
 		}
 	}
 
@@ -521,6 +517,7 @@ async function unbridgeDeletedChannel(channel, guildID) {
 	}
 
 	// leave room
+	await mUtils.setUserPower(roomID, mUtils.bot, 0, api)
 	await api.leaveRoom(roomID)
 }
 
@@ -584,6 +581,5 @@ module.exports.postApplyPowerLevels = postApplyPowerLevels
 module.exports._convertNameAndTopic = convertNameAndTopic
 module.exports._syncSpaceMember = _syncSpaceMember
 module.exports.unbridgeChannel = unbridgeChannel
-module.exports.unbridgeDeletedChannel = unbridgeDeletedChannel
 module.exports.existsOrAutocreatable = existsOrAutocreatable
 module.exports.assertExistsOrAutocreatable = assertExistsOrAutocreatable

src/d2m/actions/create-space.js

@@ -203,7 +203,7 @@ async function syncSpaceFully(guildID) {
 		if (discord.channels.has(channelID)) {
 			await createRoom.syncRoom(channelID)
 		} else {
-			await createRoom.unbridgeDeletedChannel({id: channelID}, guildID)
+			await createRoom.unbridgeChannel({id: channelID}, guildID)
 		}
 	}
 
@@ -229,14 +229,16 @@ async function syncSpaceExpressions(data, checkBeforeSync) {
 	 */
 	async function update(spaceID, key, eventKey, fn) {
 		if (!(key in data) || !data[key].length) return
-		const content = await fn(data[key])
+		const guild = discord.guilds.get(data.guild_id)
+		assert(guild)
+		const content = await fn(data[key], guild)
 		if (checkBeforeSync) {
 			let existing
 			try {
 				existing = await api.getStateEvent(spaceID, "im.ponies.room_emotes", eventKey)
 			} catch (e) {
 				// State event not found. This space doesn't have any existing emojis. We create a dummy empty event for comparison's sake.
-				existing = fn([])
+				existing = fn([], guild)
 			}
 			if (isDeepStrictEqual(existing, content)) return
 		}

src/d2m/actions/edit-message.js

@@ -21,7 +21,7 @@ const mreq = sync.require("../../matrix/mreq")
 async function editMessage(message, guild, row) {
 	const historicalRoomOfMessage = from("message_room").join("historical_channel_room", "historical_room_index").where({message_id: message.id}).select("room_id").get()
 	const currentRoom = from("channel_room").join("historical_channel_room", "room_id").where({channel_id: message.channel_id}).select("room_id", "historical_room_index").get()
-	assert(currentRoom)
+	if (!currentRoom) return
 
 	if (historicalRoomOfMessage && historicalRoomOfMessage.room_id !== currentRoom.room_id) return // tombstoned rooms should not have new events (including edits) sent to them
 
@@ -30,7 +30,7 @@ async function editMessage(message, guild, row) {
 	if (row && row.speedbump_webhook_id === message.webhook_id) {
 		// Handle the PluralKit public instance
 		if (row.speedbump_id === "466378653216014359") {
-			senderMxid = await registerPkUser.syncUser(message.id, message.author, roomID, false)
+			senderMxid = await registerPkUser.syncUser(message.id, message.author, roomID, true)
 		}
 	}
 

src/d2m/actions/expression.js

@@ -9,11 +9,12 @@ const file = sync.require("../../matrix/file")
 
 /**
  * @param {DiscordTypes.APIEmoji[]} emojis
+ * @param {DiscordTypes.APIGuild} guild
  */
-async function emojisToState(emojis) {
+async function emojisToState(emojis, guild) {
 	const result = {
 		pack: {
-			display_name: "Discord Emojis",
+			display_name: `${guild.name} (Discord Emojis)`,
 			usage: ["emoticon"] // we'll see...
 		},
 		images: {
@@ -42,11 +43,12 @@ async function emojisToState(emojis) {
 
 /**
  * @param {DiscordTypes.APISticker[]} stickers
+ * @param {DiscordTypes.APIGuild} guild
  */
-async function stickersToState(stickers) {
+async function stickersToState(stickers, guild) {
 	const result = {
 		pack: {
-			display_name: "Discord Stickers",
+			display_name: `${guild.name} (Discord Stickers)`,
 			usage: ["sticker"] // we'll see...
 		},
 		images: {

src/d2m/actions/poll-end.js

@@ -9,22 +9,15 @@ const {discord, sync, db, select, from} = passthrough
 const {reg} = require("../../matrix/read-registration")
 /** @type {import("./poll-vote")} */
 const vote = sync.require("../actions/poll-vote")
-/** @type {import("../../m2d/converters/poll-components")} */
-const pollComponents = sync.require("../../m2d/converters/poll-components")
-
-// This handles, in the following order:
-// * verifying Matrix-side votes are accurate for a poll originating on Discord, sending missed votes to Matrix if necessary
-// * sending a message to Discord if a vote in that poll has been cast on Matrix
-// This does *not* handle bridging of poll closures on Discord to Matrix; that takes place in converters/message-to-event.js.
+/** @type {import("../../discord/interactions/poll-responses")} */
+const pollResponses = sync.require("../../discord/interactions/poll-responses")
 
 /**
- * @param {number} percent
+ * @file This handles, in the following order:
+ * * verifying Matrix-side votes are accurate for a poll originating on Discord, sending missed votes to Matrix if necessary
+ * * sending a message to Discord if a vote in that poll has been cast on Matrix
+ * This does *not* handle bridging of poll closures on Discord to Matrix; that takes place in converters/message-to-event.js.
  */
-function barChart(percent) {
-	const width = 12
-	const bars = Math.floor(percent*width)
-	return "█".repeat(bars) + "▒".repeat(width-bars)
-}
 
 /**
  * @param {string} channelID
@@ -114,26 +107,14 @@ async function endPoll(closeMessage) {
 		}))
 	}
 
-	/** @type {{matrix_option: string, option_text: string, count: number}[]} */
-	const pollResults = db.prepare("SELECT matrix_option, option_text, seq, count(discord_or_matrix_user_id) as count FROM poll_option LEFT JOIN poll_vote USING (message_id, matrix_option) WHERE message_id = ? GROUP BY matrix_option ORDER BY seq").all(pollMessageID)
-	const combinedVotes = pollResults.reduce((a, c) => a + c.count, 0)
-	const totalVoters = db.prepare("SELECT count(DISTINCT discord_or_matrix_user_id) as count FROM poll_vote WHERE message_id = ?").pluck().get(pollMessageID)
+	const {combinedVotes, messageString} = pollResponses.getCombinedResults(pollMessageID, true)
 
-	if (combinedVotes !== totalVotes) { // This means some votes were cast on Matrix!
-		// Now that we've corrected the vote totals, we can get the results again and post them to Discord!
-		const topAnswers = pollResults.toSorted((a, b) => b.count - a.count)
-		let messageString = ""
-		for (const option of pollResults) {
-			const medal = pollComponents.getMedal(topAnswers, option.count)
-			const countString = `${String(option.count).padStart(String(topAnswers[0].count).length)}`
-			const votesString = option.count === 1 ? "vote " : "votes"
-			const label = medal === "🥇" ? `**${option.option_text}**` : option.option_text
-			messageString += `\`\u200b${countString} ${votesString}\u200b\` ${barChart(option.count/totalVoters)} ${label} ${medal}\n`
-		}
+	if (combinedVotes !== totalVotes) { // This means some votes were cast on Matrix. Now that we've corrected the vote totals, we can get the results again and post them to Discord.
 		return {
 			username: "Total results including Matrix votes",
-			avatar_url: `${reg.ooye.bridge_origin}/discord/poll-star-avatar.png`,
-			content: messageString
+			avatar_url: `${reg.ooye.bridge_origin}/download/file/poll-star-avatar.png`,
+			content: messageString,
+			flags: DiscordTypes.MessageFlags.SuppressEmbeds
 		}
 	}
 }

src/d2m/actions/poll-vote.js

@@ -70,13 +70,14 @@ async function sendVotes(userOrID, channelID, pollMessageID, pollEventID) {
 		return
 	}
 
+	let userID, senderMxid
 	if (typeof userOrID === "string") { // just a string when double-checking a vote removal - good thing the unvoter is already here from having voted
-		var userID = userOrID
-		var senderMxid = from("sim").join("sim_member", "mxid").where({user_id: userOrID, room_id: matchingRoomID}).pluck("mxid").get()
+		userID = userOrID
+		senderMxid = from("sim").join("sim_member", "mxid").where({user_id: userOrID, room_id: matchingRoomID}).pluck("mxid").get()
 		if (!senderMxid) return
 	} else { // sent in full when double-checking adding a vote, so we can properly ensure joined
-		var userID = userOrID.id
-		var senderMxid = await registerUser.ensureSimJoined(userOrID, matchingRoomID)
+		userID = userOrID.id
+		senderMxid = await registerUser.ensureSimJoined(userOrID, matchingRoomID)
 	}
 
 	const answersArray = select("poll_vote", "matrix_option", {discord_or_matrix_user_id: userID, message_id: pollMessageID}).pluck().all()

src/d2m/actions/register-user.js

@@ -182,6 +182,10 @@ function memberToPowerLevel(user, member, guild, channel) {
 	const everyoneCanMentionEveryone = dUtils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.MentionEveryone)
 	const userCanMentionEveryone = dUtils.hasPermission(permissions, DiscordTypes.PermissionFlagsBits.MentionEveryone)
 	if (!everyoneCanMentionEveryone && userCanMentionEveryone) return 20
+	/* PL 10 = Create Polls for technical reasons. */
+	const everyoneCanCreatePolls = dUtils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.SendPolls)
+	const userCanCreatePolls = dUtils.hasPermission(permissions, DiscordTypes.PermissionFlagsBits.SendPolls)
+	if (!everyoneCanCreatePolls && userCanCreatePolls) return 10
 	return 0
 }
 

src/d2m/actions/retrigger.js

@@ -19,7 +19,7 @@ const emitter = new EventEmitter()
  * Due to Eventual Consistency(TM) an update/delete may arrive before the original message arrives
  * (or before the it has finished being bridged to an event).
  * In this case, wait until the original message has finished bridging, then retrigger the passed function.
- * @template {(...args: any[]) => Promise<any>} T
+ * @template {(...args: any[]) => any} T
  * @param {string} inputID
  * @param {T} fn
  * @param {Parameters<T>} rest

src/d2m/actions/send-message.js

@@ -55,13 +55,14 @@ async function sendMessage(message, channel, guild, row) {
 		}
 	}
 
+	let sentResultsMessage
 	if (message.type === DiscordTypes.MessageType.PollResult) { // ensure all Discord-side votes were pushed to Matrix before a poll is closed
 		const detailedResultsMessage = await pollEnd.endPoll(message)
 		if (detailedResultsMessage) {
 			const threadParent = select("channel_room", "thread_parent", {channel_id: message.channel_id}).pluck().get()
 			const channelID = threadParent ? threadParent : message.channel_id
 			const threadID = threadParent ? message.channel_id : undefined
-			var sentResultsMessage = await channelWebhook.sendMessageWithWebhook(channelID, detailedResultsMessage, threadID)
+			sentResultsMessage = await channelWebhook.sendMessageWithWebhook(channelID, detailedResultsMessage, threadID)
 		}
 	}
 
@@ -69,7 +70,8 @@ async function sendMessage(message, channel, guild, row) {
 	const eventIDs = []
 	if (events.length) {
 		db.prepare("INSERT OR IGNORE INTO message_room (message_id, historical_room_index) VALUES (?, ?)").run(message.id, historicalRoomIndex)
-		if (senderMxid) api.sendTyping(roomID, false, senderMxid).catch(() => {})
+		const typingMxid = from("sim").join("sim_member", "mxid").where({user_id: message.author.id, room_id: roomID}).pluck("mxid").get()
+		if (typingMxid) api.sendTyping(roomID, false, typingMxid).catch(() => {})
 	}
 	for (const event of events) {
 		const part = event === events[0] ? 0 : 1
@@ -84,7 +86,19 @@ async function sendMessage(message, channel, guild, row) {
 
 		const useTimestamp = message["backfill"] ? new Date(message.timestamp).getTime() : undefined
 		const eventID = await api.sendEvent(roomID, eventType, eventWithoutType, senderMxid, useTimestamp)
-		db.prepare("INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part, reaction_part, source) VALUES (?, ?, ?, ?, ?, ?, 1)").run(eventID, eventType, event.msgtype || null, message.id, part, reactionPart) // source 1 = discord
+		eventIDs.push(eventID)
+
+		try {
+			db.prepare("INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part, reaction_part, source) VALUES (?, ?, ?, ?, ?, ?, 1)").run(eventID, eventType, event.msgtype || null, message.id, part, reactionPart) // source 1 = discord
+		} catch (e) {
+			// check if we got rugpulled
+			if (!select("message_room", "message_id", {message_id: message.id}).get()) {
+				for (const eventID of eventIDs) {
+					await api.redactEvent(roomID, eventID)
+				}
+				return []
+			}
+		}
 
 		// The primary event is part = 0 and has the most important and distinct information. It is used to provide reply previews, be pinned, and possibly future uses.
 		// The first event is chosen to be the primary part because it is usually the message text content and is more likely to be distinct.
@@ -117,11 +131,10 @@ async function sendMessage(message, channel, guild, row) {
 			db.transaction(() => {
 				db.prepare("INSERT OR IGNORE INTO message_room (message_id, historical_room_index) VALUES (?, ?)").run(sentResultsMessage.id, historicalRoomIndex)
 				db.prepare("UPDATE event_message SET reaction_part = 1 WHERE event_id = ?").run(eventID)
-				db.prepare("INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part, reaction_part, source) VALUES (?, ?, ?, ?, ?, ?, 1)").run(eventID, eventType, event.msgtype || null, sentResultsMessage.id, 1, 0) // part = 1, reaction_part = 0
+				// part = 1, reaction_part = 0, source = 0 as the results are "from Matrix" and doing otherwise breaks things when that message gets updated by Discord (it just does that sometimes)
+				db.prepare("INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part, reaction_part, source) VALUES (?, ?, ?, ?, ?, ?, 0)").run(eventID, eventType, event.msgtype || null, sentResultsMessage.id, 1, 0)
 			})()
 		}
-
-		eventIDs.push(eventID)
 	}
 
 	return eventIDs

src/d2m/actions/speedbump.js

@@ -4,6 +4,14 @@ const DiscordTypes = require("discord-api-types/v10")
 const passthrough = require("../../passthrough")
 const {discord, select, db} = passthrough
 
+const DEBUG_SPEEDBUMP = false
+
+function debugSpeedbump(message) {
+	if (DEBUG_SPEEDBUMP) {
+		console.log(message)
+	}
+}
+
 const SPEEDBUMP_SPEED = 4000 // 4 seconds delay
 const SPEEDBUMP_UPDATE_FREQUENCY = 2 * 60 * 60 // 2 hours
 
@@ -27,8 +35,8 @@ async function updateCache(channelID, lastChecked) {
 	db.prepare("UPDATE channel_room SET speedbump_id = ?, speedbump_webhook_id = ?, speedbump_checked = ? WHERE channel_id = ?").run(foundApplication, foundWebhook, now, channelID)
 }
 
-/** @type {Set<string>} set of messageID */
-const bumping = new Set()
+/** @type {Map<string, number>} messageID -> number of gateway events currently bumping */
+const bumping = new Map()
 
 /**
  * Slow down a message. After it passes the speedbump, return whether it's okay or if it's been deleted.
@@ -36,9 +44,26 @@ const bumping = new Set()
  * @returns whether it was deleted
  */
 async function doSpeedbump(messageID) {
-	bumping.add(messageID)
+	let value = (bumping.get(messageID) ?? 0) + 1
+	bumping.set(messageID, value)
+	debugSpeedbump(`[speedbump] WAIT ${messageID}++ = ${value}`)
+
 	await new Promise(resolve => setTimeout(resolve, SPEEDBUMP_SPEED))
-	return !bumping.delete(messageID)
+
+	if (!bumping.has(messageID)) {
+		debugSpeedbump(`[speedbump] DELETED ${messageID}`)
+		return true
+	}
+	value = (bumping.get(messageID) ?? 0) - 1
+	if (value <= 0) {
+		debugSpeedbump(`[speedbump] OK ${messageID}-- = ${value}`)
+		bumping.delete(messageID)
+		return false
+	} else {
+		debugSpeedbump(`[speedbump] MULTI ${messageID}-- = ${value}`)
+		bumping.set(messageID, value)
+		return true
+	}
 }
 
 /**

src/d2m/converters/edit-to-changes.js

@@ -16,8 +16,12 @@ function eventCanBeEdited(ev) {
 	if (ev.old.event_type === "m.room.message" && ev.old.event_subtype !== "m.text" && ev.old.event_subtype !== "m.emote" && ev.old.event_subtype !== "m.notice") {
 		return false
 	}
-	// Discord does not allow stickers to be edited. Poll closures are sent as "edits", but not in a way we care about.
-	if (ev.old.event_type === "m.sticker" || ev.old.event_type === "org.matrix.msc3381.poll.start") {
+	// Discord does not allow stickers to be edited.
+	if (ev.old.event_type === "m.sticker") {
+		return false
+	}
+	// Discord does not allow the data of polls to be edited, they may only be responded to.
+	if (ev.old.event_type === "org.matrix.msc3381.poll.start" || ev.old.event_type === "org.matrix.msc3381.poll.end") {
 		return false
 	}
 	// Anything else is fair game.
@@ -45,8 +49,8 @@ async function editToChanges(message, guild, api) {
 	// Since an update in August 2024, the system always provides the full data of message updates.
 	// Now, this code path is only used by generated embeds for messages that were originally sent from Matrix.
 
-	const originallyFromMatrix = oldEventRows.find(r => r.part === 0)?.source === 0
-	const isGeneratedEmbed = !("content" in message) || originallyFromMatrix
+	const originallyFromMatrix = oldEventRows.some(r => r.source === 0)
+	const mightBeGeneratedEmbed = !("content" in message) || originallyFromMatrix
 
 	// Figure out who to send as
 
@@ -79,7 +83,7 @@ async function editToChanges(message, guild, api) {
 	*/
 	/**
 	 * 1. Events that are matched, and should be edited by sending another m.replace event
-	 * @type {{old: typeof oldEventRows[0], newFallbackContent: typeof newFallbackContent[0], newInnerContent: typeof newInnerContent[0]}[]}
+	 * @type {{old: typeof oldEventRows[0], oldMentions?: any, newFallbackContent: typeof newFallbackContent[0], newInnerContent: typeof newInnerContent[0]}[]}
 	 */
 	let eventsToReplace = []
 	/**
@@ -133,20 +137,24 @@ async function editToChanges(message, guild, api) {
 
 	// If this is a generated embed update, only allow the embeds to be updated, since the system only sends data about events. Ignore changes to other things.
 	// This also prevents Matrix events that were re-subtyped during conversion (e.g. large image -> text link) from being mistakenly included.
-	if (isGeneratedEmbed) {
+	if (mightBeGeneratedEmbed) {
 		unchangedEvents = unchangedEvents.concat(
 			dUtils.filterTo(eventsToRedact, e => e.old.event_subtype === "m.notice" && e.old.source === 1), // Move everything except embeds from eventsToRedact to unchangedEvents.
 			dUtils.filterTo(eventsToReplace, e => e.old.event_subtype === "m.notice" && e.old.source === 1) // Move everything except embeds from eventsToReplace to unchangedEvents.
 		)
 		eventsToSend = eventsToSend.filter(e => e.msgtype === "m.notice") // Don't send new events that aren't the embed.
+	}
 
-		// Don't post new generated embeds for messages if it's been a while since the message was sent. Detached embeds look weird.
-		const messageTooOld = message.timestamp && new Date(message.timestamp).getTime() < Date.now() - 30 * 1000 // older than 30 seconds ago
-		// Don't post new generated embeds for messages if the setting was disabled.
-		const embedsEnabled = select("guild_space", "url_preview", {guild_id: guild?.id}).pluck().get() ?? 1
-		if (messageTooOld || !embedsEnabled) {
-			eventsToSend = []
-		}
+	// Don't post new generated embeds for messages if it's been a while since the message was sent. Detached embeds look weird.
+	const messageQuiteOld = message.timestamp && new Date(message.timestamp).getTime() < Date.now() - 30 * 1000 // older than 30 seconds ago
+	// Don't send anything new at all if it's been longer since the message was sent. Detached messages are just inappropriate.
+	const messageReallyOld = message.timestamp && new Date(message.timestamp).getTime() < Date.now() - 2 * 60 * 1000 // older than 2 minutes ago
+	// Don't post new generated embeds for messages if the setting was disabled.
+	const embedsEnabled = select("guild_space", "url_preview", {guild_id: guild?.id}).pluck().get() ?? 1
+	if (messageReallyOld) {
+		eventsToSend = [] // Only allow edits to change and delete, but not send new.
+	} else if ((messageQuiteOld || !embedsEnabled) && !message.author?.bot) {
+		eventsToSend = eventsToSend.filter(e => e.msgtype !== "m.notice") // Only send events that aren't embeds.
 	}
 
 	// Now, everything in eventsToSend and eventsToRedact is a real change, but everything in eventsToReplace might not have actually changed!
@@ -161,6 +169,7 @@ async function editToChanges(message, guild, api) {
 		const event = eventsToReplace[i]
 		if (!eventIsText(event)) continue // not text, can't analyse
 		const oldEvent = await api.getEvent(roomID, eventsToReplace[i].old.event_id)
+		eventsToReplace[i].oldMentions = oldEvent.content["m.mentions"]
 		const oldEventBodyWithoutQuotedReply = oldEvent.content.body?.replace(/^(>.*\n)*\n*/sm, "")
 		if (oldEventBodyWithoutQuotedReply !== event.newInnerContent.body) continue // event changed, must replace it
 		// Move it from eventsToRedact to unchangedEvents.
@@ -210,7 +219,7 @@ async function editToChanges(message, guild, api) {
 	// Removing unnecessary properties before returning
 	return {
 		roomID,
-		eventsToReplace: eventsToReplace.map(e => ({oldID: e.old.event_id, newContent: makeReplacementEventContent(e.old.event_id, e.newFallbackContent, e.newInnerContent)})),
+		eventsToReplace: eventsToReplace.map(e => ({oldID: e.old.event_id, newContent: makeReplacementEventContent(e.old.event_id, e.oldMentions, e.newFallbackContent, e.newInnerContent)})),
 		eventsToRedact: eventsToRedact.map(e => e.old.event_id),
 		eventsToSend,
 		senderMxid,
@@ -221,14 +230,25 @@ async function editToChanges(message, guild, api) {
 /**
  * @template T
  * @param {string} oldID
+ * @param {any} oldMentions
  * @param {T} newFallbackContent
  * @param {T} newInnerContent
  * @returns {import("../../types").Event.ReplacementContent<T>} content
  */
-function makeReplacementEventContent(oldID, newFallbackContent, newInnerContent) {
+function makeReplacementEventContent(oldID, oldMentions, newFallbackContent, newInnerContent) {
+	const mentions = {}
+	const newMentionUsers = new Set(newFallbackContent["m.mentions"]?.user_ids || [])
+	const oldMentionUsers = new Set(oldMentions?.user_ids || [])
+	const mentionDiff = newMentionUsers.difference(oldMentionUsers)
+	if (mentionDiff.size) {
+		mentions.user_ids = [...mentionDiff.values()]
+	}
+	if (newFallbackContent["m.mentions"]?.room && !oldMentions?.room) {
+		mentions.room = true
+	}
 	const content = {
 		...newFallbackContent,
-		"m.mentions": {},
+		"m.mentions": mentions,
 		"m.new_content": {
 			...newInnerContent
 		},

src/d2m/converters/edit-to-changes.test.js

@@ -42,7 +42,14 @@ test("edit2changes: bot response", async t => {
 	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.bot_response, data.guild.general, {
 		getEvent(roomID, eventID) {
 			t.equal(eventID, "$fdD9OZ55xg3EAsfvLZza5tMhtjUO91Wg3Otuo96TplY")
-			return {content: {body: "dummy"}}
+			return {
+				content: {
+					"m.mentions": {
+						user_ids: ["@cadence:cadence.moe"],
+					},
+					body: "dummy"
+				}
+			}
 		},
 		async getJoinedMembers(roomID) {
 			t.equal(roomID, "!hYnGGlPHlbujVVfktC:cadence.moe")
@@ -71,18 +78,18 @@ test("edit2changes: bot response", async t => {
 		newContent: {
 			$type: "m.room.message",
 			msgtype: "m.text",
-			body: "* :ae_botrac4r: @cadence asked ``­``, I respond: Stop drinking paint. (No)\n\nHit :bn_re: to reroll.",
+			body: "* :ae_botrac4r: [@cadence](https://matrix.to/#/@cadence:cadence.moe) asked ``­``, I respond: Stop drinking paint. (No)\n\nHit :bn_re: to reroll.",
 			format: "org.matrix.custom.html",
-			formatted_body: '* <img data-mx-emoticon height="32" src="mxc://cadence.moe/skqfuItqxNmBYekzmVKyoLzs" title=":ae_botrac4r:" alt=":ae_botrac4r:"> @cadence asked <code>­</code>, I respond: Stop drinking paint. (No)<br><br>Hit <img data-mx-emoticon height="32" src="mxc://cadence.moe/OIpqpfxTnHKokcsYqDusxkBT" title=":bn_re:" alt=":bn_re:"> to reroll.',
+			formatted_body: '* <img data-mx-emoticon height="32" src="mxc://cadence.moe/skqfuItqxNmBYekzmVKyoLzs" title=":ae_botrac4r:" alt=":ae_botrac4r:"> <a href="https://matrix.to/#/@cadence:cadence.moe">@cadence</a> asked <code>­</code>, I respond: Stop drinking paint. (No)<br><br>Hit <img data-mx-emoticon height="32" src="mxc://cadence.moe/OIpqpfxTnHKokcsYqDusxkBT" title=":bn_re:" alt=":bn_re:"> to reroll.',
 			"m.mentions": {
 				// Client-Server API spec 11.37.7: Copy Discord's behaviour by not re-notifying anyone that an *edit occurred*
 			},
 			// *** Replaced With: ***
 			"m.new_content": {
 				msgtype: "m.text",
-				body: ":ae_botrac4r: @cadence asked ``­``, I respond: Stop drinking paint. (No)\n\nHit :bn_re: to reroll.",
+				body: ":ae_botrac4r: [@cadence](https://matrix.to/#/@cadence:cadence.moe) asked ``­``, I respond: Stop drinking paint. (No)\n\nHit :bn_re: to reroll.",
 				format: "org.matrix.custom.html",
-				formatted_body: '<img data-mx-emoticon height="32" src="mxc://cadence.moe/skqfuItqxNmBYekzmVKyoLzs" title=":ae_botrac4r:" alt=":ae_botrac4r:"> @cadence asked <code>­</code>, I respond: Stop drinking paint. (No)<br><br>Hit <img data-mx-emoticon height="32" src="mxc://cadence.moe/OIpqpfxTnHKokcsYqDusxkBT" title=":bn_re:" alt=":bn_re:"> to reroll.',
+				formatted_body: '<img data-mx-emoticon height="32" src="mxc://cadence.moe/skqfuItqxNmBYekzmVKyoLzs" title=":ae_botrac4r:" alt=":ae_botrac4r:"> <a href="https://matrix.to/#/@cadence:cadence.moe">@cadence</a> asked <code>­</code>, I respond: Stop drinking paint. (No)<br><br>Hit <img data-mx-emoticon height="32" src="mxc://cadence.moe/OIpqpfxTnHKokcsYqDusxkBT" title=":bn_re:" alt=":bn_re:"> to reroll.',
 				"m.mentions": {
 					// Client-Server API spec 11.37.7: This should contain the mentions for the final version of the event
 					"user_ids": ["@cadence:cadence.moe"]
@@ -365,6 +372,7 @@ test("edit2changes: generated embed", async t => {
 
 test("edit2changes: generated embed on a reply", async t => {
 	let called = 0
+	data.message_update.embed_generated_on_reply.timestamp = new Date().toISOString()
 	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.embed_generated_on_reply, data.guild.general, {
 		getEvent(roomID, eventID) {
 			called++

src/d2m/converters/find-mentions.js

@@ -0,0 +1,161 @@
+// @ts-check
+
+const assert = require("assert")
+
+const {reg} = require("../../matrix/read-registration")
+const userRegex = reg.namespaces.users.map(u => new RegExp(u.regex))
+
+/**
+ * @typedef {{text: string, index: number, end: number}} Token
+ */
+
+/** @typedef {{mxids: {localpart: string, mxid: string, displayname?: string | null}[], names: {displaynameTokens: Token[], mxid: string}[]}} ProcessedJoined */
+
+const lengthBonusLengthCap = 50
+const lengthBonusValue = 0.5
+/**
+ * Score by how many characters in a row at the start of input are in localpart. 2x if it matches at the start. +1 tiebreaker bonus if it matches all.
+ * 0 = no match
+ * @param {string} localpart
+ * @param {string} input
+ * @param {string | null} [displayname] only for the super tiebreaker
+ * @returns {{score: number, matchedInputTokens: Token[]}}
+ */
+function scoreLocalpart(localpart, input, displayname) {
+	let score = 0
+	let atStart = false
+	let matchingLocations = []
+	do {
+		atStart = matchingLocations[0] === 0
+		let chars = input[score]
+		if (score === 0) {
+			// add all possible places
+			let i = 0
+			while ((i = localpart.indexOf(chars, i)) !== -1) {
+				matchingLocations.push(i)
+				i++
+			}
+		} else {
+			// trim down remaining places
+			matchingLocations = matchingLocations.filter(i => localpart[i+score] === input[score])
+		}
+		if (matchingLocations.length) {
+			score++
+			if (score === localpart.length) break
+		}
+	} while (matchingLocations.length)
+	/** @type {Token} */
+	const fakeToken = {text: input.slice(0, score), index: 0, end: score}
+	const displaynameLength = displayname?.length ?? 0
+	if (score === localpart.length) score = score * 2 + 1 + Math.max(((lengthBonusLengthCap-displaynameLength)/lengthBonusLengthCap)*lengthBonusValue, 0)
+	else if (atStart) score = score * 2
+	return {score, matchedInputTokens: [fakeToken]}
+}
+
+const decayDistance = 20
+const decayValue = 0.33
+/**
+ * Score by how many tokens in sequence (not necessarily back to back) at the start of input are in display name tokens. Score each token on its length. 2x if it matches at the start. +1 tiebreaker bonus if it matches all
+ * @param {Token[]} displaynameTokens
+ * @param {Token[]} inputTokens
+ * @returns {{score: number, matchedInputTokens: Token[]}}
+ */
+function scoreName(displaynameTokens, inputTokens) {
+	let matchedInputTokens = []
+	let score = 0
+	let searchFrom = 0
+	for (let nextInputTokenIndex = 0; nextInputTokenIndex < inputTokens.length; nextInputTokenIndex++) {
+		// take next
+		const nextToken = inputTokens[nextInputTokenIndex]
+		// see if it's there
+		let foundAt = displaynameTokens.findIndex((tk, idx) => idx >= searchFrom && tk.text === nextToken.text)
+		if (foundAt !== -1) {
+			// update scoring
+			matchedInputTokens.push(nextToken)
+			score += nextToken.text.length * Math.max(((decayDistance-foundAt)*(1+decayValue))/(decayDistance*(1+decayValue)), decayValue) // decay score 100%->33% the further into the displayname it's found
+			// prepare for next loop
+			searchFrom = foundAt + 1
+		} else {
+			break
+		}
+	}
+	const firstTextualInputToken = inputTokens.find(t => t.text.match(/^\w/))
+	if (matchedInputTokens[0] === inputTokens[0] || matchedInputTokens[0] === firstTextualInputToken) score *= 2
+	if (matchedInputTokens.length === displaynameTokens.length) score += 1
+	return {score, matchedInputTokens}
+}
+
+/**
+ * @param {string} name
+ * @returns {Token[]}
+ */
+function tokenise(name) {
+	name = name.replaceAll("\ufe0f", "").normalize().toLowerCase()
+	let index = 0
+	let result = []
+	for (const part of name.split(/(_|\s|\b)/g)) {
+		if (part.trim()) {
+			result.push({text: part, index, end: index + part.length})
+		}
+		index += part.length
+	}
+	return result
+}
+
+/**
+ * @param {{mxid: string, displayname?: string | null}[]} joined
+ * @returns {ProcessedJoined}
+ */
+function processJoined(joined) {
+	joined = joined.filter(j => !userRegex.some(rx => j.mxid.match(rx)))
+	return {
+		mxids: joined.map(j => {
+			const localpart = j.mxid.match(/@([^:]*)/)
+			assert(localpart)
+			return {
+				localpart: localpart[1].toLowerCase(),
+				mxid: j.mxid,
+				displayname: j.displayname
+			}
+		}),
+		names: joined.filter(j => j.displayname).map(j => {
+			return {
+				// @ts-ignore
+				displaynameTokens: tokenise(j.displayname),
+				mxid: j.mxid
+			}
+		})
+	}
+}
+
+/**
+ * @param {ProcessedJoined} pjr
+ * @param {string} maximumWrittenSection lowercase please
+ * @param {number} baseOffset
+ * @param {string} prefix
+ * @param {string} content
+ */
+function findMention(pjr, maximumWrittenSection, baseOffset, prefix, content) {
+	if (!pjr.mxids.length && !pjr.names.length) return
+	const maximumWrittenSectionTokens = tokenise(maximumWrittenSection)
+	/** @type {{mxid: string, scored: {score: number, matchedInputTokens: Token[]}}[]} */
+	let allItems = pjr.mxids.map(mxid => ({...mxid, scored: scoreLocalpart(mxid.localpart, maximumWrittenSection, mxid.displayname)}))
+	allItems = allItems.concat(pjr.names.map(name => ({...name, scored: scoreName(name.displaynameTokens, maximumWrittenSectionTokens)})))
+	const best = allItems.sort((a, b) => b.scored.score - a.scored.score)[0]
+	if (best.scored.score > 4) { // requires in smallest case perfect match of 2 characters, or in largest case a partial middle match of 5+ characters in a row
+		// Highlight the relevant part of the message
+		const start = baseOffset + best.scored.matchedInputTokens[0].index
+		const end = baseOffset + prefix.length + best.scored.matchedInputTokens.slice(-1)[0].end
+		const newContent = content.slice(0, start) + "[" + content.slice(start, end) + "](https://matrix.to/#/" + best.mxid + ")" + content.slice(end)
+		return {
+			mxid: best.mxid,
+			newContent
+		}
+	}
+}
+
+module.exports.scoreLocalpart = scoreLocalpart
+module.exports.scoreName = scoreName
+module.exports.tokenise = tokenise
+module.exports.processJoined = processJoined
+module.exports.findMention = findMention

src/d2m/converters/find-mentions.test.js

@@ -0,0 +1,129 @@
+// @ts-check
+
+const {test} = require("supertape")
+const {processJoined, scoreLocalpart, scoreName, tokenise, findMention} = require("./find-mentions")
+
+test("score localpart: score against cadence", t => {
+	const localparts = [
+		"cadence",
+		"cadence_test",
+		"roblkyogre",
+		"cat",
+		"arcade_cabinet"
+	]
+	t.deepEqual(localparts.map(l => scoreLocalpart(l, "cadence").score), [
+		15.5,
+		14,
+		0,
+		4,
+		4
+	])
+})
+
+test("score mxid: tiebreak multiple perfect matches on name length", t => {
+	const users = [
+		{displayname: "Emma [it/its] ⚡️", localpart: "emma"},
+		{displayname: "Emma [it/its]", localpart: "emma"}
+	]
+	const results = users.map(u => scoreLocalpart(u.localpart, "emma", u.displayname).score)
+	t.ok(results[0] < results[1], `comparison: ${results.join(" < ")}`)
+})
+
+test("score name: score against cadence", t => {
+	const names = [
+		"bgt lover",
+		"Ash 🦑 (xey/it)",
+		"Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆",
+		"underscore_idiot #sunshine",
+		"INX | Evil Lillith (she/her)",
+		"INX | Lillith (she/her)",
+		"🌟luna🌟",
+		"#1 Ritsuko Kinnie"
+	]
+	t.deepEqual(names.map(n => scoreName(tokenise(n), tokenise("cadence")).score), [
+		0,
+		0,
+		14,
+		0,
+		0,
+		0,
+		0,
+		0
+	])
+})
+
+test("score name: nothing scored after a token doesn't match", t => {
+	const names = [
+		"bgt lover",
+		"Ash 🦑 (xey/it)",
+		"Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆",
+		"underscore_idiot #sunshine",
+		"INX | Evil Lillith (she/her)",
+		"INX | Lillith (she/her)",
+		"🌟luna🌟",
+		"#1 Ritsuko Kinnie"
+	]
+	t.deepEqual(names.map(n => scoreName(tokenise(n), tokenise("I hope so")).score), [
+		0,
+		0,
+		0,
+		0,
+		0,
+		0,
+		0,
+		0
+	])
+})
+
+test("score name: prefers earlier match", t => {
+	const names = [
+		"INX | Lillith (she/her)",
+		"INX | Evil Lillith (she/her)"
+	]
+	const results = names.map(n => scoreName(tokenise(n), tokenise("lillith")).score)
+	t.ok(results[0] > results[1], `comparison: ${results.join(" > ")}`)
+})
+
+test("score name: matches lots of tokens", t => {
+	t.deepEqual(
+		Math.round(scoreName(tokenise("Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆"), tokenise("cadence maid of creation eye of clarity empress of hope")).score),
+		65
+	)
+})
+
+test("score name: prefers variation when you specify it", t => {
+	const names = [
+		"Cadence (test account)",
+		"Cadence"
+	]
+	const results = names.map(n => scoreName(tokenise(n), tokenise("cadence test")).score)
+	t.ok(results[0] > results[1], `comparison: ${results.join(" > ")}`)
+})
+
+test("score name: prefers original when not specified", t => {
+	const names = [
+		"Cadence (test account)",
+		"Cadence"
+	]
+	const results = names.map(n => scoreName(tokenise(n), tokenise("cadence")).score)
+	t.ok(results[0] < results[1], `comparison: ${results.join(" < ")}`)
+})
+
+test("score name: finds match location", t => {
+	const message = "evil lillith is an inspiration"
+	const result = scoreName(tokenise("INX | Evil Lillith (she/her)"), tokenise(message))
+	const startLocation = result.matchedInputTokens[0].index
+	const endLocation = result.matchedInputTokens.slice(-1)[0].end
+	t.equal(message.slice(startLocation, endLocation), "evil lillith")
+})
+
+test("find mention: test various tiebreakers", t => {
+	const found = findMention(processJoined([{
+		mxid: "@emma:conduit.rory.gay",
+		displayname: "Emma [it/its] ⚡️"
+	}, {
+		mxid: "@emma:rory.gay",
+		displayname: "Emma [it/its]"
+	}]), "emma ⚡ curious which one this prefers", 0, "@", "@emma ⚡ curious which one this prefers")
+	t.equal(found?.mxid, "@emma:conduit.rory.gay")
+})

src/d2m/converters/message-to-event.js

@@ -18,10 +18,12 @@ const lottie = sync.require("../actions/lottie")
 const mxUtils = sync.require("../../matrix/utils")
 /** @type {import("../../discord/utils")} */
 const dUtils = sync.require("../../discord/utils")
+/** @type {import("./find-mentions")} */
+const findMentions = sync.require("./find-mentions")
+/** @type {import("../../discord/interactions/poll-responses")} */
+const pollResponses = sync.require("../../discord/interactions/poll-responses")
 const {reg} = require("../../matrix/read-registration")
 
-const userRegex = reg.namespaces.users.map(u => new RegExp(u.regex))
-
 /**
  * @param {DiscordTypes.APIMessage} message
  * @param {DiscordTypes.APIGuild} guild
@@ -107,9 +109,10 @@ const embedTitleParser = markdown.markdownEngine.parserFor({
 
 /**
  * @param {{room?: boolean, user_ids?: string[]}} mentions
- * @param {DiscordTypes.APIAttachment} attachment
+ * @param {Omit<DiscordTypes.APIAttachment, "id" | "proxy_url">} attachment
+ * @param {boolean} [alwaysLink]
  */
-async function attachmentToEvent(mentions, attachment) {
+async function attachmentToEvent(mentions, attachment, alwaysLink) {
 	const external_url = dUtils.getPublicUrlForCdn(attachment.url)
 	const emoji =
 		attachment.content_type?.startsWith("image/jp") ? "📸"
@@ -130,7 +133,7 @@ async function attachmentToEvent(mentions, attachment) {
 		}
 	}
 	// for large files, always link them instead of uploading so I don't use up all the space in the content repo
-	else if (attachment.size > reg.ooye.max_file_size) {
+	else if (alwaysLink || attachment.size > reg.ooye.max_file_size) {
 		return {
 			$type: "m.room.message",
 			"m.mentions": mentions,
@@ -228,6 +231,7 @@ async function pollToEvent(poll) {
 		return matrixAnswer;
 	})
 	return {
+		/** @type {"org.matrix.msc3381.poll.start"} */
 		$type: "org.matrix.msc3381.poll.start",
 		"org.matrix.msc3381.poll.start": {
 			question: {
@@ -243,6 +247,20 @@ async function pollToEvent(poll) {
 	}
 }
 
+/**
+ * @param {DiscordTypes.APIMessageInteraction} interaction
+ * @param {boolean} isThinkingInteraction
+ */
+function getFormattedInteraction(interaction, isThinkingInteraction) {
+	const mxid = select("sim", "mxid", {user_id: interaction.user.id}).pluck().get()
+	const username = interaction.member?.nick || interaction.user.global_name || interaction.user.username
+	const thinkingText = isThinkingInteraction ? " — interaction loading..." : ""
+	return {
+		body: `↪️ ${username} used \`/${interaction.name}\`${thinkingText}`,
+		html: `<blockquote><sub>↪️ ${mxid ? tag`<a href="https://matrix.to/#/${mxid}">${username}</a>` : username} used <code>/${interaction.name}</code>${thinkingText}</sub></blockquote>`
+	}
+}
+
 /**
  * @param {DiscordTypes.APIMessage} message
  * @param {DiscordTypes.APIGuild} guild
@@ -255,6 +273,7 @@ async function pollToEvent(poll) {
  * @returns {Promise<{$type: string, $sender?: string, [x: string]: any}[]>}
  */
 async function messageToEvent(message, guild, options = {}, di) {
+	message = structuredClone(message)
 	const events = []
 
 	/* c8 ignore next 7 */
@@ -267,7 +286,21 @@ async function messageToEvent(message, guild, options = {}, di) {
 	}
 
 	if (message.type === DiscordTypes.MessageType.PollResult) {
-		const event_id = select("event_message", "event_id", {message_id: message.message_reference?.message_id}).pluck().get()
+		const pollMessageID = message.message_reference?.message_id
+		if (!pollMessageID) return []
+		const event_id = select("event_message", "event_id", {message_id: pollMessageID}).pluck().get()
+		const roomID = select("channel_room", "room_id", {channel_id: message.channel_id}).pluck().get()
+		const pollQuestionText = select("poll", "question_text", {message_id: pollMessageID}).pluck().get()
+		if (!event_id || !roomID || !pollQuestionText) return [] // drop it if the corresponding poll start was not bridged
+
+		const rep = new mxUtils.MatrixStringBuilder()
+		rep.addLine(`The poll ${pollQuestionText} has closed.`, tag`The poll <a href="https://matrix.to/#/${roomID}/${event_id}">${pollQuestionText}</a> has closed.`)
+
+		const {messageString} = pollResponses.getCombinedResults(pollMessageID, true) // poll results have already been double-checked before this point, so these totals will be accurate
+		rep.addLine(markdown.toHTML(messageString, {discordOnly: true, escapeHTML: false}), markdown.toHTML(messageString, {}))
+
+		const {body, formatted_body} = rep.get()
+
 		return [{
 			$type: "org.matrix.msc3381.poll.end",
 			"m.relates_to": {
@@ -275,7 +308,11 @@ async function messageToEvent(message, guild, options = {}, di) {
 				event_id
 			},
 			"org.matrix.msc3381.poll.end": {},
-			body: "This poll has ended.",
+			"org.matrix.msc1767.text": body,
+			"org.matrix.msc1767.html": formatted_body,
+			body: body,
+			format: "org.matrix.custom.html",
+			formatted_body: formatted_body,
 			msgtype: "m.text"
 		}]
 	}
@@ -298,14 +335,8 @@ async function messageToEvent(message, guild, options = {}, di) {
 	}
 
 	const interaction = message.interaction_metadata || message.interaction
-	if (message.type === DiscordTypes.MessageType.ChatInputCommand && interaction && "name" in interaction) {
-		// Commands are sent by the responding bot. Need to attach the metadata of the person using the command at the top.
-		let content = message.content
-		if (content) content = `\n${content}`
-		else if ((message.flags || 0) & DiscordTypes.MessageFlags.Loading) content = " — interaction loading..."
-		content = `> ↪️ <@${interaction.user.id}> used \`/${interaction.name}\`${content}`
-		message = {...message, content} // editToChanges reuses the object so we can't mutate it. have to clone it
-	}
+	const isInteraction = message.type === DiscordTypes.MessageType.ChatInputCommand && !!interaction && "name" in interaction
+	const isThinkingInteraction = isInteraction && !!((message.flags || 0) & DiscordTypes.MessageFlags.Loading)
 
 	/**
 	   @type {{room?: boolean, user_ids?: string[]}}
@@ -405,7 +436,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 	 * @param {string} [timestampChannelID]
 	 */
 	async function getHistoricalEventRow(messageID, timestampChannelID) {
-		/** @type {{room_id: string} | {event_id: string, room_id: string, reference_channel_id: string, source: number} | null} */
+		/** @type {{room_id: string} | {event_id: string, room_id: string, reference_channel_id: string, source: number} | null | undefined} */
 		let row = from("event_message").join("message_room", "message_id").join("historical_channel_room", "historical_room_index")
 			.select("event_id", "room_id", "reference_channel_id", "source").where({message_id: messageID}).and("ORDER BY part ASC").get()
 		if (!row && timestampChannelID) {
@@ -472,6 +503,12 @@ async function messageToEvent(message, guild, options = {}, di) {
 		content = transformAttachmentLinks(content)
 		content = await transformContentMessageLinks(content)
 
+		// Remove smalltext from non-bots (I don't like it). Webhooks included due to PluralKit.
+		const isHumanOrDataMissing = !message.author?.bot
+		if (isHumanOrDataMissing || dUtils.isWebhookMessage(message)) {
+			content = content.replaceAll(/^-# +([^\n].*?)/gm, "...$1")
+		}
+
 		// Handling emojis that we don't know about. The emoji has to be present in the DB for it to be picked up in the emoji markdown converter.
 		// So we scan the message ahead of time for all its emojis and ensure they are in the DB.
 		const emojiMatches = [...content.matchAll(/<(a?):([^:>]{1,64}):([0-9]+)>/g)]
@@ -538,7 +575,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 		//   1. The replied-to event is in a different room to where the reply will be sent (i.e. a room upgrade occurred between)
 		//   2. The replied-to message has no corresponding Matrix event (repliedToUnknownEvent is true)
 		// This branch is optional - do NOT change anything apart from the reply fallback, since it may not be run
-		if ((repliedToEventRow || repliedToUnknownEvent) && options.includeReplyFallback !== false) {
+		if ((repliedToEventRow || repliedToUnknownEvent) && options.includeReplyFallback !== false && events.length === 0) {
 			const latestRoomID = repliedToEventRow ? select("channel_room", "room_id", {channel_id: repliedToEventRow.channel_id}).pluck().get() : null
 			if (latestRoomID !== repliedToEventRow?.room_id) repliedToEventInDifferentRoom = true
 
@@ -546,6 +583,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 			if (repliedToEventInDifferentRoom || repliedToUnknownEvent) {
 				let referenced = message.referenced_message
 				if (!referenced) { // backend couldn't be bothered to dereference the message, have to do it ourselves
+					assert(message.message_reference?.message_id)
 					referenced = await discord.snow.channel.getChannelMessage(message.message_reference.channel_id, message.message_reference.message_id)
 				}
 
@@ -564,7 +602,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 
 				// Content
 				let repliedToContent = referenced.content
-				if (repliedToContent?.match(/^(-# )?> (-# )?<:L1:/)) {
+				if (repliedToContent?.match(/^(-# )?> (-# )?<?:L1:/)) {
 					// If the Discord user is replying to a Matrix user's reply, the fallback is going to contain the emojis and stuff from the bridged rep of the Matrix user's reply quote.
 					// Need to remove that previous reply rep from this fallback body. The fallbody body should only contain the Matrix user's actual message.
 					//                                            ┌──────A─────┐       A reply rep starting with >quote or -#smalltext >quote. Match until the end of the line.
@@ -592,6 +630,12 @@ async function messageToEvent(message, guild, options = {}, di) {
 			}
 		}
 
+		if (isInteraction && !isThinkingInteraction && events.length === 0) {
+			const formattedInteraction = getFormattedInteraction(interaction, false)
+			body = `${formattedInteraction.body}\n${body}`
+			html = `${formattedInteraction.html}${html}`
+		}
+
 		const newTextMessageEvent = {
 			$type: "m.room.message",
 			"m.mentions": mentions,
@@ -622,15 +666,25 @@ async function messageToEvent(message, guild, options = {}, di) {
 		message.content = `added a new emoji, ${message.content} :${name}:`
 	}
 
+	// Send Klipy GIFs in customised form
+	let isKlipyGIF = false
+	let isOnlyKlipyGIF = false
+	if (message.embeds?.length === 1 && message.embeds[0].provider?.name === "Klipy" && message.embeds[0].video?.url) {
+		isKlipyGIF = true
+		if (message.content.match(/^https?:\/\/klipy\.com[^ \n]+$/)) {
+			isOnlyKlipyGIF = true
+		}
+	}
+
 	// Forwarded content appears first
-	if (message.message_reference?.type === DiscordTypes.MessageReferenceType.Forward && message.message_snapshots?.length) {
+	if (message.message_reference?.type === DiscordTypes.MessageReferenceType.Forward && message.message_reference.message_id && message.message_snapshots?.length) {
 		// Forwarded notice
 		const row = await getHistoricalEventRow(message.message_reference.message_id, message.message_reference.channel_id)
 		const room = select("channel_room", ["room_id", "name", "nick"], {channel_id: message.message_reference.channel_id}).get()
 		const forwardedNotice = new mxUtils.MatrixStringBuilder()
 		if (room) {
 			const roomName = room && (room.nick || room.name)
-			if ("event_id" in row) {
+			if (row && "event_id" in row) {
 				const via = await getViaServersMemo(row.room_id)
 				forwardedNotice.addLine(
 					`[🔀 Forwarded from #${roomName}]`,
@@ -673,26 +727,42 @@ async function messageToEvent(message, guild, options = {}, di) {
 		events.push(...forwardedEvents)
 	}
 
+	if (isThinkingInteraction) {
+		const formattedInteraction = getFormattedInteraction(interaction, true)
+		await addTextEvent(formattedInteraction.body, formattedInteraction.html, "m.notice")
+	}
+
 	// Then text content
-	if (message.content) {
+	if (message.content && !isOnlyKlipyGIF && !isThinkingInteraction) {
 		// Mentions scenario 3: scan the message content for written @mentions of matrix users. Allows for up to one space between @ and mention.
-		const matches = [...message.content.matchAll(/@ ?([a-z0-9._]+)\b/gi)]
-		if (options.scanTextForMentions !== false && matches.length && matches.some(m => m[1].match(/[a-z]/i) && m[1] !== "everyone" && m[1] !== "here")) {
-			const writtenMentionsText = matches.map(m => m[1].toLowerCase())
-			const roomID = select("channel_room", "room_id", {channel_id: message.channel_id}).pluck().get()
-			assert(roomID)
-			const {joined} = await di.api.getJoinedMembers(roomID)
-			for (const [mxid, member] of Object.entries(joined)) {
-				if (!userRegex.some(rx => mxid.match(rx))) {
-					const localpart = mxid.match(/@([^:]*)/)
-					assert(localpart)
-					const displayName = member.display_name || localpart[1]
-					if (writtenMentionsText.includes(localpart[1].toLowerCase()) || writtenMentionsText.includes(displayName.toLowerCase())) addMention(mxid)
+		let content = message.content
+		if (options.scanTextForMentions !== false) {
+			const matches = [...content.matchAll(/(@ ?)([a-z0-9_.#$][^@\n]+)/gi)]
+			for (let i = matches.length; i--;) {
+				const m = matches[i]
+				const prefix = m[1]
+				const maximumWrittenSection = m[2].toLowerCase()
+				if (m.index > 0 && !content[m.index-1].match(/ |\(|\n/)) continue // must have space before it
+				if (maximumWrittenSection.match(/^everyone\b/) || maximumWrittenSection.match(/^here\b/)) continue // ignore @everyone/@here
+
+				var roomID = roomID ?? select("channel_room", "room_id", {channel_id: message.channel_id}).pluck().get()
+				assert(roomID)
+				var pjr = pjr ?? findMentions.processJoined(Object.entries((await di.api.getJoinedMembers(roomID)).joined).map(([mxid, ev]) => ({mxid, displayname: ev.display_name})))
+
+				const found = findMentions.findMention(pjr, maximumWrittenSection, m.index, prefix, content)
+				if (found) {
+					addMention(found.mxid)
+					content = found.newContent
 				}
 			}
 		}
 
-		const {body, html} = await transformContent(message.content)
+		// Scan the content for emojihax and replace them with real emojis
+		content = content.replaceAll(/\[([a-zA-Z0-9_-]{2,32})(?:~[0-9]+)?\]\(https:\/\/cdn\.discordapp\.com\/emojis\/([0-9]+)\.[^ \n)`]+\)/g, (_, name, id) => {
+			return `<:${name}:${id}>`
+		})
+
+		const {body, html} = await transformContent(content)
 		await addTextEvent(body, html, msgtype)
 	}
 
@@ -741,7 +811,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 
 	// Then attachments
 	if (message.attachments) {
-		const attachmentEvents = await Promise.all(message.attachments.map(attachmentToEvent.bind(null, mentions)))
+		const attachmentEvents = await Promise.all(message.attachments.map(attachment => attachmentToEvent(mentions, attachment)))
 
 		// Try to merge attachment events with the previous event
 		// This means that if the attachments ended up as a text link, and especially if there were many of them, the events will be joined together.
@@ -756,6 +826,106 @@ async function messageToEvent(message, guild, options = {}, di) {
 		}
 	}
 
+	// Then components
+	if (message.components?.length) {
+		const stack = new mxUtils.MatrixStringBuilderStack()
+		/** @param {DiscordTypes.APIMessageComponent} component */
+		async function processComponent(component) {
+			// Standalone components
+			if (component.type === DiscordTypes.ComponentType.TextDisplay) {
+				const {body, html} = await transformContent(component.content)
+				stack.msb.addParagraph(body, html)
+			}
+			else if (component.type === DiscordTypes.ComponentType.Separator) {
+				stack.msb.addParagraph("----", "<hr>")
+			}
+			else if (component.type === DiscordTypes.ComponentType.File) {
+				/** @type {{[k in keyof DiscordTypes.APIUnfurledMediaItem]-?: NonNullable<DiscordTypes.APIUnfurledMediaItem[k]>}} */ // @ts-ignore
+				const file = component.file
+				assert(component.name && component.size && file.content_type)
+				const ev = await attachmentToEvent({}, {...file, filename: component.name, size: component.size}, true)
+				stack.msb.addLine(ev.body, ev.formatted_body)
+			}
+			else if (component.type === DiscordTypes.ComponentType.MediaGallery) {
+				const description = component.items.length === 1 ? component.items[0].description || "Image:" : "Image gallery:"
+				const images = component.items.map(item => {
+					const publicURL = dUtils.getPublicUrlForCdn(item.media.url)
+					return {
+						url: publicURL,
+						estimatedName: item.media.url.match(/\/([^/?]+)(\?|$)/)?.[1] || publicURL
+					}
+				})
+				stack.msb.addLine(`🖼️ ${description} ${images.map(i => i.url).join(", ")}`, tag`🖼️ ${description} $${images.map(i => tag`<a href="${i.url}">${i.estimatedName}</a>`).join(", ")}`)
+			}
+			// string select, text input, user select, role select, mentionable select, channel select
+
+			// Components that can have things nested
+			else if (component.type === DiscordTypes.ComponentType.Container) {
+				// May contain action row, text display, section, media gallery, separator, file
+				stack.bump()
+				for (const innerComponent of component.components) {
+					await processComponent(innerComponent)
+				}
+				let {body, formatted_body} = stack.shift().get()
+				body = body.split("\n").map(l => "| " + l).join("\n")
+				formatted_body = `<blockquote>${formatted_body}</blockquote>`
+				if (stack.msb.body) stack.msb.body += "\n\n"
+				stack.msb.add(body, formatted_body)
+			}
+			else if (component.type === DiscordTypes.ComponentType.Section) {
+				// May contain text display, possibly more in the future
+				// Accessory may be button or thumbnail
+				stack.bump()
+				for (const innerComponent of component.components) {
+					await processComponent(innerComponent)
+				}
+				if (component.accessory) {
+					stack.bump()
+					await processComponent(component.accessory)
+					const {body, formatted_body} = stack.shift().get()
+					stack.msb.addLine(body, formatted_body)
+				}
+				const {body, formatted_body} = stack.shift().get()
+				stack.msb.addParagraph(body, formatted_body)
+			}
+			else if (component.type === DiscordTypes.ComponentType.ActionRow) {
+				const linkButtons = component.components.filter(c => c.type === DiscordTypes.ComponentType.Button && c.style === DiscordTypes.ButtonStyle.Link)
+				if (linkButtons.length) {
+					stack.msb.addLine("")
+					for (const linkButton of linkButtons) {
+						await processComponent(linkButton)
+					}
+				}
+			}
+			// Components that can only be inside things
+			else if (component.type === DiscordTypes.ComponentType.Thumbnail) {
+				// May only be a section accessory
+				stack.msb.add(`🖼️ ${component.media.url}`, tag`🖼️ <a href="${component.media.url}">${component.media.url}</a>`)
+			}
+			else if (component.type === DiscordTypes.ComponentType.Button) {
+				// May only be a section accessory or in an action row (up to 5)
+				if (component.style === DiscordTypes.ButtonStyle.Link) {
+					if (component.label) {
+						stack.msb.add(`[${component.label} ${component.url}] `, tag`<a href="${component.url}">${component.label}</a> `)
+					} else {
+						stack.msb.add(component.url)
+					}
+				}
+			}
+
+			// Not handling file upload or label because they are modal-only components
+		}
+
+		for (const component of message.components) {
+			await processComponent(component)
+		}
+
+		const {body, formatted_body} = stack.msb.get()
+		if (body.trim().length) {
+			await addTextEvent(body, formatted_body, "m.text")
+		}
+	}
+
 	// Then polls
 	if (message.poll) {
 		const pollEvent = await pollToEvent(message.poll)
@@ -773,8 +943,12 @@ async function messageToEvent(message, guild, options = {}, di) {
 			continue // Matrix's own URL previews are fine for images.
 		}
 
+		if (embed.type === "video" && embed.video?.url && !embed.title && message.content.includes(embed.video.url)) {
+			continue // Doesn't add extra information and the direct video URL is already there.
+		}
+
 		if (embed.type === "poll_result") {
-			// The code here is only for the message to be bridged to Matrix. Dealing with the Discord-side updates is in actions/poll-close.js.
+			// The code here is only for the message to be bridged to Matrix. Dealing with the Discord-side updates is in d2m/actions/poll-end.js.
 		}
 
 		if (embed.url?.startsWith("https://discord.com/")) {
@@ -791,6 +965,21 @@ async function messageToEvent(message, guild, options = {}, di) {
 		// Start building up a replica ("rep") of the embed in Discord-markdown format, which we will convert into both plaintext and formatted body at once
 		const rep = new mxUtils.MatrixStringBuilder()
 
+		if (isKlipyGIF) {
+			assert(embed.video?.url)
+			rep.add("[GIF] ", "➿ ")
+			if (embed.title) {
+				rep.add(`${embed.title} ${embed.video.url}`, tag`<a href="${embed.video.url}">${embed.title}</a>`)
+			} else {
+				rep.add(embed.video.url)
+			}
+
+			let {body, formatted_body: html} = rep.get()
+			html = `<blockquote>${html}</blockquote>`
+			await addTextEvent(body, html, "m.text")
+			continue
+		}
+
 		// Provider
 		if (embed.provider?.name && embed.provider.name !== "Tenor") {
 			if (embed.provider.url) {
@@ -900,7 +1089,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 	// Strip formatted_body where equivalent to body
 	if (!options.alwaysReturnFormattedBody) {
 		for (const event of events) {
-			if (["m.text", "m.notice"].includes(event.msgtype) && event.body === event.formatted_body) {
+			if (event.$type === "m.room.message" && "msgtype" in event && ["m.text", "m.notice"].includes(event.msgtype) && event.body === event.formatted_body) {
 				delete event.format
 				delete event.formatted_body
 			}

src/d2m/converters/message-to-event.test.components.js

@@ -0,0 +1,79 @@
+const {test} = require("supertape")
+const {messageToEvent} = require("./message-to-event")
+const data = require("../../../test/data")
+
+test("message2event components: pk question mark output", async t => {
+	const events = await messageToEvent(data.message_with_components.pk_question_mark_response, data.guild.general, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		body:
+			"| ### Lillith (INX)"
+			+ "\n| "
+			+ "\n| **Display name:** Lillith (she/her)"
+			+ "\n| **Pronouns:** She/Her"
+			+ "\n| **Message count:** 3091"
+			+ "\n| 🖼️ https://files.inx.moe/p/cdn/lillith.webp"
+			+ "\n| "
+			+ "\n| ----"
+			+ "\n| "
+			+ "\n| **Proxy tags:**"
+			+ "\n| ``l;text``"
+			+ "\n| ``l:text``"
+			+ "\n| ``l.text``"
+			+ "\n| ``textl.``"
+			+ "\n| ``textl;``"
+			+ "\n| ``textl:``"
+			+ "\n"
+			+ "\n-# System ID: `xffgnx` ∙ Member ID: `pphhoh`"
+			+ "\n-# Created: 2025-12-31 03:16:45 UTC"
+			+ "\n[View on dashboard https://dash.pluralkit.me/profile/m/pphhoh] "
+			+ "\n"
+			+ "\n----"
+			+ "\n"
+			+ "\n| **System:** INX (`xffgnx`)"
+			+ "\n| **Member:** Lillith (`pphhoh`)"
+			+ "\n| **Sent by:** infinidoge1337 (@unknown-user:)"
+			+ "\n| "
+			+ "\n| **Account Roles (7)**"
+			+ "\n| §b, !, ‼, Ears Port Ping, Ears Update Ping, Yttr Ping, unsup Ping"
+			+ "\n| 🖼️ https://files.inx.moe/p/cdn/lillith.webp"
+			+ "\n| "
+			+ "\n| ----"
+			+ "\n| "
+			+ "\n| Same hat"
+			+ "\n| 🖼️ Image: https://bridge.example.org/download/discordcdn/934955898965729280/1466556006527012987/image.png"
+			+ "\n"
+			+ "\n-# Original Message ID: 1466556003645657118 · <t:1769724599:f>",
+		format: "org.matrix.custom.html",
+		formatted_body: "<blockquote>"
+			+ "<h3>Lillith (INX)</h3>"
+			+ "<p><strong>Display name:</strong> Lillith (she/her)"
+			+ "<br><strong>Pronouns:</strong> She/Her"
+			+ "<br><strong>Message count:</strong> 3091</p>"
+			+ `🖼️ <a href="https://files.inx.moe/p/cdn/lillith.webp">https://files.inx.moe/p/cdn/lillith.webp</a>`
+			+ "<hr>"
+			+ "<p><strong>Proxy tags:</strong>"
+			+ "<br><code>l;text</code>"
+			+ "<br><code>l:text</code>"
+			+ "<br><code>l.text</code>"
+			+ "<br><code>textl.</code>"
+			+ "<br><code>textl;</code>"
+			+ "<br><code>textl:</code></p></blockquote>"
+			+ "<p><sub>System ID: <code>xffgnx</code> ∙ Member ID: <code>pphhoh</code></sub><br>"
+			+ "<sub>Created: 2025-12-31 03:16:45 UTC</sub></p>"
+			+ `<a href="https://dash.pluralkit.me/profile/m/pphhoh">View on dashboard</a> `
+			+ "<hr>"
+			+ "<blockquote><p><strong>System:</strong> INX (<code>xffgnx</code>)"
+			+ "<br><strong>Member:</strong> Lillith (<code>pphhoh</code>)"
+			+ "<br><strong>Sent by:</strong> infinidoge1337 (@unknown-user:)"
+			+ "<br><br><strong>Account Roles (7)</strong>"
+			+ "<br>§b, !, ‼, Ears Port Ping, Ears Update Ping, Yttr Ping, unsup Ping</p>"
+			+ `🖼️ <a href="https://files.inx.moe/p/cdn/lillith.webp">https://files.inx.moe/p/cdn/lillith.webp</a>`
+			+ "<hr>"
+			+ "<p>Same hat</p>"
+			+ `🖼️ Image: <a href="https://bridge.example.org/download/discordcdn/934955898965729280/1466556006527012987/image.png">image.png</a></blockquote>`
+			+ "<p><sub>Original Message ID: 1466556003645657118 · &lt;t:1769724599:f&gt;</sub></p>",
+		"m.mentions": {},
+		msgtype: "m.text",
+	}])
+})

src/d2m/converters/message-to-event.test.embeds.js

@@ -4,24 +4,31 @@ const data = require("../../../test/data")
 const {mockGetEffectivePower} = require("../../matrix/utils.test")
 const {db} = require("../../passthrough")
 
+test("message2event embeds: interaction loading", async t => {
+	const events = await messageToEvent(data.interaction_message.thinking_interaction, data.guild.general, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		body: "↪️ Brad used `/stats` — interaction loading...",
+		format: "org.matrix.custom.html",
+		formatted_body: "<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">Brad</a> used <code>/stats</code> — interaction loading...</sub></blockquote>",
+		"m.mentions": {},
+		msgtype: "m.notice",
+	}])
+})
+
 test("message2event embeds: nothing but a field", async t => {
 	const events = await messageToEvent(data.message_with_embeds.nothing_but_a_field, data.guild.general, {})
 	t.deepEqual(events, [{
-		$type: "m.room.message",
-		body: "> ↪️ @papiophidian: used `/stats`",
-		format: "org.matrix.custom.html",
-		formatted_body: "<blockquote>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">@papiophidian</a> used <code>/stats</code></blockquote>",
-		"m.mentions": {},
-		msgtype: "m.text",
-	}, {
 		$type: "m.room.message",
 		"m.mentions": {},
 		msgtype: "m.notice",
-		body: "| ### Amanda 🎵#2192 :online:"
+		body: "↪️ PapiOphidian used `/stats`"
+			+ "\n| ### Amanda 🎵#2192 :online:"
 			+ "\n| willow tree, branch 0"
 			+ "\n| **❯ Uptime:**\n| 3m 55s\n| **❯ Memory:**\n| 64.45MB",
 		format: "org.matrix.custom.html",
-		formatted_body: '<blockquote><p><strong>Amanda 🎵#2192 <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/LCEqjStXCxvRQccEkuslXEyZ\" title=\":online:\" alt=\":online:\">'
+		formatted_body: '<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">PapiOphidian</a> used <code>/stats</code></sub></blockquote>'
+			+ '<blockquote><p><strong>Amanda 🎵#2192 <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/LCEqjStXCxvRQccEkuslXEyZ\" title=\":online:\" alt=\":online:\">'
 			+ '<br>willow tree, branch 0</strong>'
 			+ '<br><strong>❯ Uptime:</strong><br>3m 55s'
 			+ '<br><strong>❯ Memory:</strong><br>64.45MB</p></blockquote>'
@@ -144,18 +151,13 @@ test("message2event embeds: crazy html is all escaped", async t => {
 test("message2event embeds: title without url", async t => {
 	const events = await messageToEvent(data.message_with_embeds.title_without_url, data.guild.general)
 	t.deepEqual(events, [{
-		$type: "m.room.message",
-		body: "> ↪️ @papiophidian: used `/stats`",
-		format: "org.matrix.custom.html",
-		formatted_body: "<blockquote>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">@papiophidian</a> used <code>/stats</code></blockquote>",
-		"m.mentions": {},
-		msgtype: "m.text",
-	}, {
 		$type: "m.room.message",
 		msgtype: "m.notice",
-		body: "| ## Hi, I'm Amanda!\n| \n| I condone pirating music!",
+		body: "↪️ PapiOphidian used `/stats`"
+			+ "\n| ## Hi, I'm Amanda!\n| \n| I condone pirating music!",
 		format: "org.matrix.custom.html",
-		formatted_body: `<blockquote><p><strong>Hi, I'm Amanda!</strong></p><p>I condone pirating music!</p></blockquote>`,
+		formatted_body: '<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">PapiOphidian</a> used <code>/stats</code></sub></blockquote>'
+			+ `<blockquote><p><strong>Hi, I'm Amanda!</strong></p><p>I condone pirating music!</p></blockquote>`,
 		"m.mentions": {}
 	}])
 })
@@ -163,18 +165,13 @@ test("message2event embeds: title without url", async t => {
 test("message2event embeds: url without title", async t => {
 	const events = await messageToEvent(data.message_with_embeds.url_without_title, data.guild.general)
 	t.deepEqual(events, [{
-		$type: "m.room.message",
-		body: "> ↪️ @papiophidian: used `/stats`",
-		format: "org.matrix.custom.html",
-		formatted_body: "<blockquote>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">@papiophidian</a> used <code>/stats</code></blockquote>",
-		"m.mentions": {},
-		msgtype: "m.text",
-	}, {
 		$type: "m.room.message",
 		msgtype: "m.notice",
-		body: "| I condone pirating music!",
+		body: "↪️ PapiOphidian used `/stats`"
+			+ "\n| I condone pirating music!",
 		format: "org.matrix.custom.html",
-		formatted_body: `<blockquote><p>I condone pirating music!</p></blockquote>`,
+		formatted_body: '<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">PapiOphidian</a> used <code>/stats</code></sub></blockquote>'
+			+ `<blockquote><p>I condone pirating music!</p></blockquote>`,
 		"m.mentions": {}
 	}])
 })
@@ -182,18 +179,13 @@ test("message2event embeds: url without title", async t => {
 test("message2event embeds: author without url", async t => {
 	const events = await messageToEvent(data.message_with_embeds.author_without_url, data.guild.general)
 	t.deepEqual(events, [{
-		$type: "m.room.message",
-		body: "> ↪️ @papiophidian: used `/stats`",
-		format: "org.matrix.custom.html",
-		formatted_body: "<blockquote>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">@papiophidian</a> used <code>/stats</code></blockquote>",
-		"m.mentions": {},
-		msgtype: "m.text",
-	}, {
 		$type: "m.room.message",
 		msgtype: "m.notice",
-		body: "| ## Amanda\n| \n| I condone pirating music!",
+		body: "↪️ PapiOphidian used `/stats`"
+			+ "\n| ## Amanda\n| \n| I condone pirating music!",
 		format: "org.matrix.custom.html",
-		formatted_body: `<blockquote><p><strong>Amanda</strong></p><p>I condone pirating music!</p></blockquote>`,
+		formatted_body: '<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">PapiOphidian</a> used <code>/stats</code></sub></blockquote>'
+			+ `<blockquote><p><strong>Amanda</strong></p><p>I condone pirating music!</p></blockquote>`,
 		"m.mentions": {}
 	}])
 })
@@ -201,18 +193,13 @@ test("message2event embeds: author without url", async t => {
 test("message2event embeds: author url without name", async t => {
 	const events = await messageToEvent(data.message_with_embeds.author_url_without_name, data.guild.general)
 	t.deepEqual(events, [{
-		$type: "m.room.message",
-		body: "> ↪️ @papiophidian: used `/stats`",
-		format: "org.matrix.custom.html",
-		formatted_body: "<blockquote>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">@papiophidian</a> used <code>/stats</code></blockquote>",
-		"m.mentions": {},
-		msgtype: "m.text",
-	}, {
 		$type: "m.room.message",
 		msgtype: "m.notice",
-		body: "| I condone pirating music!",
+		body: "↪️ PapiOphidian used `/stats`"
+			+ "\n| I condone pirating music!",
 		format: "org.matrix.custom.html",
-		formatted_body: `<blockquote><p>I condone pirating music!</p></blockquote>`,
+		formatted_body: '<blockquote><sub>↪️ <a href=\"https://matrix.to/#/@_ooye_papiophidian:cadence.moe\">PapiOphidian</a> used <code>/stats</code></sub></blockquote>'
+			+ `<blockquote><p>I condone pirating music!</p></blockquote>`,
 		"m.mentions": {}
 	}])
 })
@@ -346,6 +333,18 @@ test("message2event embeds: tenor gif should show a video link without a provide
 	}])
 })
 
+test("message2event embeds: klipy gif should send in customised format", async t => {
+	const events = await messageToEvent(data.message_with_embeds.klipy_gif, data.guild.general, {}, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		msgtype: "m.text",
+		body: "[GIF] Cute Corgi Waddle https://static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/7ndEhcilPNKJ8O.mp4",
+		format: "org.matrix.custom.html",
+		formatted_body: "<blockquote>➿ <a href=\"https://static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/7ndEhcilPNKJ8O.mp4\">Cute Corgi Waddle</a></blockquote>",
+		"m.mentions": {}
+	}])
+})
+
 test("message2event embeds: if discord creates an embed preview for a discord channel link, don't copy that embed", async t => {
 	const events = await messageToEvent(data.message_with_embeds.discord_server_included_punctuation_bad_discord, data.guild.general, {}, {
 		api: {

src/d2m/converters/message-to-event.test.js

@@ -789,11 +789,13 @@ test("message2event: simple written @mention for matrix user", async t => {
 			]
 		},
 		msgtype: "m.text",
-		body: "@ash do you need anything from the store btw as I'm heading there after gym"
+		body: "[@ash](https://matrix.to/#/@she_who_brings_destruction:cadence.moe) do you need anything from the store btw as I'm heading there after gym",
+		format: "org.matrix.custom.html",
+		formatted_body: `<a href="https://matrix.to/#/@she_who_brings_destruction:cadence.moe">@ash</a> do you need anything from the store btw as I'm heading there after gym`
 	}])
 })
 
-test("message2event: advanced written @mentions for matrix users", async t => {
+test("message2event: many written @mentions for matrix users", async t => {
 	let called = 0
 	const events = await messageToEvent(data.message.advanced_written_at_mention_for_matrix, data.guild.general, {}, {
 		api: {
@@ -831,16 +833,186 @@ test("message2event: advanced written @mentions for matrix users", async t => {
 		$type: "m.room.message",
 		"m.mentions": {
 			user_ids: [
-				"@cadence:cadence.moe",
-				"@huckleton:cadence.moe"
+				"@huckleton:cadence.moe",
+				"@cadence:cadence.moe"
 			]
 		},
 		msgtype: "m.text",
-		body: "@Cadence, tell me about @Phil, the creator of the Chin Trick, who has become ever more powerful under the mentorship of @botrac4r and @huck"
+		body: "[@Cadence](https://matrix.to/#/@cadence:cadence.moe), tell me about @Phil, the creator of the Chin Trick, who has become ever more powerful under the mentorship of @botrac4r and [@huck](https://matrix.to/#/@huckleton:cadence.moe)",
+		format: "org.matrix.custom.html",
+		formatted_body: `<a href="https://matrix.to/#/@cadence:cadence.moe">@Cadence</a>, tell me about @Phil, the creator of the Chin Trick, who has become ever more powerful under the mentorship of @botrac4r and <a href="https://matrix.to/#/@huckleton:cadence.moe">@huck</a>`
 	}])
 	t.equal(called, 1, "should only look up the member list once")
 })
 
+test("message2event: written @mentions may match part of the name", async t => {
+	let called = 0
+	const events = await messageToEvent({
+		...data.message.advanced_written_at_mention_for_matrix,
+		content: "I wonder if @cadence saw this?"
+	}, data.guild.general, {}, {
+		api: {
+			async getJoinedMembers(roomID) {
+				called++
+				t.equal(roomID, "!kLRqKKUQXcibIMtOpl:cadence.moe")
+				return new Promise(resolve => {
+					setTimeout(() => {
+						resolve({
+							joined: {
+								"@secret:cadence.moe": {
+									display_name: "cadence [they]",
+									avatar_url: "whatever"
+								},
+								"@huckleton:cadence.moe": {
+									display_name: "huck",
+									avatar_url: "whatever"
+								},
+								"@_ooye_botrac4r:cadence.moe": {
+									display_name: "botrac4r",
+									avatar_url: "whatever"
+								},
+								"@_ooye_bot:cadence.moe": {
+									display_name: "Out Of Your Element",
+									avatar_url: "whatever"
+								}
+							}
+						})
+					})
+				})
+			}
+		}
+	})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.mentions": {
+			user_ids: [
+				"@secret:cadence.moe",
+			]
+		},
+		msgtype: "m.text",
+		body: "I wonder if [@cadence](https://matrix.to/#/@secret:cadence.moe) saw this?",
+		format: "org.matrix.custom.html",
+		formatted_body: `I wonder if <a href="https://matrix.to/#/@secret:cadence.moe">@cadence</a> saw this?`
+	}])
+})
+
+test("message2event: written @mentions may match part of the mxid", async t => {
+	let called = 0
+	const events = await messageToEvent({
+		...data.message.advanced_written_at_mention_for_matrix,
+		content: "I wonder if @huck saw this?"
+	}, data.guild.general, {}, {
+		api: {
+			async getJoinedMembers(roomID) {
+				called++
+				t.equal(roomID, "!kLRqKKUQXcibIMtOpl:cadence.moe")
+				return new Promise(resolve => {
+					setTimeout(() => {
+						resolve({
+							joined: {
+								"@cadence:cadence.moe": {
+									display_name: "cadence [they]",
+									avatar_url: "whatever"
+								},
+								"@huckleton:cadence.moe": {
+									display_name: "wa",
+									avatar_url: "whatever"
+								},
+								"@_ooye_botrac4r:cadence.moe": {
+									display_name: "botrac4r",
+									avatar_url: "whatever"
+								},
+								"@_ooye_bot:cadence.moe": {
+									display_name: "Out Of Your Element",
+									avatar_url: "whatever"
+								}
+							}
+						})
+					})
+				})
+			}
+		}
+	})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.mentions": {
+			user_ids: [
+				"@huckleton:cadence.moe",
+			]
+		},
+		msgtype: "m.text",
+		body: "I wonder if [@huck](https://matrix.to/#/@huckleton:cadence.moe) saw this?",
+		format: "org.matrix.custom.html",
+		formatted_body: `I wonder if <a href="https://matrix.to/#/@huckleton:cadence.moe">@huck</a> saw this?`
+	}])
+})
+
+test("message2event: written @mentions do not match in URLs", async t => {
+	const events = await messageToEvent({
+		...data.message.advanced_written_at_mention_for_matrix,
+		content: "the fucking around with pixel composer continues https://pub.mastodon.sleeping.town/@exa/116037641900024965"
+	}, data.guild.general, {}, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.mentions": {},
+		msgtype: "m.text",
+		body: "the fucking around with pixel composer continues https://pub.mastodon.sleeping.town/@exa/116037641900024965",
+		format: "org.matrix.custom.html",
+		formatted_body: `the fucking around with pixel composer continues <a href="https://pub.mastodon.sleeping.town/@exa/116037641900024965">https://pub.mastodon.sleeping.town/@exa/116037641900024965</a>`
+	}])
+})
+
+test("message2event: entire message may match elaborate display name", async t => {
+	let called = 0
+	const events = await messageToEvent({
+		...data.message.advanced_written_at_mention_for_matrix,
+		content: "@Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆"
+	}, data.guild.general, {}, {
+		api: {
+			async getJoinedMembers(roomID) {
+				called++
+				t.equal(roomID, "!kLRqKKUQXcibIMtOpl:cadence.moe")
+				return new Promise(resolve => {
+					setTimeout(() => {
+						resolve({
+							joined: {
+								"@wa:cadence.moe": {
+									display_name: "Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆",
+									avatar_url: "whatever"
+								},
+								"@huckleton:cadence.moe": {
+									display_name: "huck",
+									avatar_url: "whatever"
+								},
+								"@_ooye_botrac4r:cadence.moe": {
+									display_name: "botrac4r",
+									avatar_url: "whatever"
+								},
+								"@_ooye_bot:cadence.moe": {
+									display_name: "Out Of Your Element",
+									avatar_url: "whatever"
+								}
+							}
+						})
+					})
+				})
+			}
+		}
+	})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.mentions": {
+			user_ids: [
+				"@wa:cadence.moe",
+			]
+		},
+		msgtype: "m.text",
+		body: "[@Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆](https://matrix.to/#/@wa:cadence.moe)",
+		format: "org.matrix.custom.html",
+		formatted_body: `<a href="https://matrix.to/#/@wa:cadence.moe">@Cadence, Maid of Creation, Eye of Clarity, Empress of Hope ☆</a>`
+	}])
+})
+
 test("message2event: spoilers are removed from plaintext body", async t => {
 	const events = await messageToEvent({
 		content: "||**beatrice**||"
@@ -999,6 +1171,18 @@ test("message2event: emoji that hasn't been registered yet", async t => {
 	}])
 })
 
+test("message2event: emojihax", async t => {
+	const events = await messageToEvent(data.message.emojihax, data.guild.general, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.mentions": {},
+		msgtype: "m.text",
+		body: "I only violate the don't modify our console part of terms of service :troll:",
+		format: "org.matrix.custom.html",
+		formatted_body: `I only violate the don't modify our console part of terms of service <img data-mx-emoticon height="32" src="mxc://cadence.moe/bvVJFgOIyNcAknKCbmaHDktG" title=":troll:" alt=":troll:">`
+	}])
+})
+
 test("message2event: emoji triple long name", async t => {
 	const events = await messageToEvent(data.message.emoji_triple_long_name, data.guild.general, {})
 	t.deepEqual(events, [{
@@ -1605,3 +1789,18 @@ test("message2event: multiple-choice poll", async t => {
 		"org.matrix.msc1767.text": "more than one answer allowed\n1. [😭] no\n2. oh no\n3. oh noooooo"
 	}])
 })
+
+test("message2event: smalltext from regular user", async t => {
+	const events = await messageToEvent({
+		content: "-# hmm",
+		author: {
+			bot: false
+		}
+	})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		msgtype: "m.text",
+		"m.mentions": {},
+		body: "...hmm"
+	}])
+})

src/d2m/converters/user-to-mxid.js

@@ -20,10 +20,10 @@ const SPECIAL_USER_MAPPINGS = new Map([
 function downcaseUsername(user) {
 	// First, try to convert the username to the set of allowed characters
 	let downcased = user.username.toLowerCase()
-		// spaces to underscores...
-		.replace(/ /g, "_")
+		// spaces and slashes to underscores...
+		.replace(/[ /]/g, "_")
 		// remove disallowed characters...
-		.replace(/[^a-z0-9._=/-]*/g, "")
+		.replace(/[^a-z0-9._=-]*/g, "")
 		// remove leading and trailing dashes and underscores...
 		.replace(/(?:^[_-]*|[_-]*$)/g, "")
 	// If requested, also make the Discord user ID part of the username

src/d2m/converters/user-to-mxid.test.js

@@ -21,8 +21,12 @@ test("user2name: works on single emoji at the end", t => {
    t.equal(userToSimName({username: "Melody 🎵", discriminator: "2192"}), "melody")
 })
 
-test("user2name: works on crazy name", t => {
-   t.equal(userToSimName({username: "*** D3 &W (89) _7//-", discriminator: "0001"}), "d3_w_89__7//")
+test("user2name: works on really weird name", t => {
+   t.equal(userToSimName({username: "*** D3 &W (89) _7//-", discriminator: "0001"}), "d3_w_89__7")
+})
+
+test("user2name: treats slashes", t => {
+	t.equal(userToSimName({username: "Evil Lillith (she/her)", discriminator: "5892"}), "evil_lillith_she_her")
 })
 
 test("user2name: adds discriminator if name is unavailable (old tag format)", t => {

src/d2m/discord-packets.js

@@ -47,6 +47,7 @@ const utils = {
 
 			if (listen === "full") {
 				try {
+					interactions.registerInteractions()
 					await eventDispatcher.checkMissedExpressions(message.d)
 					await eventDispatcher.checkMissedPins(client, message.d)
 					await eventDispatcher.checkMissedMessages(client, message.d)

src/d2m/event-dispatcher.js

@@ -51,13 +51,15 @@ module.exports = {
 	 * @param {import("cloudstorm").IGatewayMessage} gatewayMessage
 	 */
 	async onError(client, e, gatewayMessage) {
+		if (gatewayMessage.t === "TYPING_START") return
+
+		matrixEventDispatcher.printError(gatewayMessage.t, "Discord", e, gatewayMessage)
+
 		const channelID = gatewayMessage.d["channel_id"]
 		if (!channelID) return
 		const roomID = select("channel_room", "room_id", {channel_id: channelID}).pluck().get()
 		if (!roomID) return
 
-		if (gatewayMessage.t === "TYPING_START") return
-
 		await matrixEventDispatcher.sendError(roomID, "Discord", gatewayMessage.t, e, gatewayMessage)
 	},
 
@@ -194,6 +196,21 @@ module.exports = {
 		await createSpace.syncSpace(guild)
 	},
 
+	/**
+	 * @param {import("./discord-client")} client
+	 * @param {DiscordTypes.GatewayGuildRoleUpdateDispatchData} data
+	 */
+	async GUILD_ROLE_UPDATE(client, data) {
+		const guild = client.guilds.get(data.guild_id)
+		if (!guild) return
+		const spaceID = select("guild_space", "space_id", {guild_id: data.guild_id}).pluck().get()
+		if (!spaceID) return
+
+		if (data.role.id === data.guild_id) { // @everyone role changed - find a way to do this more efficiently in the future to handle many role updates
+			await createSpace.syncSpaceFully(guild)
+		}
+	},
+
 	/**
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayChannelUpdateDispatchData} channelOrThread
@@ -233,7 +250,7 @@ module.exports = {
 		const roomID = select("channel_room", "room_id", {channel_id: channel.id}).pluck().get()
 		if (!roomID) return // channel wasn't being bridged in the first place
 		// @ts-ignore
-		await createRoom.unbridgeDeletedChannel(channel, guildID)
+		await createRoom.unbridgeChannel(channel, guildID)
 	},
 
 	/**
@@ -274,7 +291,7 @@ module.exports = {
 		// Based on looking at data they've sent me over the gateway, this is the best way to check for meaningful changes.
 		// If the message content is a string then it includes all interesting fields and is meaningful.
 		// Otherwise, if there are embeds, then the system generated URL preview embeds.
-		if (!(typeof data.content === "string" || "embeds" in data)) return
+		if (!(typeof data.content === "string" || "embeds" in data || "components" in data)) return
 
 		if (dUtils.isEphemeralMessage(data)) return // Ephemeral messages are for the eyes of the receiver only!
 
@@ -282,8 +299,10 @@ module.exports = {
 		const {affected, row} = await speedbump.maybeDoSpeedbump(data.channel_id, data.id)
 		if (affected) return
 
-		// Check that the sending-to room exists, and deal with Eventual Consistency(TM)
-		if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.MESSAGE_UPDATE, client, data)) return
+		if (!row) {
+			// Check that the sending-to room exists, and deal with Eventual Consistency(TM)
+			if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.MESSAGE_UPDATE, client, data)) return
+		}
 
 		/** @type {DiscordTypes.GatewayMessageCreateDispatchData} */
 		// @ts-ignore
@@ -303,8 +322,8 @@ module.exports = {
 	 */
 	async MESSAGE_REACTION_ADD(client, data) {
 		if (data.user_id === client.user.id) return // m2d reactions are added by the discord bot user - do not reflect them back to matrix.
-		if (data.emoji.name === "❓" && select("event_message", "message_id", {message_id: data.message_id, source: 0})) {
-			const guild_id = data.guild_id ?? client.channels.get(data.channel_id)["guild_id"]
+		if (data.emoji.name === "❓" && select("event_message", "message_id", {message_id: data.message_id, source: 0, part: 0}).get()) { // source 0 = matrix
+			const guild_id = data.guild_id ?? client.channels.get(data.channel_id)?.["guild_id"]
 			await Promise.all([
 				client.snow.channel.deleteReaction(data.channel_id, data.message_id, data.emoji.name).catch(() => {}),
 				// @ts-ignore - this is all you need for it to do a matrix-side lookup

src/db/migrations/0033-add-missing-profile-to-member-cache.sql

@@ -0,0 +1,5 @@
+BEGIN TRANSACTION;
+
+ALTER TABLE member_cache ADD COLUMN missing_profile INTEGER;
+
+COMMIT;

src/db/migrations/0034-slash-not-allowed-in-mxid.sql

@@ -0,0 +1,5 @@
+BEGIN TRANSACTION;
+
+DELETE FROM sim WHERE sim_name like '%/%';
+
+COMMIT;

src/db/orm-defs.d.ts

@@ -90,6 +90,7 @@ export type Models = {
 		displayname: string | null
 		avatar_url: string | null,
 		power_level: number
+		missing_profile: number | null
 	}
 
 	member_power: {

src/discord/interactions/matrix-info.js

@@ -20,7 +20,7 @@ const webGuild = sync.require("../../web/routes/guild")
  */
 async function _interact({guild_id, data}, {api}) {
 	const message = from("event_message").join("message_room", "message_id").join("historical_channel_room", "historical_room_index")
-		.select("source", "reference_channel_id", "room_id", "event_id").where({message_id: data.target_id, part: 0}).get()
+		.select("source", "reference_channel_id", "room_id", "event_id").where({message_id: data.target_id}).and("ORDER BY part").get()
 
 	if (!message) {
 		return {
@@ -54,8 +54,11 @@ async function _interact({guild_id, data}, {api}) {
 	// from Matrix
 	const event = await api.getEvent(message.room_id, message.event_id)
 	const via = await utils.getViaServersQuery(message.room_id, api)
-	const inChannels = discord.guildChannelMap.get(guild_id)
-		.map(cid => discord.channels.get(cid))
+	const channelsInGuild = discord.guildChannelMap.get(guild_id)
+	assert(channelsInGuild)
+	const inChannels = channelsInGuild
+		// @ts-ignore
+		.map(/** @returns {DiscordTypes.APIGuildChannel} */ cid => discord.channels.get(cid))
 		.sort((a, b) => webGuild._getPosition(a, discord.channels) - webGuild._getPosition(b, discord.channels))
 		.filter(channel => from("channel_room").join("member_cache", "room_id").select("mxid").where({channel_id: channel.id, mxid: event.sender}).get())
 	const matrixMember = select("member_cache", ["displayname", "avatar_url"], {room_id: message.room_id, mxid: event.sender}).get()
@@ -67,7 +70,7 @@ async function _interact({guild_id, data}, {api}) {
 				author: {
 					name,
 					url: `https://matrix.to/#/${event.sender}`,
-					icon_url: utils.getPublicUrlForMxc(matrixMember.avatar_url)
+					icon_url: utils.getPublicUrlForMxc(matrixMember?.avatar_url)
 				},
 				description: `This Matrix message was delivered to Discord by **Out Of Your Element**.\n[View on Matrix →](<https://matrix.to/#/${message.room_id}/${message.event_id}?${via}>)\n\n**User ID**: [${event.sender}](<https://matrix.to/#/${event.sender}>)`,
 				color: 0x0dbd8b,
@@ -96,7 +99,7 @@ async function dm(interaction) {
 	const channel = await discord.snow.user.createDirectMessageChannel(interaction.member.user.id)
 	const response = await _interact(interaction, {api})
 	assert(response.type === DiscordTypes.InteractionResponseType.ChannelMessageWithSource)
-	response.data.flags &= 0 // not ephemeral
+	response.data.flags = 0 & 0 // not ephemeral
 	await discord.snow.channel.createMessage(channel.id, response.data)
 }
 

src/discord/interactions/ping.js

@@ -0,0 +1,199 @@
+// @ts-check
+
+const assert = require("assert").strict
+const Ty = require("../../types")
+const DiscordTypes = require("discord-api-types/v10")
+const {discord, sync, select, from} = require("../../passthrough")
+const {id: botID} = require("../../../addbot")
+const {InteractionMethods} = require("snowtransfer")
+
+/** @type {import("../../matrix/api")} */
+const api = sync.require("../../matrix/api")
+/** @type {import("../../matrix/utils")} */
+const utils = sync.require("../../matrix/utils")
+/** @type {import("../../web/routes/guild")} */
+const webGuild = sync.require("../../web/routes/guild")
+
+/**
+ * @param {DiscordTypes.APIApplicationCommandAutocompleteGuildInteraction} interaction
+ * @param {{api: typeof api}} di
+ * @returns {AsyncGenerator<{[k in keyof InteractionMethods]?: Parameters<InteractionMethods[k]>[2]}>}
+ */
+async function* _interactAutocomplete({data, channel}, {api}) {
+	function exit() {
+		return {createInteractionResponse: {
+			/** @type {DiscordTypes.InteractionResponseType.ApplicationCommandAutocompleteResult} */
+			type: DiscordTypes.InteractionResponseType.ApplicationCommandAutocompleteResult,
+			data: {
+				choices: []
+			}
+		}}
+	}
+
+	// Check it was used in a bridged channel
+	const roomID = select("channel_room", "room_id", {channel_id: channel?.id}).pluck().get()
+	if (!roomID) return yield exit()
+
+	// Check we are in fact autocompleting the first option, the user
+	if (!data.options?.[0] || data.options[0].type !== DiscordTypes.ApplicationCommandOptionType.String || !data.options[0].focused) {
+		return yield exit()
+	}
+
+	/** @type {{displayname: string | null, mxid: string}[][]} */
+	const providedMatches = []
+
+	const input = data.options[0].value
+	if (input === "") {
+		const events = await api.getEvents(roomID, "b", {limit: 40})
+		const recents = new Set(events.chunk.map(e => e.sender))
+		const matches = select("member_cache", ["mxid", "displayname"], {room_id: roomID}, "AND displayname IS NOT NULL LIMIT 25").all()
+		matches.sort((a, b) => +recents.has(b.mxid) - +recents.has(a.mxid))
+		providedMatches.push(matches)
+	} else if (input.startsWith("@")) { // only autocomplete mxids
+		const query = input.replaceAll(/[%_$]/g, char => `$${char}`) + "%"
+		const matches = select("member_cache", ["mxid", "displayname"], {room_id: roomID}, "AND mxid LIKE ? ESCAPE '$' LIMIT 25").all(query)
+		providedMatches.push(matches)
+	} else {
+		const query = "%" + input.replaceAll(/[%_$]/g, char => `$${char}`) + "%"
+		const displaynameMatches = select("member_cache", ["mxid", "displayname"], {room_id: roomID}, "AND displayname IS NOT NULL AND displayname LIKE ? ESCAPE '$' LIMIT 25").all(query)
+		// prioritise matches closer to the start
+		displaynameMatches.sort((a, b) => {
+			let ai = a.displayname?.toLowerCase().indexOf(input.toLowerCase()) ?? -1
+			if (ai === -1) ai = 999
+			let bi = b.displayname?.toLowerCase().indexOf(input.toLowerCase()) ?? -1
+			if (bi === -1) bi = 999
+			return ai - bi
+		})
+		providedMatches.push(displaynameMatches)
+		let mxidMatches = select("member_cache", ["mxid", "displayname"], {room_id: roomID}, "AND displayname IS NOT NULL AND mxid LIKE ? ESCAPE '$' LIMIT 25").all(query)
+		mxidMatches = mxidMatches.filter(match => {
+			// don't include matches in domain part of mxid
+			if (!match.mxid.match(/^[^:]*/)?.includes(query)) return false
+			if (displaynameMatches.some(m => m.mxid === match.mxid)) return false
+			return true
+		})
+		providedMatches.push(mxidMatches)
+	}
+
+	// merge together
+	let matches = providedMatches.flat()
+
+	// don't include bot
+	matches = matches.filter(m => m.mxid !== utils.bot)
+
+	// remove duplicates and count up to 25
+	const limitedMatches = []
+	const seen = new Set()
+	for (const match of matches) {
+		if (limitedMatches.length >= 25) break
+		if (seen.has(match.mxid)) continue
+		limitedMatches.push(match)
+		seen.add(match.mxid)
+	}
+
+	yield {createInteractionResponse: {
+		type: DiscordTypes.InteractionResponseType.ApplicationCommandAutocompleteResult,
+		data: {
+			choices: limitedMatches.map(row => ({name: (row.displayname || row.mxid).slice(0, 100), value: row.mxid.slice(0, 100)}))
+		}
+	}}
+}
+
+/**
+ * @param {DiscordTypes.APIChatInputApplicationCommandGuildInteraction & {channel: DiscordTypes.APIGuildTextChannel}} interaction
+ * @param {{api: typeof api}} di
+ * @returns {AsyncGenerator<{[k in keyof InteractionMethods]?: Parameters<InteractionMethods[k]>[2]}>}
+ */
+async function* _interactCommand({data, channel, guild_id}, {api}) {
+	const roomID = select("channel_room", "room_id", {channel_id: channel.id}).pluck().get()
+	if (!roomID) {
+		return yield {createInteractionResponse: {
+			type: DiscordTypes.InteractionResponseType.ChannelMessageWithSource,
+			data: {
+				flags: DiscordTypes.MessageFlags.Ephemeral,
+				content: "This channel isn't bridged to Matrix."
+			}
+		}}
+	}
+
+	assert(data.options?.[0]?.type === DiscordTypes.ApplicationCommandOptionType.String)
+	const mxid = data.options[0].value
+	if (!mxid.match(/^@[^:]*:./)) {
+		return yield {createInteractionResponse: {
+			type: DiscordTypes.InteractionResponseType.ChannelMessageWithSource,
+			data: {
+				flags: DiscordTypes.MessageFlags.Ephemeral,
+				content: "⚠️ To use `/ping`, you must select an option from autocomplete, or type a full Matrix ID.\n> Tip: This command is not necessary. You can also ping Matrix users just by typing @their name in your message. It won't look like anything, but it does go through."
+			}
+		}}
+	}
+
+	yield {createInteractionResponse: {
+		type: DiscordTypes.InteractionResponseType.DeferredChannelMessageWithSource
+	}}
+
+	let member
+	try {
+		/** @type {Ty.Event.M_Room_Member} */
+		member = await api.getStateEvent(roomID, "m.room.member", mxid)
+	} catch (e) {}
+
+	if (!member || member.membership !== "join") {
+		const channelsInGuild = discord.guildChannelMap.get(guild_id)
+		assert(channelsInGuild)
+		const inChannels = channelsInGuild
+		// @ts-ignore
+		.map(/** @returns {DiscordTypes.APIGuildChannel} */ cid => discord.channels.get(cid))
+			.sort((a, b) => webGuild._getPosition(a, discord.channels) - webGuild._getPosition(b, discord.channels))
+			.filter(channel => from("channel_room").join("member_cache", "room_id").select("mxid").where({channel_id: channel.id, mxid}).get())
+		if (inChannels.length) {
+			return yield {editOriginalInteractionResponse: {
+				content: `That person isn't in this channel. They have only joined the following channels:\n${inChannels.map(c => `<#${c.id}>`).join(" • ")}\nYou can ask them to join this channel with \`/invite\`.`,
+			}}
+		} else {
+			return yield {editOriginalInteractionResponse: {
+				content: "That person isn't in this channel. You can invite them with `/invite`."
+			}}
+		}
+	}
+
+	yield {editOriginalInteractionResponse: {
+		content: "@" + (member.displayname || mxid)
+	}}
+
+	yield {createFollowupMessage: {
+		flags: DiscordTypes.MessageFlags.Ephemeral | DiscordTypes.MessageFlags.IsComponentsV2,
+		components: [{
+			type: DiscordTypes.ComponentType.Container,
+			components: [{
+				type: DiscordTypes.ComponentType.TextDisplay,
+				content: "Tip: This command is not necessary. You can also ping Matrix users just by typing @their name in your message. It won't look like anything, but it does go through."
+			}]
+		}]
+	}}
+}
+
+/* c8 ignore start */
+
+/** @param {(DiscordTypes.APIChatInputApplicationCommandGuildInteraction & {channel: DiscordTypes.APIGuildTextChannel}) | DiscordTypes.APIApplicationCommandAutocompleteGuildInteraction} interaction */
+async function interact(interaction) {
+	if (interaction.type === DiscordTypes.InteractionType.ApplicationCommandAutocomplete) {
+		for await (const response of _interactAutocomplete(interaction, {api})) {
+			if (response.createInteractionResponse) {
+				await discord.snow.interaction.createInteractionResponse(interaction.id, interaction.token, response.createInteractionResponse)
+			}
+		}
+	} else {
+		for await (const response of _interactCommand(interaction, {api})) {
+			if (response.createInteractionResponse) {
+				await discord.snow.interaction.createInteractionResponse(interaction.id, interaction.token, response.createInteractionResponse)
+			} else if (response.editOriginalInteractionResponse) {
+				await discord.snow.interaction.editOriginalInteractionResponse(botID, interaction.token, response.editOriginalInteractionResponse)
+			} else if (response.createFollowupMessage) {
+				await discord.snow.interaction.createFollowupMessage(botID, interaction.token, response.createFollowupMessage)
+			}
+		}
+	}
+}
+
+module.exports.interact = interact

src/discord/interactions/poll-responses.js

@@ -0,0 +1,94 @@
+// @ts-check
+
+const DiscordTypes = require("discord-api-types/v10")
+const {discord, sync, db, select, from} = require("../../passthrough")
+const {id: botID} = require("../../../addbot")
+const {InteractionMethods} = require("snowtransfer")
+
+/** @type {import("../../matrix/api")} */
+const api = sync.require("../../matrix/api")
+/** @type {import("../../m2d/converters/poll-components")} */
+const pollComponents = sync.require("../../m2d/converters/poll-components")
+const {reg} = require("../../matrix/read-registration")
+
+/**
+ * @param {number} percent
+ */
+function barChart(percent) {
+	const width = 12
+	const bars = Math.floor(percent*width)
+	return "█".repeat(bars) + "▒".repeat(width-bars)
+}
+
+/**
+ * @param {string} pollMessageID
+ * @param {boolean} isClosed
+ */
+function getCombinedResults(pollMessageID, isClosed) {
+	/** @type {{matrix_option: string, option_text: string, count: number}[]} */
+	const pollResults = db.prepare("SELECT matrix_option, option_text, seq, count(discord_or_matrix_user_id) as count FROM poll_option LEFT JOIN poll_vote USING (message_id, matrix_option) WHERE message_id = ? GROUP BY matrix_option ORDER BY seq").all(pollMessageID)
+	const combinedVotes = pollResults.reduce((a, c) => a + c.count, 0)
+	const totalVoters = db.prepare("SELECT count(DISTINCT discord_or_matrix_user_id) as count FROM poll_vote WHERE message_id = ?").pluck().get(pollMessageID)
+	const topAnswers = pollResults.toSorted((a, b) => b.count - a.count)
+
+	let messageString = ""
+	for (const option of pollResults) {
+		const medal = isClosed ? pollComponents.getMedal(topAnswers, option.count) : ""
+		const countString = `${String(option.count).padStart(String(topAnswers[0].count).length)}`
+		const votesString = option.count === 1 ? "vote " : "votes"
+		const label = medal === "🥇" ? `**${option.option_text}**` : option.option_text
+		messageString += `\`\u200b${countString} ${votesString}\u200b\` ${barChart(option.count/totalVoters)} ${label} ${medal}\n`
+	}
+
+	return {messageString, combinedVotes, totalVoters}
+}
+
+/**
+ * @param {DiscordTypes.APIMessageApplicationCommandGuildInteraction} interaction
+ * @param {{api: typeof api}} di
+ * @returns {AsyncGenerator<{[k in keyof InteractionMethods]?: Parameters<InteractionMethods[k]>[2]}>}
+ */
+async function* _interact({data}, {api}) {
+	const row = select("poll", "is_closed", {message_id: data.target_id}).get()
+
+	if (!row) {
+		return yield {createInteractionResponse: {
+			type: DiscordTypes.InteractionResponseType.ChannelMessageWithSource,
+			data: {
+				content: "This poll hasn't been bridged to Matrix.",
+				flags: DiscordTypes.MessageFlags.Ephemeral
+			}
+		}}
+	}
+
+	const {messageString} = getCombinedResults(data.target_id, !!row.is_closed)
+
+	return yield {createInteractionResponse: {
+		type: DiscordTypes.InteractionResponseType.ChannelMessageWithSource,
+		data: {
+			embeds: [{
+				author: {
+					name: "Current results including Matrix votes",
+					icon_url: `${reg.ooye.bridge_origin}/download/file/poll-star-avatar.png`
+				},
+				description: messageString
+			}],
+			flags: DiscordTypes.MessageFlags.Ephemeral
+		}
+	}}
+}
+
+/* c8 ignore start */
+
+/** @param {DiscordTypes.APIMessageApplicationCommandGuildInteraction} interaction */
+async function interact(interaction) {
+	for await (const response of _interact(interaction, {api})) {
+		if (response.createInteractionResponse) {
+			await discord.snow.interaction.createInteractionResponse(interaction.id, interaction.token, response.createInteractionResponse)
+		}
+	}
+}
+
+module.exports.interact = interact
+module.exports._interact = _interact
+module.exports.getCombinedResults = getCombinedResults

src/discord/interactions/poll.js

@@ -43,7 +43,8 @@ async function* _interact({data, message, member, user}, {api}) {
 			default: alreadySelected.includes(option.matrix_option)
 		}))
 		const checkboxGroupExtras = maxSelections === 1 && options.length > 1 ? {} : {
-			type: 22, // DiscordTypes.ComponentType.CheckboxGroup
+			/** @type {DiscordTypes.ComponentType.CheckboxGroup} */
+			type: DiscordTypes.ComponentType.CheckboxGroup,
 			min_values: 0,
 			max_values: maxSelections
 		}
@@ -58,20 +59,12 @@ async function* _interact({data, message, member, user}, {api}) {
 				}, {
 					type: DiscordTypes.ComponentType.Label,
 					label: pollRow.question_text,
-					component: /* {
-						type: 21, // DiscordTypes.ComponentType.RadioGroup
+					component: {
+						type: DiscordTypes.ComponentType.RadioGroup,
 						custom_id: "POLL_MODAL_SELECTION",
 						options,
 						required: false,
 						...checkboxGroupExtras
-					} */
-					{
-						type: DiscordTypes.ComponentType.StringSelect,
-						custom_id: "POLL_MODAL_SELECTION",
-						options,
-						required: false,
-						min_values: 0,
-						max_values: maxSelections,
 					}
 				}]
 			}
@@ -80,14 +73,15 @@ async function* _interact({data, message, member, user}, {api}) {
 
 	if (data.custom_id === "POLL_MODAL") {
 		// Clicked options via modal
-		/** @type {DiscordTypes.APIMessageStringSelectInteractionData} */ // @ts-ignore - close enough to the real thing
+		/** @type {DiscordTypes.APIModalSubmitRadioGroupComponent | DiscordTypes.APIModalSubmitCheckboxGroupComponent} */ // @ts-ignore - close enough to the real thing
 		const component = data.components[1].component
 		assert.equal(component.custom_id, "POLL_MODAL_SELECTION")
+		const values = "values" in component ? component.values : [component.value]
 
 		// Replace votes with selection
 		db.transaction(() => {
 			db.prepare("DELETE FROM poll_vote WHERE message_id = ? AND discord_or_matrix_user_id = ?").run(message.id, userID)
-			for (const option of component.values) {
+			for (const option of values) {
 				db.prepare("INSERT OR IGNORE INTO poll_vote (discord_or_matrix_user_id, message_id, matrix_option) VALUES (?, ?, ?)").run(userID, message.id, option)
 			}
 		})()

src/discord/register-interactions.js

@@ -10,64 +10,82 @@ const permissions = sync.require("./interactions/permissions.js")
 const reactions = sync.require("./interactions/reactions.js")
 const privacy = sync.require("./interactions/privacy.js")
 const poll = sync.require("./interactions/poll.js")
+const pollResponses = sync.require("./interactions/poll-responses.js")
+const ping = sync.require("./interactions/ping.js")
 
 // User must have EVERY permission in default_member_permissions to be able to use the command
 
-discord.snow.interaction.bulkOverwriteApplicationCommands(id, [{
-	name: "Matrix info",
-	contexts: [DiscordTypes.InteractionContextType.Guild],
-	type: DiscordTypes.ApplicationCommandType.Message,
-}, {
-	name: "Permissions",
-	contexts: [DiscordTypes.InteractionContextType.Guild],
-	type: DiscordTypes.ApplicationCommandType.Message,
-	default_member_permissions: String(DiscordTypes.PermissionFlagsBits.KickMembers | DiscordTypes.PermissionFlagsBits.ManageRoles)
-}, {
-	name: "Reactions",
-	contexts: [DiscordTypes.InteractionContextType.Guild],
-	type: DiscordTypes.ApplicationCommandType.Message
-}, {
-	name: "invite",
-	contexts: [DiscordTypes.InteractionContextType.Guild],
-	type: DiscordTypes.ApplicationCommandType.ChatInput,
-	description: "Invite a Matrix user to this Discord server",
-	default_member_permissions: String(DiscordTypes.PermissionFlagsBits.CreateInstantInvite),
-	options: [
-		{
-			type: DiscordTypes.ApplicationCommandOptionType.String,
-			description: "The Matrix user to invite, e.g. @username:example.org",
-			name: "user"
-		}
-	]
-}, {
-	name: "privacy",
-	contexts: [DiscordTypes.InteractionContextType.Guild],
-	type: DiscordTypes.ApplicationCommandType.ChatInput,
-	description: "Change whether Matrix users can join through direct invites, links, or the public directory.",
-	default_member_permissions: String(DiscordTypes.PermissionFlagsBits.ManageGuild),
-	options: [
-		{
-			type: DiscordTypes.ApplicationCommandOptionType.String,
-			description: "Check or set the new privacy level",
-			name: "level",
-			choices: [{
-				name: "❓️ Check the current privacy level and get more information.",
-				value: "info"
-			}, {
-				name: "🤝 Only allow joining with a direct in-app invite from another user. No shareable invite links.",
-				value: "invite"
-			}, {
-				name: "🔗 Matrix links can be created and shared like Discord's invite links. In-app invites still work.",
-				value: "link",
-			}, {
-				name: "🌏️ Publicly visible in the Matrix directory, like Server Discovery. Invites and links still work.",
-				value: "directory"
-			}]
-		}
-	]
-}]).catch(e => {
-	console.error(e)
-})
+function registerInteractions() {
+	discord.snow.interaction.bulkOverwriteApplicationCommands(id, [{
+		name: "Matrix info",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.Message,
+	}, {
+		name: "Permissions",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.Message,
+		default_member_permissions: String(DiscordTypes.PermissionFlagsBits.KickMembers | DiscordTypes.PermissionFlagsBits.ManageRoles)
+	}, {
+		name: "Responses",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.Message
+	}, {
+		name: "invite",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.ChatInput,
+		description: "Invite a Matrix user to this Discord server",
+		default_member_permissions: String(DiscordTypes.PermissionFlagsBits.CreateInstantInvite),
+		options: [
+			{
+				type: DiscordTypes.ApplicationCommandOptionType.String,
+				description: "The Matrix user to invite, e.g. @username:example.org",
+				name: "user"
+			}
+		],
+	}, {
+		name: "ping",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.ChatInput,
+		description: "Ping a Matrix user.",
+		options: [
+			{
+				type: DiscordTypes.ApplicationCommandOptionType.String,
+				description: "Display name or ID of the Matrix user",
+				name: "user",
+				autocomplete: true,
+				required: true
+			}
+		]
+	}, {
+		name: "privacy",
+		contexts: [DiscordTypes.InteractionContextType.Guild],
+		type: DiscordTypes.ApplicationCommandType.ChatInput,
+		description: "Change whether Matrix users can join through direct invites, links, or the public directory.",
+		default_member_permissions: String(DiscordTypes.PermissionFlagsBits.ManageGuild),
+		options: [
+			{
+				type: DiscordTypes.ApplicationCommandOptionType.String,
+				description: "Check or set the new privacy level",
+				name: "level",
+				choices: [{
+					name: "❓️ Check the current privacy level and get more information.",
+					value: "info"
+				}, {
+					name: "🤝 Only allow joining with a direct in-app invite from another user. No shareable invite links.",
+					value: "invite"
+				}, {
+					name: "🔗 Matrix links can be created and shared like Discord's invite links. In-app invites still work.",
+					value: "link",
+				}, {
+					name: "🌏️ Publicly visible in the Matrix directory, like Server Discovery. Invites and links still work.",
+					value: "directory"
+				}]
+			}
+		]
+	}]).catch(e => {
+		console.error(e)
+	})
+}
 
 /** @param {DiscordTypes.APIInteraction} interaction */
 async function dispatchInteraction(interaction) {
@@ -92,8 +110,16 @@ async function dispatchInteraction(interaction) {
 				await permissions.interact(interaction)
 			} else if (interactionId === "permissions_edit") {
 				await permissions.interactEdit(interaction)
-			} else if (interactionId === "Reactions") {
-				await reactions.interact(interaction)
+			} else if (interactionId === "Responses") {
+				/** @type {DiscordTypes.APIMessageApplicationCommandGuildInteraction} */ // @ts-ignore
+				const messageInteraction = interaction
+				if (select("poll", "message_id", {message_id: messageInteraction.data.target_id}).get()) {
+					await pollResponses.interact(messageInteraction)
+				} else {
+					await reactions.interact(messageInteraction)
+				}
+			} else if (interactionId === "ping") {
+				await ping.interact(interaction)
 			} else if (interactionId === "privacy") {
 				await privacy.interact(interaction)
 			} else {
@@ -123,3 +149,4 @@ async function dispatchInteraction(interaction) {
 }
 
 module.exports.dispatchInteraction = dispatchInteraction
+module.exports.registerInteractions = registerInteractions

src/m2d/actions/add-reaction.js

@@ -17,7 +17,7 @@ const retrigger = sync.require("../../d2m/actions/retrigger")
  */
 async function addReaction(event) {
 	// Wait until the corresponding channel and message have already been bridged
-	if (retrigger.eventNotFoundThenRetrigger(event.content["m.relates_to"].event_id, as.emit.bind(as, "type:m.reaction", event))) return
+	if (retrigger.eventNotFoundThenRetrigger(event.content["m.relates_to"].event_id, () => as.emit("type:m.reaction", event))) return
 
 	// These will exist because it passed retrigger
 	const row = from("event_message").join("message_room", "message_id").join("historical_channel_room", "historical_room_index")

src/m2d/actions/redact.js

@@ -58,7 +58,7 @@ async function handle(event) {
 	await removeReaction(event)
 
 	// Or, it might be for removing a message or suppressing embeds. But to do that, the message needs to be bridged first.
-	if (retrigger.eventNotFoundThenRetrigger(event.redacts, as.emit.bind(as, "type:m.room.redaction", event))) return
+	if (retrigger.eventNotFoundThenRetrigger(event.redacts, () => as.emit("type:m.room.redaction", event))) return
 
 	const row = select("event_message", ["event_type", "event_subtype", "part"], {event_id: event.redacts}).get()
 	if (row && row.event_type === "m.room.message" && row.event_subtype === "m.notice" && row.part === 1) {

src/m2d/actions/sticker.js

@@ -0,0 +1,40 @@
+// @ts-check
+
+const {Readable} = require("stream")
+const {ReadableStream} = require("stream/web")
+
+const {sync} = require("../../passthrough")
+const sharp = require("sharp")
+/** @type {import("../../matrix/api")} */
+const api = sync.require("../../matrix/api")
+/** @type {import("../../matrix/mreq")} */
+const mreq = sync.require("../../matrix/mreq")
+const streamMimeType = require("stream-mime-type")
+
+const WIDTH = 160
+const HEIGHT = 160
+/**
+ * Downloads the sticker from the web and converts to webp data.
+ * @param {string} mxc a single mxc:// URL
+ * @returns {Promise<ReadableStream>} sticker webp data, or undefined if the downloaded sticker is not valid
+ */
+async function getAndResizeSticker(mxc) {
+	const res = await api.getMedia(mxc)
+	if (res.status !== 200) {
+		const root = await res.json()
+		throw new mreq.MatrixServerError(root, {mxc})
+	}
+
+	const streamIn = Readable.fromWeb(res.body)
+	const { stream, mime } = await streamMimeType.getMimeType(streamIn)
+	const animated = ["image/gif", "image/webp"].includes(mime)
+
+	const transformer = sharp({animated: animated})
+		.resize(WIDTH, HEIGHT, {fit: "inside", background: {r: 0, g: 0, b: 0, alpha: 0}})
+		.webp()
+	stream.pipe(transformer)
+	return Readable.toWeb(transformer)
+}
+
+
+module.exports.getAndResizeSticker = getAndResizeSticker

src/m2d/converters/event-to-message.js

@@ -1,4 +1,5 @@
 // @ts-check
+/// <reference lib="dom" />
 
 const Ty = require("../../types")
 const DiscordTypes = require("discord-api-types/v10")
@@ -68,7 +69,7 @@ turndownService.escape = function (string) {
 	return string.replace(/\s+|\S+/g, part => { // match chunks of spaces or non-spaces
 		if (part.match(/\s/)) return part // don't process spaces
 
-		if (part.match(/^https?:\/\//)) {
+		if (part.match(/^<?https?:\/\//)) {
 			return part
 		} else {
 			return markdownEscapes.reduce(function (accumulator, escape) {
@@ -138,7 +139,9 @@ turndownService.addRule("inlineLink", {
 		if (node.getAttribute("data-message-id")) return `https://discord.com/channels/${node.getAttribute("data-guild-id")}/${node.getAttribute("data-channel-id")}/${node.getAttribute("data-message-id")}`
 		if (node.getAttribute("data-channel-id")) return `<#${node.getAttribute("data-channel-id")}>`
 		const href = node.getAttribute("href")
-		const suppressedHref = node.hasAttribute("data-suppress") ? "<" + href + ">" : href
+		let shouldSuppress = node.hasAttribute("data-suppress")
+		if (href.match(/^https?:\/\/matrix.to\//)) shouldSuppress = false // avoid double-escaping
+		const suppressedHref = shouldSuppress ? "<" + href + ">" : href
 		content = content.replace(/ @.*/, "")
 		if (href === content) return suppressedHref
 		if (decodeURIComponent(href).startsWith("https://matrix.to/#/@") && content[0] !== "@") content = "@" + content
@@ -287,8 +290,8 @@ function convertEmoji(mxcUrl, nameForGuess, allowSpriteSheetIndicator, allowLink
  * @returns {Promise<{displayname?: string?, avatar_url?: string?}>}
  */
 async function getMemberFromCacheOrHomeserver(roomID, mxid, api) {
-	const row = select("member_cache", ["displayname", "avatar_url"], {room_id: roomID, mxid}).get()
-	if (row) return row
+	const row = select("member_cache", ["displayname", "avatar_url", "missing_profile"], {room_id: roomID, mxid}).get()
+	if (row && !row.missing_profile) return row
 	return api.getStateEvent(roomID, "m.room.member", mxid).then(event => {
 		const room = select("channel_room", "room_id", {room_id: roomID}).get()
 		if (room) {
@@ -296,7 +299,7 @@ async function getMemberFromCacheOrHomeserver(roomID, mxid, api) {
 			// the cache will be kept in sync by the `m.room.member` event listener
 			const displayname = event?.displayname || null
 			const avatar_url = event?.avatar_url || null
-			db.prepare("INSERT INTO member_cache (room_id, mxid, displayname, avatar_url) VALUES (?, ?, ?, ?) ON CONFLICT DO UPDATE SET displayname = ?, avatar_url = ?").run(
+			db.prepare("INSERT INTO member_cache (room_id, mxid, displayname, avatar_url) VALUES (?, ?, ?, ?) ON CONFLICT DO UPDATE SET displayname = ?, avatar_url = ?, missing_profile = NULL").run(
 				roomID, mxid,
 				displayname, avatar_url,
 				displayname, avatar_url
@@ -346,26 +349,39 @@ function getUserOrProxyOwnerMention(mxid) {
 
 /**
  * At the time of this executing, we know what the end of message emojis are, and we know that at least one of them is unknown.
- * This function will strip them from the content and generate the correct pending file of the sprite sheet.
+ * This function will strip them from the content and add a link that Discord will preview with a sprite sheet of emojis.
  * @param {string} content
- * @param {{id: string, filename: string}[]} attachments
- * @param {({name: string, mxc: string} | {name: string, mxc: string, key: string, iv: string} | {name: string, buffer: Buffer})[]} pendingFiles
- * @param {(mxc: string) => Promise<Buffer | undefined>} mxcDownloader function that will download the mxc URLs and convert to uncompressed PNG data. use `getAndConvertEmoji` or a mock.
+ * @returns {string} new content with emoji sheet link
  */
-async function uploadEndOfMessageSpriteSheet(content, attachments, pendingFiles, mxcDownloader) {
+function linkEndOfMessageSpriteSheet(content) {
 	if (!content.includes("<::>")) return content // No unknown emojis, nothing to do
 	// Remove known and unknown emojis from the end of the message
 	const r = /<a?:[a-zA-Z0-9_]*:[0-9]*>\s*$/
+
 	while (content.match(r)) {
 		content = content.replace(r, "")
 	}
-	// Create a sprite sheet of known and unknown emojis from the end of the message
-	const buffer = await emojiSheet.compositeMatrixEmojis(endOfMessageEmojis, mxcDownloader)
-	// Attach it
-	const filename = "emojis.png"
-	attachments.push({id: String(attachments.length), filename})
-	pendingFiles.push({name: filename, buffer})
-	return content
+
+	// Use a markdown link to hide the URL. If this is the only thing in the message, Discord will hide it entirely, same as lone URLs. Good for us.
+	content = content.trimEnd()
+	content += " [\u2800](" // U+2800 Braille Pattern Blank is invisible on all known platforms but is digitally not a whitespace character
+	const afterLink = ")"
+
+	// Make emojis URL params
+	const params = new URLSearchParams()
+	for (const mxc of endOfMessageEmojis) {
+		// We can do up to 2000 chars max. (In this maximal case it will get chunked to a separate message.) Ignore additional emojis.
+		const withoutMxc = mxUtils.makeMxcPublic(mxc)
+		assert(withoutMxc)
+		const emojisLength = params.toString().length + encodeURIComponent(withoutMxc).length + 2
+		if (content.length + emojisLength + afterLink.length > 2000) {
+			break
+		}
+		params.append("e", withoutMxc)
+	}
+
+	const url = `${reg.ooye.bridge_origin}/download/sheet?${params.toString()}`
+	return content + url + afterLink
 }
 
 /**
@@ -522,7 +538,7 @@ async function getL1L2ReplyLine(called = false) {
  * @param {Ty.Event.Outer_M_Room_Message | Ty.Event.Outer_M_Room_Message_File | Ty.Event.Outer_M_Sticker | Ty.Event.Outer_M_Room_Message_Encrypted_File | Ty.Event.Outer_Org_Matrix_Msc3381_Poll_Start | Ty.Event.Outer_Org_Matrix_Msc3381_Poll_End} event
  * @param {DiscordTypes.APIGuild} guild
  * @param {DiscordTypes.APIGuildTextChannel} channel
- * @param {{api: import("../../matrix/api"), snow: import("snowtransfer").SnowTransfer, mxcDownloader: (mxc: string) => Promise<Buffer | undefined>, pollEnd?: {messageID: string}}} di simple-as-nails dependency injection for the matrix API
+ * @param {{api: import("../../matrix/api"), snow: import("snowtransfer").SnowTransfer, pollEnd?: {messageID: string}}} di simple-as-nails dependency injection for the matrix API
  */
 async function eventToMessage(event, guild, channel, di) {
 	let displayName = event.sender
@@ -561,8 +577,8 @@ async function eventToMessage(event, guild, channel, di) {
 	let shouldProcessTextEvent = event.type === "m.room.message" && (event.content.msgtype === "m.text" || event.content.msgtype === "m.emote")
 	if (event.type === "m.room.message" && (event.content.msgtype === "m.file" || event.content.msgtype === "m.video" || event.content.msgtype === "m.audio" || event.content.msgtype === "m.image")) {
 		// Build message content in addition to the uploaded file
-		const fileIsSpoiler = event.content["page.codeberg.everypizza.msc4193.spoiler"]
-		const fileSpoilerReason = event.content["page.codeberg.everypizza.msc4193.spoiler.reason"]
+		const fileIsSpoiler = event.content["page.codeberg.everypizza.msc4193.spoiler"] || event.content["town.robin.msc3725.content_warning"]
+		const fileSpoilerReason = event.content["page.codeberg.everypizza.msc4193.spoiler.reason"] || event.content["town.robin.msc3725.content_warning"]?.description
 		content = ""
 		const captionContent = new mxUtils.MatrixStringBuilder()
 
@@ -615,23 +631,10 @@ async function eventToMessage(event, guild, channel, di) {
 	}
 
 	if (event.type === "m.sticker") {
-		content = ""
-		let filename = event.content.body
-		if (event.type === "m.sticker") {
-			let mimetype
-			if (event.content.info?.mimetype?.includes("/")) {
-				mimetype = event.content.info.mimetype
-			} else {
-				const res = await di.api.getMedia(event.content.url, {method: "HEAD"})
-				if (res.status === 200) {
-					mimetype = res.headers.get("content-type")
-				}
-				if (!mimetype) throw new Error(`Server error ${res.status} or missing content-type while detecting sticker mimetype`)
-			}
-			filename += "." + mimetype.split("/")[1]
-		}
-		attachments.push({id: "0", filename})
-		pendingFiles.push({name: filename, mxc: event.content.url})
+		const withoutMxc = mxUtils.makeMxcPublic(event.content.url)
+		assert(withoutMxc)
+		const url = `${reg.ooye.bridge_origin}/download/sticker/${withoutMxc}/_.webp`
+		content = `[${event.content.body || "\u2800"}](${url})`
 
 	} else if (event.type === "org.matrix.msc3381.poll.start") {
 		const pollContent = event.content["org.matrix.msc3381.poll.start"] // just for convenience
@@ -653,7 +656,7 @@ async function eventToMessage(event, guild, channel, di) {
 		pollMessages.push(pollComponents.getPollComponentsFromDatabase(di.pollEnd.messageID))
 		pollMessages.push({
 			...await pollComponents.getPollEndMessageFromDatabase(channel.id, di.pollEnd.messageID),
-			avatar_url: `${reg.ooye.bridge_origin}/discord/poll-star-avatar.png`
+			avatar_url: `${reg.ooye.bridge_origin}/download/file/poll-star-avatar.png`
 		})
 
 	} else {
@@ -828,7 +831,6 @@ async function eventToMessage(event, guild, channel, di) {
 				// The matrix spec hasn't decided whether \n counts as a newline or not, but I'm going to count it, because if it's in the data it's there for a reason.
 				// But I should not count it if it's between block elements.
 				input = input.replace(/(<\/?([^ >]+)[^>]*>)?\n(<\/?([^ >]+)[^>]*>)?/g, (whole, beforeContext, beforeTag, afterContext, afterTag) => {
-					// console.error(beforeContext, beforeTag, afterContext, afterTag)
 					if (typeof beforeTag !== "string" && typeof afterTag !== "string") {
 						return "<br>"
 					}
@@ -935,7 +937,7 @@ async function eventToMessage(event, guild, channel, di) {
 				if (replyLine && content.startsWith("> ")) content = "\n" + content
 
 				// SPRITE SHEET EMOJIS FEATURE:
-				content = await uploadEndOfMessageSpriteSheet(content, attachments, pendingFiles, di?.mxcDownloader)
+				content = await linkEndOfMessageSpriteSheet(content)
 			} else {
 				// Looks like we're using the plaintext body!
 				content = event.content.body

src/m2d/converters/event-to-message.test.js

@@ -1,22 +1,11 @@
 const assert = require("assert").strict
-const fs = require("fs")
 const {test} = require("supertape")
 const DiscordTypes = require("discord-api-types/v10")
 const {eventToMessage} = require("./event-to-message")
-const {convertImageStream} = require("./emoji-sheet")
 const data = require("../../../test/data")
 const {MatrixServerError} = require("../../matrix/mreq")
 const {select, discord} = require("../../passthrough")
 
-/* c8 ignore next 7 */
-function slow() {
-	if (process.argv.includes("--slow")) {
-		return test
-	} else {
-		return test.skip
-	}
-}
-
 /**
  * @param {string} roomID
  * @param {string} eventID
@@ -49,25 +38,6 @@ function sameFirstContentAndWhitespace(t, a, b) {
 	t.equal(a2, b2)
 }
 
-/**
- * MOCK: Gets the emoji from the filesystem and converts to uncompressed PNG data.
- * @param {string} mxc a single mxc:// URL
- * @returns {Promise<Buffer | undefined>} uncompressed PNG data, or undefined if the downloaded emoji is not valid
-*/
-async function mockGetAndConvertEmoji(mxc) {
-	const id = mxc.match(/\/([^./]*)$/)?.[1]
-	let s
-	if (fs.existsSync(`test/res/${id}.png`)) {
-		s = fs.createReadStream(`test/res/${id}.png`)
-	} else {
-		s = fs.createReadStream(`test/res/${id}.gif`)
-	}
-	return convertImageStream(s, () => {
-		s.pause()
-		s.emit("end")
-	})
-}
-
 test("event2message: body is used when there is no formatted_body", async t => {
 	t.deepEqual(
 		await eventToMessage({
@@ -303,6 +273,69 @@ test("event2message: markdown in link text does not attempt to be escaped becaus
 	)
 })
 
+test("event2message: markdown in link url does not attempt to be escaped (plaintext body, not suppressed)", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			content: {
+				msgtype: "m.text",
+				body: "the wikimedia commons freaks are gonna love this one https://commons.wikimedia.org/wiki/File:Car_covered_in_traffic_cones.jpg"
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+			sender: "@cadence:cadence.moe",
+			type: "m.room.message"
+		}),
+		{
+			ensureJoined: [],
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "the wikimedia commons freaks are gonna love this one https://commons.wikimedia.org/wiki/File:Car_covered_in_traffic_cones.jpg",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}]
+		}
+	)
+})
+
+test("event2message: markdown in link url does not attempt to be escaped (plaintext body, link suppressed)", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			content: {
+				msgtype: "m.text",
+				body: "the wikimedia commons freaks are gonna love this one https://commons.wikimedia.org/wiki/File:Car_covered_in_traffic_cones.jpg"
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+			sender: "@cadence:cadence.moe",
+			type: "m.room.message"
+		}, {
+			id: "123",
+			roles: [{
+				id: "123",
+				name: "@everyone",
+				permissions: DiscordTypes.PermissionFlagsBits.SendMessages
+			}]
+		}, {}),
+		{
+			ensureJoined: [],
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "the wikimedia commons freaks are gonna love this one <https://commons.wikimedia.org/wiki/File:Car_covered_in_traffic_cones.jpg>",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}]
+		}
+	)
+})
+
 test("event2message: embeds are suppressed if the guild does not have embed links permission (formatted body)", async t => {
 	t.deepEqual(
 		await eventToMessage({
@@ -3314,6 +3347,47 @@ test("event2message: mentioning matrix users works", async t => {
 	)
 })
 
+test("event2message: matrix mentions are not double-escaped when embed links permission is denied", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			content: {
+				msgtype: "m.text",
+				body: "wrong body",
+				format: "org.matrix.custom.html",
+				formatted_body: `I'm just <a href="https://matrix.to/#/@rnl:cadence.moe">▲</a> testing mentions`
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			origin_server_ts: 1688301929913,
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+			sender: "@cadence:cadence.moe",
+			type: "m.room.message",
+			unsigned: {
+				age: 405299
+			}
+		}, {
+			id: "123",
+			roles: [{
+				id: "123",
+				name: "@everyone",
+				permissions: DiscordTypes.PermissionFlagsBits.SendMessages
+			}]
+		}),
+		{
+			ensureJoined: [],
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "I'm just [@▲](<https://matrix.to/#/@rnl:cadence.moe>) testing mentions",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}]
+		}
+	)
+})
+
 test("event2message: multiple mentions are both escaped", async t => {
 	t.deepEqual(
 		await eventToMessage({
@@ -5294,102 +5368,122 @@ test("event2message: table", async t => {
 	)
 })
 
-slow()("event2message: unknown emoji at the end is reuploaded as a sprite sheet", async t => {
-	const messages = await eventToMessage({
-		type: "m.room.message",
-		sender: "@cadence:cadence.moe",
-		content: {
-			msgtype: "m.text",
-			body: "wrong body",
-			format: "org.matrix.custom.html",
-			formatted_body: 'a b <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/RLMgJGfgTPjIQtvvWZsYjhjy\" title=\":ms_robot_grin:\" alt=\":ms_robot_grin:\">'
-		},
-		event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
-		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
-	}, {}, {}, {mxcDownloader: mockGetAndConvertEmoji})
-	const testResult = {
-		content: messages.messagesToSend[0].content,
-		fileName: messages.messagesToSend[0].pendingFiles[0].name,
-		fileContentStart: messages.messagesToSend[0].pendingFiles[0].buffer.subarray(0, 90).toString("base64")
-	}
-	t.deepEqual(testResult, {
-		content: "a b",
-		fileName: "emojis.png",
-		fileContentStart: "iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAPoAAAD6AG1e1JrAAALoklEQVR4nM1ZaVBU2RU+LZSIGnAvFUtcRkSk6abpbkDH"
-	})
+test("event2message: unknown emoji at the end is used for sprite sheet", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			type: "m.room.message",
+			sender: "@cadence:cadence.moe",
+			content: {
+				msgtype: "m.text",
+				body: "wrong body",
+				format: "org.matrix.custom.html",
+				formatted_body: 'a b <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/RLMgJGfgTPjIQtvvWZsYjhjy\" title=\":ms_robot_grin:\" alt=\":ms_robot_grin:\">'
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+		}),
+		{
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "a b [\u2800](https://bridge.example.org/download/sheet?e=cadence.moe%2FRLMgJGfgTPjIQtvvWZsYjhjy)",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}],
+			ensureJoined: []
+		}
+	)
 })
 
-slow()("event2message: known emoji from an unreachable server at the end is reuploaded as a sprite sheet", async t => {
-	const messages = await eventToMessage({
-		type: "m.room.message",
-		sender: "@cadence:cadence.moe",
-		content: {
-			msgtype: "m.text",
-			body: "wrong body",
-			format: "org.matrix.custom.html",
-			formatted_body: 'a b <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/bZFuuUSEebJYXUMSxuuSuLTa\" title=\":emoji_from_unreachable_server:\" alt=\":emoji_from_unreachable_server:\">'
-		},
-		event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
-		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
-	}, {}, {}, {mxcDownloader: mockGetAndConvertEmoji})
-	const testResult = {
-		content: messages.messagesToSend[0].content,
-		fileName: messages.messagesToSend[0].pendingFiles[0].name,
-		fileContentStart: messages.messagesToSend[0].pendingFiles[0].buffer.subarray(0, 90).toString("base64")
-	}
-	t.deepEqual(testResult, {
-		content: "a b",
-		fileName: "emojis.png",
-		fileContentStart: "iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAOoUlEQVR4nM1aCXBbx3l+Eu8bN0CAuO+TAHGTFAmAJHgT"
-	})
+test("event2message: known emoji from an unreachable server at the end is used for sprite sheet", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			type: "m.room.message",
+			sender: "@cadence:cadence.moe",
+			content: {
+				msgtype: "m.text",
+				body: "wrong body",
+				format: "org.matrix.custom.html",
+				formatted_body: 'a b <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/bZFuuUSEebJYXUMSxuuSuLTa\" title=\":emoji_from_unreachable_server:\" alt=\":emoji_from_unreachable_server:\">'
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+		}),
+		{
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "a b [\u2800](https://bridge.example.org/download/sheet?e=cadence.moe%2FbZFuuUSEebJYXUMSxuuSuLTa)",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}],
+			ensureJoined: []
+		}
+	)
 })
 
-slow()("event2message: known and unknown emojis in the end are reuploaded as a sprite sheet", async t => {
-	const messages = await eventToMessage({
-		type: "m.room.message",
-		sender: "@cadence:cadence.moe",
-		content: {
-			msgtype: "m.text",
-			body: "wrong body",
-			format: "org.matrix.custom.html",
-			formatted_body: 'known unknown: <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/qWmbXeRspZRLPcjseyLmeyXC\" title=\":hippo:\" alt=\":hippo:\"> <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/wcouHVjbKJJYajkhJLsyeJAA\" title=\":ms_robot_dress:\" alt=\":ms_robot_dress:\"> and known unknown: <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/WbYqNlACRuicynBfdnPYtmvc\" title=\":hipposcope:\" alt=\":hipposcope:\"> <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HYcztccFIPgevDvoaWNsEtGJ\" title=\":ms_robot_cat:\" alt=\":ms_robot_cat:\">'
-		},
-		event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
-		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
-	}, {}, {}, {mxcDownloader: mockGetAndConvertEmoji})
-	const testResult = {
-		content: messages.messagesToSend[0].content,
-		fileName: messages.messagesToSend[0].pendingFiles[0].name,
-		fileContentStart: messages.messagesToSend[0].pendingFiles[0].buffer.subarray(0, 90).toString("base64")
-	}
-	t.deepEqual(testResult, {
-		content: "known unknown: <:hippo:230201364309868544> [:ms_robot_dress:](https://bridge.example.org/download/matrix/cadence.moe/wcouHVjbKJJYajkhJLsyeJAA) and known unknown:",
-		fileName: "emojis.png",
-		fileContentStart: "iVBORw0KGgoAAAANSUhEUgAAAGAAAAAwCAYAAADuFn/PAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAT/klEQVR4nOVcC3CVRZbuS2KAIMpDQt5PQkIScm/uvYRX"
-	})
+test("event2message: known and unknown emojis in the end are used for sprite sheet", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			type: "m.room.message",
+			sender: "@cadence:cadence.moe",
+			content: {
+				msgtype: "m.text",
+				body: "wrong body",
+				format: "org.matrix.custom.html",
+				formatted_body: 'known unknown: <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/qWmbXeRspZRLPcjseyLmeyXC\" title=\":hippo:\" alt=\":hippo:\"> <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/wcouHVjbKJJYajkhJLsyeJAA\" title=\":ms_robot_dress:\" alt=\":ms_robot_dress:\"> and known unknown: <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/WbYqNlACRuicynBfdnPYtmvc\" title=\":hipposcope:\" alt=\":hipposcope:\"> <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HYcztccFIPgevDvoaWNsEtGJ\" title=\":ms_robot_cat:\" alt=\":ms_robot_cat:\">'
+			},
+			event_id: "$g07oYSZFWBkxohNEfywldwgcWj1hbhDzQ1sBAKvqOOU",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+		}),
+		{
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "known unknown: <:hippo:230201364309868544> [:ms_robot_dress:](https://bridge.example.org/download/matrix/cadence.moe/wcouHVjbKJJYajkhJLsyeJAA) and known unknown: [\u2800](https://bridge.example.org/download/sheet?e=cadence.moe%2FWbYqNlACRuicynBfdnPYtmvc&e=cadence.moe%2FHYcztccFIPgevDvoaWNsEtGJ)",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}],
+			ensureJoined: []
+		}
+	)
 })
 
-slow()("event2message: all unknown chess emojis are reuploaded as a sprite sheet", async t => {
-	const messages = await eventToMessage({
-		type: "m.room.message",
-		sender: "@cadence:cadence.moe",
-		content: {
-			msgtype: "m.text",
-			body: "testing :chess_good_move::chess_incorrect::chess_blund::chess_brilliant_move::chess_blundest::chess_draw_black::chess_good_move::chess_incorrect::chess_blund::chess_brilliant_move::chess_blundest::chess_draw_black:",
-			format: "org.matrix.custom.html",
-			formatted_body: "testing <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/lHfmJpzgoNyNtYHdAmBHxXix\" title=\":chess_good_move:\" alt=\":chess_good_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/MtRdXixoKjKKOyHJGWLsWLNU\" title=\":chess_incorrect:\" alt=\":chess_incorrect:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HXfFuougamkURPPMflTJRxGc\" title=\":chess_blund:\" alt=\":chess_blund:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/ikYKbkhGhMERAuPPbsnQzZiX\" title=\":chess_brilliant_move:\" alt=\":chess_brilliant_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/AYPpqXzVJvZdzMQJGjioIQBZ\" title=\":chess_blundest:\" alt=\":chess_blundest:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/UVuzvpVUhqjiueMxYXJiFEAj\" title=\":chess_draw_black:\" alt=\":chess_draw_black:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/lHfmJpzgoNyNtYHdAmBHxXix\" title=\":chess_good_move:\" alt=\":chess_good_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/MtRdXixoKjKKOyHJGWLsWLNU\" title=\":chess_incorrect:\" alt=\":chess_incorrect:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HXfFuougamkURPPMflTJRxGc\" title=\":chess_blund:\" alt=\":chess_blund:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/ikYKbkhGhMERAuPPbsnQzZiX\" title=\":chess_brilliant_move:\" alt=\":chess_brilliant_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/AYPpqXzVJvZdzMQJGjioIQBZ\" title=\":chess_blundest:\" alt=\":chess_blundest:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/UVuzvpVUhqjiueMxYXJiFEAj\" title=\":chess_draw_black:\" alt=\":chess_draw_black:\">"
-		},
-		event_id: "$Me6iE8C8CZyrDEOYYrXKSYRuuh_25Jj9kZaNrf7LKr4",
-		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
-	}, {}, {}, {mxcDownloader: mockGetAndConvertEmoji})
-	const testResult = {
-		content: messages.messagesToSend[0].content,
-		fileName: messages.messagesToSend[0].pendingFiles[0].name,
-		fileContentStart: messages.messagesToSend[0].pendingFiles[0].buffer.subarray(0, 90).toString("base64")
-	}
-	t.deepEqual(testResult, {
-		content: "testing",
-		fileName: "emojis.png",
-		fileContentStart: "iVBORw0KGgoAAAANSUhEUgAAAYAAAABgCAYAAAAU9KWJAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAgAElEQVR4nOx9B3hUVdr/KIpKL2nT0pPpLRNQkdXddV1c"
-	})
+test("event2message: all unknown chess emojis are used for sprite sheet", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			type: "m.room.message",
+			sender: "@cadence:cadence.moe",
+			content: {
+				msgtype: "m.text",
+				body: "testing :chess_good_move::chess_incorrect::chess_blund::chess_brilliant_move::chess_blundest::chess_draw_black::chess_good_move::chess_incorrect::chess_blund::chess_brilliant_move::chess_blundest::chess_draw_black:",
+				format: "org.matrix.custom.html",
+				formatted_body: "testing <img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/lHfmJpzgoNyNtYHdAmBHxXix\" title=\":chess_good_move:\" alt=\":chess_good_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/MtRdXixoKjKKOyHJGWLsWLNU\" title=\":chess_incorrect:\" alt=\":chess_incorrect:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HXfFuougamkURPPMflTJRxGc\" title=\":chess_blund:\" alt=\":chess_blund:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/ikYKbkhGhMERAuPPbsnQzZiX\" title=\":chess_brilliant_move:\" alt=\":chess_brilliant_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/AYPpqXzVJvZdzMQJGjioIQBZ\" title=\":chess_blundest:\" alt=\":chess_blundest:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/UVuzvpVUhqjiueMxYXJiFEAj\" title=\":chess_draw_black:\" alt=\":chess_draw_black:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/lHfmJpzgoNyNtYHdAmBHxXix\" title=\":chess_good_move:\" alt=\":chess_good_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/MtRdXixoKjKKOyHJGWLsWLNU\" title=\":chess_incorrect:\" alt=\":chess_incorrect:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/HXfFuougamkURPPMflTJRxGc\" title=\":chess_blund:\" alt=\":chess_blund:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/ikYKbkhGhMERAuPPbsnQzZiX\" title=\":chess_brilliant_move:\" alt=\":chess_brilliant_move:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/AYPpqXzVJvZdzMQJGjioIQBZ\" title=\":chess_blundest:\" alt=\":chess_blundest:\"><img data-mx-emoticon height=\"32\" src=\"mxc://cadence.moe/UVuzvpVUhqjiueMxYXJiFEAj\" title=\":chess_draw_black:\" alt=\":chess_draw_black:\">"
+			},
+			event_id: "$Me6iE8C8CZyrDEOYYrXKSYRuuh_25Jj9kZaNrf7LKr4",
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+		}),
+		{
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "testing [\u2800](https://bridge.example.org/download/sheet?e=cadence.moe%2FlHfmJpzgoNyNtYHdAmBHxXix&e=cadence.moe%2FMtRdXixoKjKKOyHJGWLsWLNU&e=cadence.moe%2FHXfFuougamkURPPMflTJRxGc&e=cadence.moe%2FikYKbkhGhMERAuPPbsnQzZiX&e=cadence.moe%2FAYPpqXzVJvZdzMQJGjioIQBZ&e=cadence.moe%2FUVuzvpVUhqjiueMxYXJiFEAj&e=cadence.moe%2FlHfmJpzgoNyNtYHdAmBHxXix&e=cadence.moe%2FMtRdXixoKjKKOyHJGWLsWLNU&e=cadence.moe%2FHXfFuougamkURPPMflTJRxGc&e=cadence.moe%2FikYKbkhGhMERAuPPbsnQzZiX&e=cadence.moe%2FAYPpqXzVJvZdzMQJGjioIQBZ&e=cadence.moe%2FUVuzvpVUhqjiueMxYXJiFEAj)",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}],
+			ensureJoined: []
+		}
+	)
 })

src/m2d/event-dispatcher.js

@@ -88,6 +88,12 @@ function stringifyErrorStack(err, depth = 0) {
 	return collapsed;
 }
 
+function printError(type, source, e, payload) {
+	console.error(`Error while processing a ${type} ${source} event:`)
+	console.error(e)
+	console.dir(payload, {depth: null})
+}
+
 /**
  * @param {string} roomID
  * @param {"Discord" | "Matrix"} source
@@ -96,9 +102,9 @@ function stringifyErrorStack(err, depth = 0) {
  * @param {any} payload
  */
 async function sendError(roomID, source, type, e, payload) {
-	console.error(`Error while processing a ${type} ${source} event:`)
-	console.error(e)
-	console.dir(payload, {depth: null})
+	if (source === "Matrix") {
+		printError(type, source, e, payload)
+	}
 
 	if (Date.now() - lastReportedEvent < 5000) return null
 	lastReportedEvent = Date.now()
@@ -357,15 +363,7 @@ async event => {
 	await api.ackEvent(event)
 }))
 
-function getFromInviteRoomState(inviteRoomState, nskey, key) {
-	if (!Array.isArray(inviteRoomState)) return null
-	for (const event of inviteRoomState) {
-		if (event.type === nskey && event.state_key === "") {
-			return event.content[key]
-		}
-	}
-	return null
-}
+
 
 sync.addTemporaryListener(as, "type:m.space.child", guard("m.space.child",
 /**
@@ -373,7 +371,18 @@ sync.addTemporaryListener(as, "type:m.space.child", guard("m.space.child",
  */
 async event => {
 	if (Array.isArray(event.content.via) && event.content.via.length) { // space child is being added
-		await api.joinRoom(event.state_key).catch(() => {}) // try to join if able, it's okay if it doesn't want, bot will still respond to invites
+		try {
+			// try to join if able, it's okay if it doesn't want, bot will still respond to invites
+			await api.joinRoom(event.state_key)
+			// if autojoined a child space, store it in invite (otherwise the child space will be impossible to use with self-service in the future)
+			const hierarchy = await api.getHierarchy(event.state_key, {limit: 1})
+			const roomProperties = hierarchy.rooms?.[0]
+			if (roomProperties?.room_id === event.state_key && roomProperties.room_type === "m.space" && roomProperties.name) {
+				db.prepare("INSERT OR IGNORE INTO invite (mxid, room_id, type, name, topic, avatar) VALUES (?, ?, ?, ?, ?, ?)")
+					.run(event.sender, event.state_key, roomProperties.room_type, roomProperties.name, roomProperties.topic, roomProperties.avatar_url)
+				await updateMemberCachePowerLevels(event.state_key) // store privileged users in member_cache so they are also allowed to perform self-service
+			}
+		} catch (e) {}
 	}
 }))
 
@@ -398,38 +407,32 @@ async event => {
 		}
 
 		// We were invited to a room. We should join, and register the invite details for future reference in web.
-		let attemptedApiMessage = "According to unsigned invite data."
-		let inviteRoomState = event.unsigned?.invite_room_state
-		if (!Array.isArray(inviteRoomState) || inviteRoomState.length === 0) {
-			try {
-				inviteRoomState = await api.getInviteState(event.room_id)
-				attemptedApiMessage = "According to SSS API."
-			} catch (e) {
-				attemptedApiMessage = "According to unsigned invite data. SSS API unavailable: " + e.toString()
-			}
+		try {
+			var inviteRoomState = await api.getInviteState(event.room_id, event)
+		} catch (e) {
+			console.error(e)
+			return await api.leaveRoomWithReason(event.room_id, `I wasn't able to find out what this room is. Please report this as a bug. Check console for more details. (${e.toString()})`)
 		}
-		const name = getFromInviteRoomState(inviteRoomState, "m.room.name", "name")
-		const topic = getFromInviteRoomState(inviteRoomState, "m.room.topic", "topic")
-		const avatar = getFromInviteRoomState(inviteRoomState, "m.room.avatar", "url")
-		const creationType = getFromInviteRoomState(inviteRoomState, "m.room.create", "type")
-		if (!name) return await api.leaveRoomWithReason(event.room_id, `Please only invite me to rooms that have a name/avatar set. Update the room details and reinvite! (${attemptedApiMessage})`)
+		if (!inviteRoomState?.name) return await api.leaveRoomWithReason(event.room_id, `Please only invite me to rooms that have a name/avatar set. Update the room details and reinvite.`)
 		await api.joinRoom(event.room_id)
-		db.prepare("INSERT OR IGNORE INTO invite (mxid, room_id, type, name, topic, avatar) VALUES (?, ?, ?, ?, ?, ?)").run(event.sender, event.room_id, creationType, name, topic, avatar)
-		if (avatar) utils.getPublicUrlForMxc(avatar) // make sure it's available in the media_proxy allowed URLs
+		db.prepare("REPLACE INTO invite (mxid, room_id, type, name, topic, avatar) VALUES (?, ?, ?, ?, ?, ?)").run(event.sender, event.room_id, inviteRoomState.type, inviteRoomState.name, inviteRoomState.topic, inviteRoomState.avatar)
+		if (inviteRoomState.avatar) utils.getPublicUrlForMxc(inviteRoomState.avatar) // make sure it's available in the media_proxy allowed URLs
+		await updateMemberCachePowerLevels(event.room_id) // store privileged users in member_cache so they are also allowed to perform self-service
 	}
 
-	if (utils.eventSenderIsFromDiscord(event.state_key)) return
-
 	if (event.content.membership === "leave" || event.content.membership === "ban") {
 		// Member is gone
 		db.prepare("DELETE FROM member_cache WHERE room_id = ? and mxid = ?").run(event.room_id, event.state_key)
 
-		// Unregister room's use as a direct chat if the bot itself left
+		// Unregister room's use as a direct chat and/or an invite target if the bot itself left
 		if (event.state_key === utils.bot) {
 			db.prepare("DELETE FROM direct WHERE room_id = ?").run(event.room_id)
+			db.prepare("DELETE FROM invite WHERE room_id = ?").run(event.room_id)
 		}
 	}
 
+	if (utils.eventSenderIsFromDiscord(event.state_key)) return
+
 	const exists = select("channel_room", "room_id", {room_id: event.room_id}) ?? select("guild_space", "space_id", {space_id: event.room_id})
 	if (!exists) return // don't cache members in unbridged rooms
 
@@ -438,7 +441,7 @@ async event => {
 	if (memberPower === Infinity) memberPower = tombstone // database storage compatibility
 	const displayname = event.content.displayname || null
 	const avatar_url = event.content.avatar_url
-	db.prepare("INSERT INTO member_cache (room_id, mxid, displayname, avatar_url, power_level) VALUES (?, ?, ?, ?, ?) ON CONFLICT DO UPDATE SET displayname = ?, avatar_url = ?, power_level = ?").run(
+	db.prepare("INSERT INTO member_cache (room_id, mxid, displayname, avatar_url, power_level) VALUES (?, ?, ?, ?, ?) ON CONFLICT DO UPDATE SET displayname = ?, avatar_url = ?, power_level = ?, missing_profile = NULL").run(
 		event.room_id, event.state_key,
 		displayname, avatar_url, memberPower,
 		displayname, avatar_url, memberPower
@@ -451,16 +454,25 @@ sync.addTemporaryListener(as, "type:m.room.power_levels", guard("m.room.power_le
  */
 async event => {
 	if (event.state_key !== "") return
-	const existingPower = select("member_cache", "mxid", {room_id: event.room_id}).pluck().all()
-	const {allCreators} = await utils.getEffectivePower(event.room_id, [], api)
-	const newPower = event.content.users || {}
-	for (const mxid of existingPower) {
-		if (!allCreators.includes(mxid)) {
-			db.prepare("UPDATE member_cache SET power_level = ? WHERE room_id = ? AND mxid = ?").run(newPower[mxid] || 0, event.room_id, mxid)
-		}
-	}
+	await updateMemberCachePowerLevels(event.room_id)
 }))
 
+/**
+ * @param {string} roomID
+ */
+async function updateMemberCachePowerLevels(roomID) {
+	const existingPower = select("member_cache", "mxid", {room_id: roomID}).pluck().all()
+	const {powerLevels, allCreators, tombstone} = await utils.getEffectivePower(roomID, [], api)
+	const newPower = powerLevels.users || {}
+	const newPowerUsers = Object.keys(newPower)
+	const relevantUsers = existingPower.concat(newPowerUsers).concat(allCreators)
+	for (const mxid of [...new Set(relevantUsers)]) {
+		const level = allCreators.includes(mxid) ? tombstone : newPower[mxid] ?? powerLevels.users_default ?? 0
+		db.prepare("INSERT INTO member_cache (room_id, mxid, power_level, missing_profile) VALUES (?, ?, ?, 1) ON CONFLICT DO UPDATE SET power_level = ?")
+			.run(roomID, mxid, level, level)
+	}
+}
+
 sync.addTemporaryListener(as, "type:m.room.tombstone", guard("m.room.tombstone",
 /**
  * @param {Ty.Event.StateOuter<Ty.Event.M_Room_Tombstone>} event
@@ -473,3 +485,4 @@ async event => {
 
 module.exports.stringifyErrorStack = stringifyErrorStack
 module.exports.sendError = sendError
+module.exports.printError = printError

src/matrix/api.js

@@ -128,6 +128,19 @@ async function getEventForTimestamp(roomID, ts) {
 	return root
 }
 
+/**
+ * @param {string} roomID
+ * @param {"b" | "f"} dir
+ * @param {{from?: string, limit?: any}} [pagination]
+ * @param {any} [filter]
+ */
+async function getEvents(roomID, dir, pagination = {}, filter) {
+	filter = filter && JSON.stringify(filter)
+	/** @type {Ty.MessagesPagination<Ty.Event.Outer<any>>} */
+	const root = await mreq.mreq("GET", path(`/client/v3/rooms/${roomID}/messages`, null, {...pagination, dir, filter}))
+	return root
+}
+
 /**
  * @param {string} roomID
  * @returns {Promise<Ty.Event.StateOuter<any>[]>}
@@ -158,20 +171,83 @@ function getStateEventOuter(roomID, type, key) {
 
 /**
  * @param {string} roomID
- * @returns {Promise<Ty.Event.InviteStrippedState[]>}
+ * @param {{unsigned?: {invite_room_state?: Ty.Event.InviteStrippedState[]}}} [event]
+ * @returns {Promise<{name: string?, topic: string?, avatar: string?, type: string?}>}
  */
-async function getInviteState(roomID) {
-	/** @type {Ty.R.SSS} */
-	const root = await mreq.mreq("POST", path("/client/unstable/org.matrix.simplified_msc3575/sync", `@${reg.sender_localpart}:${reg.ooye.server_name}`, {timeout: "0"}), {
-		room_subscriptions: {
-			[roomID]: {
-				timeline_limit: 0,
-				required_state: []
+async function getInviteState(roomID, event) {
+	function getFromInviteRoomState(strippedState, nskey, key) {
+		if (!Array.isArray(strippedState)) return null
+		for (const event of strippedState) {
+			if (event.type === nskey && event.state_key === "") {
+				return event.content[key]
 			}
 		}
-	})
-	const roomResponse = root.rooms[roomID]
-	return "stripped_state" in roomResponse ? roomResponse.stripped_state : roomResponse.invite_state
+		return null
+	}
+
+	// Try extracting from event (if passed)
+	if (Array.isArray(event?.unsigned?.invite_room_state) && event.unsigned.invite_room_state.length) {
+		return {
+			name: getFromInviteRoomState(event.unsigned.invite_room_state, "m.room.name", "name"),
+			topic: getFromInviteRoomState(event.unsigned.invite_room_state, "m.room.topic", "topic"),
+			avatar: getFromInviteRoomState(event.unsigned.invite_room_state, "m.room.avatar", "url"),
+			type: getFromInviteRoomState(event.unsigned.invite_room_state, "m.room.create", "type")
+		}
+	}
+
+	// Try calling sliding sync API and extracting from stripped state
+	let root
+	try {
+		/** @type {Ty.R.SSS} */
+		root = await mreq.mreq("POST", path("/client/unstable/org.matrix.simplified_msc3575/sync", `@${reg.sender_localpart}:${reg.ooye.server_name}`, {timeout: "0"}), {
+			lists: {
+				a: {
+					ranges: [[0, 999]],
+					timeline_limit: 0,
+					required_state: [],
+					filters: {
+						is_invite: true
+					}
+				}
+			}
+		})
+
+		// Extract from sliding sync response if valid (seems to be okay on Synapse, Tuwunel and Continuwuity at time of writing)
+		if ("lists" in root) {
+			if (!root.rooms?.[roomID]) {
+				const e = new Error("Room data unavailable via SSS")
+				e["data_sss"] = root
+				throw e
+			}
+
+			const roomResponse = root.rooms[roomID]
+			const strippedState = "stripped_state" in roomResponse ? roomResponse.stripped_state : roomResponse.invite_state
+
+			return {
+				name: getFromInviteRoomState(strippedState, "m.room.name", "name"),
+				topic: getFromInviteRoomState(strippedState, "m.room.topic", "topic"),
+				avatar: getFromInviteRoomState(strippedState, "m.room.avatar", "url"),
+				type: getFromInviteRoomState(strippedState, "m.room.create", "type")
+			}
+		}
+	} catch (e) {}
+
+	// Invalid sliding sync response, try alternative (required for Conduit at time of writing)
+	const hierarchy = await getHierarchy(roomID, {limit: 1})
+	if (hierarchy?.rooms?.[0]?.room_id === roomID) {
+		const room = hierarchy?.rooms?.[0]
+		return {
+			name: room.name ?? null,
+			topic: room.topic ?? null,
+			avatar: room.avatar_url ?? null,
+			type: room.room_type ?? null
+		}
+	}
+
+	const e = new Error("Room data unavailable via SSS/hierarchy")
+	e["data_sss"] = root
+	e["data_hierarchy"] = hierarchy
+	throw e
 }
 
 /**
@@ -351,7 +427,7 @@ async function profileSetDisplayname(mxid, displayname, inhibitPropagate) {
 
 /**
  * @param {string} mxid
- * @param {string} avatar_url
+ * @param {string | null | undefined} avatar_url
  * @param {boolean} [inhibitPropagate]
  */
 async function profileSetAvatarUrl(mxid, avatar_url, inhibitPropagate) {
@@ -521,6 +597,7 @@ module.exports.leaveRoom = leaveRoom
 module.exports.leaveRoomWithReason = leaveRoomWithReason
 module.exports.getEvent = getEvent
 module.exports.getEventForTimestamp = getEventForTimestamp
+module.exports.getEvents = getEvents
 module.exports.getAllState = getAllState
 module.exports.getStateEvent = getStateEvent
 module.exports.getStateEventOuter = getStateEventOuter

src/matrix/kstate.js

@@ -149,8 +149,10 @@ async function roomToKState(roomID, limitToEvents) {
 	} else {
 		const root = []
 		await Promise.all(limitToEvents.map(async ([type, key]) => {
-			const outer = await api.getStateEventOuter(roomID, type, key)
-			root.push(outer)
+			try {
+				const outer = await api.getStateEventOuter(roomID, type, key)
+				root.push(outer)
+			} catch (e) {}
 		}))
 		return stateToKState(root)
 	}

src/matrix/matrix-command-handler.js

@@ -124,7 +124,7 @@ const commands = [{
 			if (matrixOnlyReason) {
 				// If uploading to Matrix, check if we have permission
 				const {powerLevels, powers: {[mxUtils.bot]: botPower}} = await mxUtils.getEffectivePower(event.room_id, [mxUtils.bot], api)
-				const requiredPower = powerLevels.events["im.ponies.room_emotes"] ?? powerLevels.state_default ?? 50
+				const requiredPower = powerLevels.events?.["im.ponies.room_emotes"] ?? powerLevels.state_default ?? 50
 				if (botPower < requiredPower) {
 					return api.sendEvent(event.room_id, "m.room.message", {
 						...ctx,

src/matrix/mreq.js

@@ -77,6 +77,7 @@ async function mreq(method, url, bodyIn, extra = {}) {
 		/** @type {any} */
 		var root = JSON.parse(text)
 	} catch (e) {
+		delete opts.headers?.["Authorization"]
 		throw new MatrixServerError(text, {baseUrl, url, ...opts})
 	}
 

src/matrix/read-registration.js

@@ -11,7 +11,7 @@ const registrationFilePath = path.join(process.cwd(), "registration.yaml")
 function checkRegistration(reg) {
 	reg["ooye"].invite = reg.ooye.invite.filter(mxid => mxid.endsWith(`:${reg.ooye.server_name}`)) // one day I will understand why typescript disagrees with dot notation on this line
 	assert(reg.ooye?.max_file_size)
-	assert(reg.ooye?.namespace_prefix)
+	assert(reg.ooye?.namespace_prefix != null)
 	assert(reg.ooye?.server_name)
 	assert(reg.sender_localpart?.startsWith(reg.ooye.namespace_prefix), "appservice's localpart must be in the namespace it controls")
 	assert(reg.ooye?.server_origin.match(/^https?:\/\//), "server origin must start with http or https")
@@ -22,7 +22,7 @@ function checkRegistration(reg) {
 /* c8 ignore next 4 */
 /** @param {import("../types").AppServiceRegistrationConfig} reg */
 function writeRegistration(reg) {
-	fs.writeFileSync(registrationFilePath, JSON.stringify(reg, null, 2))
+	fs.writeFileSync(registrationFilePath, JSON.stringify(reg, null, 2) + "\n")
 }
 
 /**

src/matrix/room-upgrade.js

@@ -57,12 +57,12 @@ async function onBotMembership(event, api, createRoom) {
 	// Check if an upgrade is pending for this room
 	const newRoomID = event.room_id
 	const oldRoomID = select("room_upgrade_pending", "old_room_id", {new_room_id: newRoomID}).pluck().get()
-	if (!oldRoomID) return
+	if (!oldRoomID) return false
 	const channelRow = from("channel_room").join("guild_space", "guild_id").where({room_id: oldRoomID}).select("space_id", "guild_id", "channel_id").get()
 	assert(channelRow) // this could only fail if the channel was unbridged or something between upgrade and joining
 
 	// Check if is join/invite
-	if (event.content.membership !== "invite" && event.content.membership !== "join") return
+	if (event.content.membership !== "invite" && event.content.membership !== "join") return false
 
 	return await roomUpgradeSema.request(async () => {
 		// If invited, join

src/matrix/utils.js

@@ -2,6 +2,7 @@
 
 const assert = require("assert").strict
 const Ty = require("../types")
+const {tag} = require("@cloudrac3r/html-template-tag")
 const passthrough = require("../passthrough")
 const {db} = passthrough
 
@@ -59,6 +60,26 @@ function getEventIDHash(eventID) {
 	return signedHash
 }
 
+class MatrixStringBuilderStack {
+	constructor() {
+		this.stack = [new MatrixStringBuilder()]
+	}
+
+	get msb() {
+		return this.stack[0]
+	}
+
+	bump() {
+		this.stack.unshift(new MatrixStringBuilder())
+	}
+
+	shift() {
+		const msb = this.stack.shift()
+		assert(msb)
+		return msb
+	}
+}
+
 class MatrixStringBuilder {
 	constructor() {
 		this.body = ""
@@ -72,7 +93,7 @@ class MatrixStringBuilder {
 	 */
 	add(body, formattedBody, condition = true) {
 		if (condition) {
-			if (formattedBody == undefined) formattedBody = body
+			if (formattedBody == undefined) formattedBody = tag`${body}`
 			this.body += body
 			this.formattedBody += formattedBody
 		}
@@ -86,7 +107,7 @@ class MatrixStringBuilder {
 	 */
 	addLine(body, formattedBody, condition = true) {
 		if (condition) {
-			if (formattedBody == undefined) formattedBody = body
+			if (formattedBody == undefined) formattedBody = tag`${body}`
 			if (this.body.length && this.body.slice(-1) !== "\n") this.body += "\n"
 			this.body += body
 			const match = this.formattedBody.match(/<\/?([a-zA-Z]+[a-zA-Z0-9]*)[^>]*>\s*$/)
@@ -103,10 +124,11 @@ class MatrixStringBuilder {
 	 */
 	addParagraph(body, formattedBody, condition = true) {
 		if (condition) {
-			if (formattedBody == undefined) formattedBody = body
+			if (formattedBody == undefined) formattedBody = tag`${body}`
 			if (this.body.length && this.body.slice(-1) !== "\n") this.body += "\n\n"
 			this.body += body
-			formattedBody = `<p>${formattedBody}</p>`
+			const match = formattedBody.match(/^<([a-zA-Z]+[a-zA-Z0-9]*)/)
+			if (!match || !BLOCK_ELEMENTS.includes(match[1].toUpperCase())) formattedBody = `<p>${formattedBody}</p>`
 			this.formattedBody += formattedBody
 		}
 		return this
@@ -203,6 +225,19 @@ async function getViaServersQuery(roomID, api) {
 	return qs
 }
 
+function generatePermittedMediaHash(mxc) {
+	assert(hasher, "xxhash is not ready yet")
+	const mediaParts = mxc?.match(/^mxc:\/\/([^/]+)\/(\w+)$/)
+	if (!mediaParts) return undefined
+
+	const serverAndMediaID = `${mediaParts[1]}/${mediaParts[2]}`
+	const unsignedHash = hasher.h64(serverAndMediaID)
+	const signedHash = unsignedHash - 0x8000000000000000n // shifting down to signed 64-bit range
+	db.prepare("INSERT OR IGNORE INTO media_proxy (permitted_hash) VALUES (?)").run(signedHash)
+
+	return serverAndMediaID
+}
+
 /**
  * Since the introduction of authenticated media, this can no longer just be the /_matrix/media/r0/download URL
  * because Discord and Discord users cannot use those URLs. Media now has to be proxied through the bridge.
@@ -213,10 +248,20 @@ async function getViaServersQuery(roomID, api) {
  * @see https://matrix.org/blog/2024/06/26/sunsetting-unauthenticated-media/ background
  * @see https://matrix.org/blog/2024/06/20/matrix-v1.11-release/ implementation details
  * @see https://www.sqlite.org/fileformat2.html#record_format SQLite integer field size
- * @param {string} mxc
+ * @param {string | null | undefined} mxc
  * @returns {string | undefined}
  */
 function getPublicUrlForMxc(mxc) {
+	const serverAndMediaID = makeMxcPublic(mxc)
+	if(!serverAndMediaID) return undefined
+	return `${reg.ooye.bridge_origin}/download/matrix/${serverAndMediaID}`
+}
+
+/**
+ * @param {string | null | undefined} mxc
+ * @returns {string | undefined} mxc URL with protocol stripped, e.g. "cadence.moe/abcdef1234"
+ */
+function makeMxcPublic(mxc) {
 	assert(hasher, "xxhash is not ready yet")
 	const mediaParts = mxc?.match(/^mxc:\/\/([^/]+)\/(\w+)$/)
 	if (!mediaParts) return undefined
@@ -226,7 +271,7 @@ function getPublicUrlForMxc(mxc) {
 	const signedHash = unsignedHash - 0x8000000000000000n // shifting down to signed 64-bit range
 	db.prepare("INSERT OR IGNORE INTO media_proxy (permitted_hash) VALUES (?)").run(signedHash)
 
-	return `${reg.ooye.bridge_origin}/download/matrix/${serverAndMediaID}`
+	return serverAndMediaID
 }
 
 /**
@@ -264,7 +309,7 @@ function roomHasAtLeastVersion(roomVersionString, desiredVersion) {
  */
 function removeCreatorsFromPowerLevels(roomCreateOuter, powerLevels) {
 	assert(roomCreateOuter.sender)
-	if (roomHasAtLeastVersion(roomCreateOuter.content.room_version, 12)) {
+	if (roomHasAtLeastVersion(roomCreateOuter.content.room_version, 12) && powerLevels.users) {
 		for (const creator of (roomCreateOuter.content.additional_creators ?? []).concat(roomCreateOuter.sender)) {
 			delete powerLevels.users[creator]
 		}
@@ -356,9 +401,11 @@ async function setUserPowerCascade(spaceID, mxid, power, api) {
 module.exports.bot = bot
 module.exports.BLOCK_ELEMENTS = BLOCK_ELEMENTS
 module.exports.eventSenderIsFromDiscord = eventSenderIsFromDiscord
+module.exports.makeMxcPublic = makeMxcPublic
 module.exports.getPublicUrlForMxc = getPublicUrlForMxc
 module.exports.getEventIDHash = getEventIDHash
 module.exports.MatrixStringBuilder = MatrixStringBuilder
+module.exports.MatrixStringBuilderStack = MatrixStringBuilderStack
 module.exports.getViaServers = getViaServers
 module.exports.getViaServersQuery = getViaServersQuery
 module.exports.roomHasAtLeastVersion = roomHasAtLeastVersion

src/types.d.ts

@@ -433,6 +433,7 @@ export namespace R {
 		guest_can_join: boolean
 		join_rule?: string
 		name?: string
+		topic?: string
 		num_joined_members: number
 		room_id: string
 		room_type?: string
@@ -497,7 +498,13 @@ export type Membership = "invite" | "knock" | "join" | "leave" | "ban"
 export type Pagination<T> = {
 	chunk: T[]
 	next_batch?: string
-	prev_match?: string
+	prev_batch?: string
+}
+
+export type MessagesPagination<T> = {
+	chunk: T[]
+	start: string
+	end?: string
 }
 
 export type HierarchyPagination<T> = {

src/web/pug-sync.js

@@ -31,7 +31,15 @@ function addGlobals(obj) {
  */
 function render(event, filename, locals) {
 	const path = join(__dirname, "pug", filename)
+	return renderPath(event, path, locals)
+}
 
+/**
+ * @param {import("h3").H3Event} event
+ * @param {string} path
+ * @param {Record<string, any>} locals
+ */
+function renderPath(event, path, locals) {
 	function compile() {
 		try {
 			const template = compileFile(path, {pretty})
@@ -89,4 +97,5 @@ function createRoute(router, url, filename) {
 
 module.exports.addGlobals = addGlobals
 module.exports.render = render
+module.exports.renderPath = renderPath
 module.exports.createRoute = createRoute

src/web/pug/guild.pug

@@ -75,6 +75,7 @@ block body
         button.s-btn(class=space_id ? "s-btn__muted" : "s-btn__filled" hx-get=rel(`/qr?guild_id=${guild_id}`) hx-indicator="closest button" hx-swap="outerHTML" hx-disabled-elt="this") Show QR
 
   if space_id
+    h2.mt48.fs-headline1 Server settings
     h3.mt32.fs-category Privacy level
       span#privacy-level-loading
     .s-card
@@ -104,7 +105,7 @@ block body
           p.s-description.m0 Shareable invite links, like Discord
           p.s-description.m0 Publicly listed in directory, like Discord server discovery
 
-    h2.mt48.fs-headline1 Features
+    h3.mt32.fs-category Features
     .s-card.d-grid.px0.g16
       form.d-flex.ai-center.g16
         #url-preview-loading.p8
@@ -138,13 +139,13 @@ block body
 
     h3.mt32.fs-category Linked channels
     .s-card.bs-sm.p0
-      form.s-table-container(method="post" action=rel("/api/unlink") hx-confirm="Do you want to unlink these channels?\nIt may take a moment to clean up Matrix resources.")
+      form.s-table-container(method="post" action=rel("/api/unlink"))
         input(type="hidden" name="guild_id" value=guild_id)
         table.s-table.s-table__bx-simple
           each row in linkedChannelsWithDetails
             tr
               td.w40: +discord(row.channel)
-              td.p2: button.s-btn.s-btn__muted.s-btn__xs(name="channel_id" value=row.channel.id hx-post=rel("/api/unlink") hx-trigger="click" hx-disabled-elt="this")!= icons.Icons.IconLinkSm
+              td.p2: button.s-btn.s-btn__muted.s-btn__xs(name="channel_id" cx-prevent-default hx-post=rel("/api/unlink") hx-confirm="Do you want to unlink these channels?\nIt may take a moment to clean up Matrix resources." value=row.channel.id hx-indicator="this" hx-disabled-elt="this")!= icons.Icons.IconLinkSm
               td: +matrix(row)
           else
             tr
@@ -185,6 +186,19 @@ block body
           != icons.Icons.IconMerge
           = ` Link`
 
+  h3.mt32.fs-category Unlink server
+  form.s-card.d-flex.gx16.ai-center(method="post" action=rel("/api/unlink-space"))
+    input(type="hidden" name="guild_id" value=guild.id)
+    .fl-grow1.s-prose.s-prose__sm.lh-lg
+      p.fc-medium.
+        Not using this bridge, or just made a mistake? You can unlink the whole server and all its channels.#[br]
+        This may take a minute to process. Please be patient and wait until the page refreshes.
+    div
+      button.s-btn.s-btn__icon.s-btn__danger.s-btn__outlined(cx-prevent-default hx-post=rel("/api/unlink-space") hx-confirm="Do you want to unlink this server and all its channels?\nIt may take a minute to clean up Matrix resources." hx-indicator="this" hx-disabled-elt="this")
+        != icons.Icons.IconUnsync
+        span.ml4= ` Unlink`
+
+  if space_id
     details.mt48
       summary Debug room list
       .d-grid.grid__2.gx24
@@ -205,7 +219,7 @@ block body
             ul.my8.ml24
               each row in removedWrongTypeChannels
                 li: a(href=`https://discord.com/channels/${guild_id}/${row.id}`) (#{row.type}) #{row.name}
-          h3.mt24 Unavailable channels: Bridge can't access
+          h3.mt24 Unavailable channels: Discord bot can't access
           .s-card.p0
             ul.my8.ml24
               each row in removedPrivateChannels

src/web/pug/guild_not_linked.pug

@@ -42,12 +42,23 @@ block body
               | You need to log in with Matrix first.
               a.s-btn.s-btn__matrix.s-btn__outlined(href=rel(`/log-in-with-matrix`, {next: `./guild?guild_id=${guild_id}`})) Log in with Matrix
 
-  h3.mt48.fs-category Auto-create
-  .s-card
+  h3.mt48.fs-category Other choices
+  .s-card.d-grid.g16
     form.d-flex.ai-center.g8(method="post" action=rel("/api/autocreate") hx-post=rel("/api/autocreate") hx-indicator="#easy-mode-button")
       input(type="hidden" name="guild_id" value=guild_id)
       input(type="hidden" name="autocreate" value="true")
       label.s-label.fl-grow1
-        | Changed your mind?
+        | Do it automatically
         p.s-description If you want, OOYE can create and manage the Matrix space so you don't have to.
-      button.s-btn.s-btn__outlined#easy-mode-button Use easy mode
+      button.s-btn.s-btn__icon.s-btn__outlined#easy-mode-button
+        != icons.Icons.IconWand
+        span.ml4= ` Use easy mode`
+
+    form.d-flex.gx16.ai-center(method="post" action=rel("/api/unlink-space"))
+      input(type="hidden" name="guild_id" value=guild.id)
+      label.s-label.fl-grow1
+        | Cancel
+        p.s-description Don't want to link this server after all? Here's the button for you.
+      button.s-btn.s-btn__icon.s-btn__muted.s-btn__outlined(cx-prevent-default hx-post=rel("/api/unlink-space") hx-indicator="this" hx-disabled-elt="this")
+        != icons.Icons.IconUnsync
+        span.ml4= ` Unlink`

src/web/pug/home.pug

@@ -41,16 +41,18 @@ block body
           = ` Set up self-service`
 
   .s-prose
-    h2 What is this?
-    p #[a(href="https://gitdab.com/cadence/out-of-your-element") Out Of Your Element] is a bridge between the Discord and Matrix chat apps. It lets people on both platforms chat with each other without needing to get everyone on the same app.
-    p Just chat like usual, and the bridge will forward messages back and forth between the two platforms, so everyone sees the whole conversation.
-    p All kinds of content are supported, including pictures, threads, emojis, and @mentions.
-    p It's really easy to set up, even if you only have Discord. Just add the bot to your server, and it'll make everything available on Matrix automatically.
+    block bridge-info
+      h2 What is this?
+      p #[a(href="https://gitdab.com/cadence/out-of-your-element") Out Of Your Element] is a bridge between the Discord and Matrix chat apps. It lets people on both platforms chat with each other without needing to get everyone on the same app.
+      p Just chat like usual, and the bridge will forward messages back and forth between the two platforms, so everyone sees the whole conversation.
+      p All kinds of content are supported, including pictures, threads, emojis, and @mentions.
+      p It's really easy to set up, even if you only have Discord. Just add the bot to your server, and it'll make everything available on Matrix automatically.
 
     if locked
-      h2 This is a private instance
-      p Anybody can run their own instance of the Out Of Your Element software. The person running this instance has made it private, so you can't add it to your server just yet. If you know who's in charge of #{reg.ooye.server_name}, ask them for the password.
+      block locked-info
+        h2 This is a private instance
+        p Anybody can run their own instance of the Out Of Your Element software. The person running this instance has made it private, so you can't add it to your server just yet. If you know who's in charge of #{reg.ooye.server_name}, ask them for the password.
 
-      h2 Run your own instance
-      p You can still use Out Of Your Element by running your own copy of the software, but this requires some technical skill.
-      p To get started, #[a(href="https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md") check the installation instructions.]
+        h2 Run your own instance
+        p You can still use Out Of Your Element by running your own copy of the software, but this requires some technical skill.
+        p To get started, #[a(href="https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md") check the installation instructions.]

src/web/pug/includes/template.pug

@@ -1,4 +1,10 @@
-mixin guild(guild)
+mixin guild-menuitem(guild)
+  - let bridgedRoomCount = from("channel_room").selectUnsafe("count(*) as count").where({guild_id: guild.id}).and("AND thread_parent IS NULL").get().count
+  li(role="menuitem")
+    a.s-topbar--item.s-user-card.d-flex.p4(href=rel(`/guild?guild_id=${guild.id}`) class={"bg-purple-200": bridgedRoomCount === 0, "h:bg-purple-300": bridgedRoomCount === 0})
+      +guild(guild, bridgedRoomCount)
+
+mixin guild(guild, bridgedRoomCount)
   span.s-avatar.s-avatar__32.s-user-card--avatar
     if guild.icon
       img.s-avatar--image(src=`https://cdn.discordapp.com/icons/${guild.id}/${guild.icon}.png?size=32` alt="")
@@ -6,8 +12,12 @@ mixin guild(guild)
       .s-avatar--letter.bg-silver-400.bar-md(aria-hidden="true")= guild.name[0]
   .s-user-card--info.ai-start
     strong= guild.name
-    ul.s-user-card--awards
-      li #{discord.guildChannelMap.get(guild.id).filter(c => [0, 5, 15, 16].includes(discord.channels.get(c).type)).length} channels
+    if bridgedRoomCount != null
+      ul.s-user-card--awards
+        if bridgedRoomCount
+          li #{bridgedRoomCount} bridged rooms
+        else
+          li.fc-purple Not yet linked
 
 mixin define-theme(name, h, s, l)
   style.
@@ -58,6 +68,8 @@ html(lang="en")
     title Out Of Your Element
     <meta name="viewport" content="width=device-width, initial-scale=1">
     link(rel="stylesheet" type="text/css" href=rel("/static/stacks.min.css"))
+    //- Please use responsibly!!!!!
+    link(rel="stylesheet" type="text/css" href=rel("/custom.css"))
     <link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 80%22><text y=%22.83em%22 font-size=%2283%22>💬</text></svg>">
     meta(name="htmx-config" content='{"requestClass":"is-loading"}')
     style.
@@ -79,6 +91,14 @@ html(lang="en")
       .s-btn__dropdown:has(+ :popover-open) {
         background-color: var(--theme-topbar-item-background-hover, var(--black-200)) !important;
       }
+      @media (prefers-color-scheme: dark) {
+        body.theme-system .s-popover {
+          --_po-bg: var(--black-100);
+          --_po-bc: var(--bc-light);
+          --_po-bs: var(--bs-lg);
+          --_po-arrow-fc: var(--black-100);
+        }
+      }
     +define-themed-button("matrix", "black")
   body.themed.theme-system
     header.s-topbar
@@ -114,9 +134,7 @@ html(lang="en")
                 .s-popover--content.overflow-y-auto.overflow-x-hidden
                   ul.s-menu(role="menu")
                     each guild in [...managed].map(id => discord.guilds.get(id)).filter(g => g).sort((a, b) => a.name.toLowerCase() < b.name.toLowerCase() ? -1 : 1)
-                      li(role="menuitem")
-                        a.s-topbar--item.s-user-card.d-flex.p4(href=rel(`/guild?guild_id=${guild.id}`))
-                          +guild(guild)
+                      +guild-menuitem(guild)
     //- Body
     .mx-auto.w100.wmx9.py24.px8.fs-body1#content
       block body
@@ -129,6 +147,13 @@ html(lang="en")
           document.styleSheets[0].insertRule(t, document.styleSheets[0].cssRules.length)
         })
       })
+    //- Prevent default
+    script.
+      document.querySelectorAll("[cx-prevent-default]").forEach(e => {
+        e.addEventListener("click", event => {
+          event.preventDefault()
+        })
+      })
     script(src=rel("/static/htmx.js"))
     //- Error dialog
     aside.s-modal#server-error(aria-hidden="true")

src/web/routes/download-discord.js

@@ -31,7 +31,7 @@ function getSnow(event) {
 /** @type {Map<string, Promise<string>>} */
 const cache = new Map()
 
-/** @param {string | undefined} url */
+/** @param {string} url */
 function timeUntilExpiry(url) {
 	const params = new URL(url).searchParams
 	const ex = params.get("ex")

src/web/routes/download-matrix.js

@@ -1,7 +1,7 @@
 // @ts-check
 
 const assert = require("assert/strict")
-const {defineEventHandler, getValidatedRouterParams, setResponseStatus, setResponseHeader, sendStream, createError, H3Event} = require("h3")
+const {defineEventHandler, getValidatedRouterParams, setResponseStatus, setResponseHeader, createError, H3Event, getValidatedQuery} = require("h3")
 const {z} = require("zod")
 
 /** @type {import("xxhash-wasm").XXHashAPI} */ // @ts-ignore
@@ -11,10 +11,25 @@ require("xxhash-wasm")().then(h => hasher = h)
 
 const {sync, as, select} = require("../../passthrough")
 
+/** @type {import("../../m2d/actions/emoji-sheet")} */
+const emojiSheet = sync.require("../../m2d/actions/emoji-sheet")
+/** @type {import("../../m2d/converters/emoji-sheet")} */
+const emojiSheetConverter = sync.require("../../m2d/converters/emoji-sheet")
+
+/** @type {import("../../m2d/actions/sticker")} */
+const sticker = sync.require("../../m2d/actions/sticker")
+
 const schema = {
 	params: z.object({
 		server_name: z.string(),
 		media_id: z.string()
+	}),
+	sheet: z.object({
+		e: z.array(z.string()).or(z.string())
+	}),
+	sticker: z.object({
+		server_name: z.string().regex(/^[^/]+$/),
+		media_id: z.string().regex(/^[A-Za-z0-9_-]+$/)
 	})
 }
 
@@ -27,10 +42,16 @@ function getAPI(event) {
 	return event.context.api || sync.require("../../matrix/api")
 }
 
-as.router.get(`/download/matrix/:server_name/:media_id`, defineEventHandler(async event => {
-	const params = await getValidatedRouterParams(event, schema.params.parse)
+/**
+ * @param {H3Event} event
+ * @returns {typeof emojiSheet["getAndConvertEmoji"]}
+ */
+function getMxcDownloader(event) {
+	/* c8 ignore next */
+	return event.context.mxcDownloader || emojiSheet.getAndConvertEmoji
+}
 
-	const serverAndMediaID = `${params.server_name}/${params.media_id}`
+function verifyMediaHash(serverAndMediaID) {
 	const unsignedHash = hasher.h64(serverAndMediaID)
 	const signedHash = unsignedHash - 0x8000000000000000n // shifting down to signed 64-bit range
 
@@ -41,7 +62,12 @@ as.router.get(`/download/matrix/:server_name/:media_id`, defineEventHandler(asyn
 			data: `The file you requested isn't permitted by this media proxy.`
 		})
 	}
+}
 
+as.router.get(`/download/matrix/:server_name/:media_id`, defineEventHandler(async event => {
+	const params = await getValidatedRouterParams(event, schema.params.parse)
+
+	verifyMediaHash(`${params.server_name}/${params.media_id}`)
 	const api = getAPI(event)
 	const res = await api.getMedia(`mxc://${params.server_name}/${params.media_id}`)
 
@@ -53,3 +79,32 @@ as.router.get(`/download/matrix/:server_name/:media_id`, defineEventHandler(asyn
 	setResponseHeader(event, "Transfer-Encoding", "chunked")
 	return res.body
 }))
+
+as.router.get(`/download/sheet`, defineEventHandler(async event => {
+	const query = await getValidatedQuery(event, schema.sheet.parse)
+
+	/** remember that these have no mxc:// protocol in the string for space reasons */
+	let mxcs = query.e
+	if (!Array.isArray(mxcs)) {
+		mxcs = [mxcs]
+	}
+
+	for (const serverAndMediaID of mxcs) {
+		verifyMediaHash(serverAndMediaID)
+	}
+
+	const buffer = await emojiSheetConverter.compositeMatrixEmojis(mxcs.map(s => `mxc://${s}`), getMxcDownloader(event))
+	setResponseHeader(event, "Content-Type", "image/png")
+	return buffer
+}))
+
+as.router.get(`/download/sticker/:server_name/:media_id/_.webp`, defineEventHandler(async event => {
+	const {server_name, media_id} = await getValidatedRouterParams(event, schema.sticker.parse)
+	/** remember that this has no mxc:// protocol in the string */
+	const mxc = server_name + "/" + media_id
+	verifyMediaHash(mxc)
+
+	const stream = await sticker.getAndResizeSticker(`mxc://${mxc}`)
+	setResponseHeader(event, "Content-Type", "image/webp")
+	return stream
+}))

src/web/routes/download-matrix.test.js

@@ -1,8 +1,11 @@
 // @ts-check
 
+const fs = require("fs")
+const {convertImageStream} = require("../../m2d/converters/emoji-sheet")
 const tryToCatch = require("try-to-catch")
 const {test} = require("supertape")
 const {router} = require("../../../test/web")
+const streamWeb = require("stream/web")
 
 test("web download matrix: access denied if not a known attachment", async t => {
 	const [error] = await tryToCatch(() =>
@@ -25,6 +28,7 @@ test("web download matrix: works if a known attachment", async t => {
 		},
 		event,
 		api: {
+			// @ts-ignore
 			async getMedia(mxc, init) {
 				return new Response("", {status: 200, headers: {"content-type": "image/png"}})
 			}
@@ -33,3 +37,52 @@ test("web download matrix: works if a known attachment", async t => {
 	t.equal(event.node.res.statusCode, 200)
 	t.equal(event.node.res.getHeader("content-type"), "image/png")
 })
+
+/**
+ * MOCK: Gets the emoji from the filesystem and converts to uncompressed PNG data.
+ * @param {string} mxc a single mxc:// URL
+ * @returns {Promise<Buffer | undefined>} uncompressed PNG data, or undefined if the downloaded emoji is not valid
+*/
+async function mockGetAndConvertEmoji(mxc) {
+	const id = mxc.match(/\/([^./]*)$/)?.[1]
+	let s
+	if (fs.existsSync(`test/res/${id}.png`)) {
+		s = fs.createReadStream(`test/res/${id}.png`)
+	} else {
+		s = fs.createReadStream(`test/res/${id}.gif`)
+	}
+	return convertImageStream(s, () => {
+		s.pause()
+		s.emit("end")
+	})
+}
+
+test("web sheet: single emoji", async t => {
+	const event = {}
+	const sheet = await router.test("get", "/download/sheet?e=cadence.moe%2FRLMgJGfgTPjIQtvvWZsYjhjy", {
+		event,
+		mxcDownloader: mockGetAndConvertEmoji
+	})
+	t.equal(event.node.res.statusCode, 200)
+	t.equal(sheet.subarray(0, 90).toString("base64"), "iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAPoAAAD6AG1e1JrAAALoklEQVR4nM1ZaVBU2RU+LZSIGnAvFUtcRkSk6abpbkDH")
+})
+
+test("web sheet: multiple sources", async t => {
+	const event = {}
+	const sheet = await router.test("get", "/download/sheet?e=cadence.moe%2FWbYqNlACRuicynBfdnPYtmvc&e=cadence.moe%2FHYcztccFIPgevDvoaWNsEtGJ", {
+		event,
+		mxcDownloader: mockGetAndConvertEmoji
+	})
+	t.equal(event.node.res.statusCode, 200)
+	t.equal(sheet.subarray(0, 90).toString("base64"), "iVBORw0KGgoAAAANSUhEUgAAAGAAAAAwCAYAAADuFn/PAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAT/klEQVR4nOVcC3CVRZbuS2KAIMpDQt5PQkIScm/uvYRX")
+})
+
+test("web sheet: big sheet", async t => {
+	const event = {}
+	const sheet = await router.test("get", "/download/sheet?e=cadence.moe%2FlHfmJpzgoNyNtYHdAmBHxXix&e=cadence.moe%2FMtRdXixoKjKKOyHJGWLsWLNU&e=cadence.moe%2FHXfFuougamkURPPMflTJRxGc&e=cadence.moe%2FikYKbkhGhMERAuPPbsnQzZiX&e=cadence.moe%2FAYPpqXzVJvZdzMQJGjioIQBZ&e=cadence.moe%2FUVuzvpVUhqjiueMxYXJiFEAj&e=cadence.moe%2FlHfmJpzgoNyNtYHdAmBHxXix&e=cadence.moe%2FMtRdXixoKjKKOyHJGWLsWLNU&e=cadence.moe%2FHXfFuougamkURPPMflTJRxGc&e=cadence.moe%2FikYKbkhGhMERAuPPbsnQzZiX&e=cadence.moe%2FAYPpqXzVJvZdzMQJGjioIQBZ&e=cadence.moe%2FUVuzvpVUhqjiueMxYXJiFEAj", {
+		event,
+		mxcDownloader: mockGetAndConvertEmoji
+	})
+	t.equal(event.node.res.statusCode, 200)
+	t.equal(sheet.subarray(0, 90).toString("base64"), "iVBORw0KGgoAAAANSUhEUgAAAYAAAABgCAYAAAAU9KWJAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAgAElEQVR4nOx9B3hUVdr/KIpKL2nT0pPpLRNQkdXddV1c")
+})

src/web/routes/guild.js

@@ -54,8 +54,8 @@ function getAPI(event) {
 const validNonce = new LRUCache({max: 200})
 
 /**
- * @param {{type: number, parent_id?: string, position?: number}} channel
- * @param {Map<string, {type: number, parent_id?: string, position?: number}>} channels
+ * @param {{type: number, parent_id?: string | null, position?: number}} channel
+ * @param {Map<string, {type: number, parent_id?: string | null, position?: number}>} channels
  */
 function getPosition(channel, channels) {
 	let position = 0
@@ -65,9 +65,11 @@ function getPosition(channel, channels) {
 	// Categories are size 2000.
 	let foundCategory = channel
 	while (foundCategory.parent_id) {
-		foundCategory = channels.get(foundCategory.parent_id)
+		const f = channels.get(foundCategory.parent_id)
+		assert(f)
+		foundCategory = f
 	}
-	if (foundCategory.type === DiscordTypes.ChannelType.GuildCategory) position = (foundCategory.position + 1) * 2000
+	if (foundCategory.type === DiscordTypes.ChannelType.GuildCategory) position = ((foundCategory.position || 0) + 1) * 2000
 
 	// Categories always appear above what they contain.
 	if (channel.type === DiscordTypes.ChannelType.GuildCategory) position -= 0.5
@@ -81,7 +83,7 @@ function getPosition(channel, channels) {
 	// Threads appear below their channel.
 	if ([DiscordTypes.ChannelType.PublicThread, DiscordTypes.ChannelType.PrivateThread, DiscordTypes.ChannelType.AnnouncementThread].includes(channel.type)) {
 		position += 0.5
-		let parent = channels.get(channel.parent_id)
+		let parent = channels.get(channel.parent_id || "")
 		if (parent && parent["position"]) position += parent["position"]
 	}
 
@@ -98,7 +100,11 @@ function getChannelRoomsLinks(guild, rooms, roles) {
 	assert(channelIDs)
 
 	let linkedChannels = select("channel_room", ["channel_id", "room_id", "name", "nick"], {channel_id: channelIDs}).all()
-	let linkedChannelsWithDetails = linkedChannels.map(c => ({channel: discord.channels.get(c.channel_id), ...c}))
+	let linkedChannelsWithDetails = linkedChannels.map(c => ({
+		// @ts-ignore
+		/** @type {DiscordTypes.APIGuildChannel} */ channel: discord.channels.get(c.channel_id),
+		...c
+	}))
 	let removedUncachedChannels = dUtils.filterTo(linkedChannelsWithDetails, c => c.channel)
 	let linkedChannelIDs = linkedChannelsWithDetails.map(c => c.channel_id)
 	linkedChannelsWithDetails.sort((a, b) => getPosition(a.channel, discord.channels) - getPosition(b.channel, discord.channels))
@@ -109,7 +115,7 @@ function getChannelRoomsLinks(guild, rooms, roles) {
 	let removedWrongTypeChannels = dUtils.filterTo(unlinkedChannels, c => c && [0, 5].includes(c.type))
 	let removedPrivateChannels = dUtils.filterTo(unlinkedChannels, c => {
 		const permissions = dUtils.getPermissions(guild.id, roles, guild.roles, botID, c["permission_overwrites"])
-		return dUtils.hasPermission(permissions, DiscordTypes.PermissionFlagsBits.ViewChannel)
+		return dUtils.hasSomePermissions(permissions, ["Administrator", "ViewChannel"])
 	})
 	unlinkedChannels.sort((a, b) => getPosition(a, discord.channels) - getPosition(b, discord.channels))
 
@@ -127,6 +133,20 @@ function getChannelRoomsLinks(guild, rooms, roles) {
 	}
 }
 
+/**
+ * @param {string} mxid
+ */
+function getInviteTargetSpaces(mxid) {
+	/** @type {{room_id: string, mxid: string, type: string, name: string, topic: string?, avatar: string?}[]} */
+	const spaces =
+		// invited spaces
+		db.prepare("SELECT room_id, invite.mxid, type, name, topic, avatar FROM invite LEFT JOIN guild_space ON invite.room_id = guild_space.space_id WHERE mxid = ? AND space_id IS NULL AND type = 'm.space'").all(mxid)
+		// moderated spaces
+		.concat(db.prepare("SELECT room_id, invite.mxid, type, name, topic, avatar FROM invite LEFT JOIN guild_space ON invite.room_id = guild_space.space_id INNER JOIN member_cache USING (room_id) WHERE member_cache.mxid = ? AND power_level >= 50 AND space_id IS NULL AND type = 'm.space'").all(mxid))
+	const seen = new Set(spaces.map(s => s.room_id))
+	return spaces.filter(s => seen.delete(s.room_id))
+}
+
 as.router.get("/guild", defineEventHandler(async event => {
 	const {guild_id} = await getValidatedQuery(event, schema.guild.parse)
 	const session = await auth.useSession(event)
@@ -142,7 +162,7 @@ as.router.get("/guild", defineEventHandler(async event => {
 
 	// Self-service guild that hasn't been linked yet - needs a special page encouraging the link flow
 	if (!row.space_id && row.autocreate === 0) {
-		const spaces = db.prepare("SELECT room_id, type, name, topic, avatar FROM invite LEFT JOIN guild_space ON invite.room_id = guild_space.space_id WHERE mxid = ? AND space_id IS NULL AND type = 'm.space'").all(session.data.mxid)
+		const spaces = session.data.mxid ? getInviteTargetSpaces(session.data.mxid) : []
 		return pugSync.render(event, "guild_not_linked.pug", {guild, guild_id, spaces})
 	}
 
@@ -245,3 +265,4 @@ as.router.post("/api/invite", defineEventHandler(async event => {
 }))
 
 module.exports._getPosition = getPosition
+module.exports.getInviteTargetSpaces = getInviteTargetSpaces

src/web/routes/link.js

@@ -1,5 +1,6 @@
 // @ts-check
 
+const assert = require("assert").strict
 const {z} = require("zod")
 const {defineEventHandler, createError, readValidatedBody, setResponseHeader, H3Event} = require("h3")
 const Ty = require("../../types")
@@ -8,11 +9,10 @@ const DiscordTypes = require("discord-api-types/v10")
 const {discord, db, as, sync, select, from} = require("../../passthrough")
 /** @type {import("../auth")} */
 const auth = sync.require("../auth")
-/** @type {import("../../matrix/mreq")} */
-const mreq = sync.require("../../matrix/mreq")
 /** @type {import("../../matrix/utils")}*/
 const utils = sync.require("../../matrix/utils")
-const {reg} = require("../../matrix/read-registration")
+/** @type {import("./guild")}*/
+const guildRoute = sync.require("./guild")
 
 /**
  * @param {H3Event} event
@@ -41,6 +41,15 @@ function getCreateSpace(event) {
 	return event.context.createSpace || sync.require("../../d2m/actions/create-space")
 }
 
+/**
+ * @param {H3Event} event
+ * @returns {import("snowtransfer").SnowTransfer}
+ */
+function getSnow(event) {
+	/* c8 ignore next */
+	return event.context.snow || discord.snow
+}
+
 const schema = {
 	linkSpace: z.object({
 		guild_id: z.string(),
@@ -54,7 +63,37 @@ const schema = {
 	unlink: z.object({
 		guild_id: z.string(),
 		channel_id: z.string()
-	})
+	}),
+	unlinkSpace: z.object({
+		guild_id: z.string(),
+	}),
+}
+
+/**
+ * @param {H3Event} event
+ * @param {string} channel_id
+ * @param {string} guild_id
+ */
+async function validateAndUnbridgeChannel(event, channel_id, guild_id) {
+	const createRoom = getCreateRoom(event)
+
+	// Check channel is currently bridged
+	const row = select("channel_room", "channel_id", {channel_id: channel_id}).get()
+	if (!row) throw createError({status: 400, message: "Bad Request", data: `Channel ID ${channel_id} is not currently bridged`})
+
+	// Check that the channel (if it exists) is part of this guild
+	/** @type {any} */
+	let channel = discord.channels.get(channel_id)
+	if (channel) {
+		if (!("guild_id" in channel) || channel.guild_id !== guild_id) throw createError({status: 400, message: "Bad Request", data: `Channel ID ${channel_id} is not part of guild ${guild_id}`})
+	} else {
+		// Otherwise, if the channel isn't cached, it must have been deleted.
+		// There's no other authentication here - it's okay for anyone to unlink a deleted channel just by knowing its ID.
+		channel = {id: channel_id}
+	}
+
+	// Do it
+	await createRoom.unbridgeChannel(channel, guild_id)
 }
 
 as.router.post("/api/link-space", defineEventHandler(async event => {
@@ -70,14 +109,18 @@ as.router.post("/api/link-space", defineEventHandler(async event => {
 	// Check space ID
 	if (!session.data.mxid) throw createError({status: 403, message: "Forbidden", data: "Can't link with your Matrix space if you aren't logged in to Matrix"})
 	const spaceID = parsedBody.space_id
-	const inviteRow = select("invite", ["mxid", "type"], {mxid: session.data.mxid, room_id: spaceID}).get()
-	if (!inviteRow || inviteRow.type !== "m.space") throw createError({status: 403, message: "Forbidden", data: "You personally must invite OOYE to that space on Matrix"})
 
 	// Check they are not already bridged
 	const existing = select("guild_space", "guild_id", {}, "WHERE guild_id = ? OR space_id = ?").get(guildID, spaceID)
 	if (existing) throw createError({status: 400, message: "Bad Request", data: `Guild ID ${guildID} or space ID ${spaceID} are already bridged and cannot be reused`})
 
-	const via = [inviteRow.mxid.match(/:(.*)/)[1]]
+	// Check space ID is a valid invite target
+	const inviteRow = guildRoute.getInviteTargetSpaces(session.data.mxid).find(s => s.room_id === spaceID)
+	if (!inviteRow) throw createError({status: 403, message: "Forbidden", data: "You personally must invite OOYE to that space on Matrix"})
+
+	const inviteServer = inviteRow.mxid.match(/:(.*)/)?.[1]
+	assert(inviteServer)
+	const via = [inviteServer]
 
 	// Check space exists and bridge is joined
 	try {
@@ -192,7 +235,6 @@ as.router.post("/api/link", defineEventHandler(async event => {
 as.router.post("/api/unlink", defineEventHandler(async event => {
 	const {channel_id, guild_id} = await readValidatedBody(event, schema.unlink.parse)
 	const managed = await auth.getManagedGuilds(event)
-	const createRoom = getCreateRoom(event)
 
 	// Check guild ID or nonce
 	if (!managed.has(guild_id)) throw createError({status: 403, message: "Forbidden", data: "Can't edit a guild you don't have Manage Server permissions in"})
@@ -201,24 +243,56 @@ as.router.post("/api/unlink", defineEventHandler(async event => {
 	const guild = discord.guilds.get(guild_id)
 	if (!guild) throw createError({status: 400, message: "Bad Request", data: "Discord guild does not exist or bot has not joined it"})
 
-	// Check that the channel (if it exists) is part of this guild
-	/** @type {any} */
-	let channel = discord.channels.get(channel_id)
-	if (channel) {
-		if (!("guild_id" in channel) || channel.guild_id !== guild_id) throw createError({status: 400, message: "Bad Request", data: `Channel ID ${channel_id} is not part of guild ${guild_id}`})
-	} else {
-		// Otherwise, if the channel isn't cached, it must have been deleted.
-		// There's no other authentication here - it's okay for anyone to unlink a deleted channel just by knowing its ID.
-		channel = {id: channel_id}
-	}
-
-	// Check channel is currently bridged
-	const row = select("channel_room", "channel_id", {channel_id: channel_id}).get()
-	if (!row) throw createError({status: 400, message: "Bad Request", data: `Channel ID ${channel_id} is not currently bridged`})
-
-	// Do it
-	await createRoom.unbridgeDeletedChannel(channel, guild_id)
+	await validateAndUnbridgeChannel(event, channel_id, guild_id)
 
 	setResponseHeader(event, "HX-Refresh", "true")
 	return null // 204
 }))
+
+as.router.post("/api/unlink-space", defineEventHandler(async event => {
+	const {guild_id} = await readValidatedBody(event, schema.unlinkSpace.parse)
+	const managed = await auth.getManagedGuilds(event)
+	const api = getAPI(event)
+	const snow = getSnow(event)
+
+	// Check guild ID or nonce
+	if (!managed.has(guild_id)) throw createError({status: 403, message: "Forbidden", data: "Can't edit a guild you don't have Manage Server permissions in"})
+
+	// Check guild exists
+	const guild = discord.guilds.get(guild_id)
+	if (!guild) throw createError({status: 400, message: "Bad Request", data: "Discord guild does not exist or bot has not joined it"})
+
+	const active = select("guild_active", "guild_id", {guild_id: guild_id}).get()
+	if (!active) {
+		throw createError({status: 400, message: "Bad Request", data: "Discord guild has not been considered for bridging"})
+	}
+
+	// Check if there are Matrix resources
+	const spaceID = select("guild_space", "space_id", {guild_id: guild_id}).pluck().get()
+	if (spaceID) {
+		// Unlink all rooms
+		const linkedChannels = select("channel_room", ["channel_id", "room_id", "name", "nick"], {guild_id: guild_id}).all()
+		for (const channel of linkedChannels) {
+			await validateAndUnbridgeChannel(event, channel.channel_id, guild_id)
+		}
+
+		// Verify all rooms were unlinked
+		const remainingLinkedChannels = select("channel_room", ["channel_id", "room_id", "name", "nick"], {guild_id: guild_id}).all()
+		if (remainingLinkedChannels.length) {
+			throw createError({status: 500, message: "Internal Server Error", data: "Failed to unlink some rooms. Please try doing it manually, or report a bug. The space will not be unlinked until all rooms are."})
+		}
+
+		// Unlink space
+		await utils.setUserPower(spaceID, utils.bot, 0, api)
+		await api.leaveRoom(spaceID)
+		db.prepare("DELETE FROM guild_space WHERE guild_id = ? AND space_id = ?").run(guild_id, spaceID)
+		db.prepare("DELETE FROM invite WHERE room_id = ?").run(spaceID)
+	}
+
+	// Mark as not considered for bridging
+	db.prepare("DELETE FROM guild_active WHERE guild_id = ?").run(guild_id)
+	await snow.user.leaveGuild(guild_id)
+
+	setResponseHeader(event, "HX-Redirect", "/")
+	return null
+}))

src/web/routes/link.test.js

@@ -613,7 +613,7 @@ test("web unlink room: checks that the channel is part of the guild", async t =>
 	t.equal(error.data, "Channel ID 112760669178241024 is not part of guild 665289423482519565")
 })
 
-test("web unlink room: successfully calls unbridgeDeletedChannel when the channel does exist", async t => {
+test("web unlink room: successfully calls unbridgeChannel when the channel does exist", async t => {
 	let called = 0
 	await router.test("post", "/api/unlink", {
 		sessionData: {
@@ -624,7 +624,7 @@ test("web unlink room: successfully calls unbridgeDeletedChannel when the channe
 			guild_id: "665289423482519565"
 		},
 		createRoom: {
-			async unbridgeDeletedChannel(channel) {
+			async unbridgeChannel(channel) {
 				called++
 				t.equal(channel.id, "665310973967597573")
 			}
@@ -633,7 +633,7 @@ test("web unlink room: successfully calls unbridgeDeletedChannel when the channe
 	t.equal(called, 1)
 })
 
-test("web unlink room: successfully calls unbridgeDeletedChannel when the channel does not exist", async t => {
+test("web unlink room: successfully calls unbridgeChannel when the channel does not exist", async t => {
 	let called = 0
 	await router.test("post", "/api/unlink", {
 		sessionData: {
@@ -644,7 +644,7 @@ test("web unlink room: successfully calls unbridgeDeletedChannel when the channe
 			guild_id: "112760669178241024"
 		},
 		createRoom: {
-			async unbridgeDeletedChannel(channel) {
+			async unbridgeChannel(channel) {
 				called++
 				t.equal(channel.id, "489237891895768942")
 			}
@@ -654,7 +654,9 @@ test("web unlink room: successfully calls unbridgeDeletedChannel when the channe
 })
 
 test("web unlink room: checks that the channel is bridged", async t => {
+	const row = db.prepare("SELECT * FROM channel_room WHERE channel_id = '665310973967597573'").get()
 	db.prepare("DELETE FROM channel_room WHERE channel_id = '665310973967597573'").run()
+
 	const [error] = await tryToCatch(() => router.test("post", "/api/unlink", {
 		sessionData: {
 			managedGuilds: ["665289423482519565"]
@@ -665,4 +667,179 @@ test("web unlink room: checks that the channel is bridged", async t => {
 		}
 	}))
 	t.equal(error.data, "Channel ID 665310973967597573 is not currently bridged")
+
+	db.prepare("INSERT INTO channel_room (channel_id, room_id, name, nick, thread_parent, custom_avatar, last_bridged_pin_timestamp, speedbump_id, speedbump_checked, speedbump_webhook_id, guild_id, custom_topic) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)").run(row.channel_id, row.room_id, row.name, row.nick, row.thread_parent, row.custom_avatar, row.last_bridged_pin_timestamp, row.speedbump_id, row.speedbump_checked, row.speedbump_webhook_id, row.guild_id, row.custom_topic)
+	const new_row = db.prepare("SELECT * FROM channel_room WHERE channel_id = '665310973967597573'").get()
+	t.deepEqual(row, new_row)
+})
+
+// *****
+
+test("web unlink space: access denied if not logged in to Discord", async t => {
+	const [error] = await tryToCatch(() => router.test("post", "/api/unlink-space", {
+		body: {
+			guild_id: "665289423482519565"
+		}
+	}))
+	t.equal(error.data, "Can't edit a guild you don't have Manage Server permissions in")
+})
+
+test("web unlink space: checks that guild exists", async t => {
+	const [error] = await tryToCatch(() => router.test("post", "/api/unlink-space", {
+		sessionData: {
+			managedGuilds: ["2"]
+		},
+		body: {
+			guild_id: "2"
+		}
+	}))
+	t.equal(error.data, "Discord guild does not exist or bot has not joined it")
+})
+
+test("web unlink space: checks that a space is linked to the guild before trying to unlink the space", async t => {
+	db.exec("BEGIN TRANSACTION")
+	db.prepare("DELETE FROM guild_active WHERE guild_id = '665289423482519565'").run()
+
+	const [error] = await tryToCatch(() => router.test("post", "/api/unlink-space", {
+		sessionData: {
+			managedGuilds: ["665289423482519565"]
+		},
+		body: {
+			guild_id: "665289423482519565"
+		}
+	}))
+	t.equal(error.data, "Discord guild has not been considered for bridging")
+
+	db.exec("ROLLBACK") // ぬ
+})
+
+test("web unlink space: correctly abort unlinking if some linked channels remain after trying to unlink them all", async t => {
+	let unbridgedChannel = false
+
+	const [error] = await tryToCatch(() => router.test("post", "/api/unlink-space", {
+		sessionData: {
+			managedGuilds: ["665289423482519565"]
+		},
+		body: {
+			guild_id: "665289423482519565",
+		},
+		createRoom: {
+			async unbridgeChannel(channel, guildID) {
+				unbridgedChannel = true
+				t.ok(["1438284564815548418", "665310973967597573"].includes(channel.id))
+				t.equal(guildID, "665289423482519565")
+				// Do not actually delete the link from DB, should trigger error later in check
+			}
+		},
+		api: {
+			async *generateFullHierarchy(spaceID) {
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: [],
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
+			},
+		}
+	}))
+
+	t.equal(error.data, "Failed to unlink some rooms. Please try doing it manually, or report a bug. The space will not be unlinked until all rooms are.")
+	t.equal(unbridgedChannel, true)
+})
+
+test("web unlink space: successfully calls unbridgeChannel on linked channels in space, self-downgrade power level, leave space, and delete link from DB", async t => {
+	const {reg} = require("../../matrix/read-registration")
+	const me = `@${reg.sender_localpart}:${reg.ooye.server_name}`
+
+	const getLinkRowQuery = "SELECT * FROM guild_space WHERE guild_id = '665289423482519565'"
+
+	const row = db.prepare(getLinkRowQuery).get()
+	t.equal(row.space_id, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+
+	let unbridgedChannel = false
+	let downgradedPowerLevel = false
+	let leftRoom = false
+	await router.test("post", "/api/unlink-space", {
+		sessionData: {
+			managedGuilds: ["665289423482519565"]
+		},
+		body: {
+			guild_id: "665289423482519565",
+		},
+		createRoom: {
+			async unbridgeChannel(channel, guildID) {
+				unbridgedChannel = true
+				t.ok(["1438284564815548418", "665310973967597573"].includes(channel.id))
+				t.equal(guildID, "665289423482519565")
+
+				// In order to not simulate channel deletion and not trigger the post unlink channels, pre-unlink space check
+				db.prepare("DELETE FROM channel_room WHERE channel_id = ?").run(channel.id)
+			}
+		},
+		snow: {
+			user: {
+				// @ts-ignore - snowtransfer or discord-api-types broken, 204 No Content should be mapped to void but is actually mapped to never
+				async leaveGuild(guildID) {
+					t.equal(guildID, "665289423482519565")
+				}
+			}
+		},
+		api: {
+			async *generateFullHierarchy(spaceID) {
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: [],
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
+			},
+
+			async getStateEvent(roomID, type, key) { // getting power levels from space to apply to room
+				t.equal(type, "m.room.power_levels")
+				t.equal(key, "")
+				return {users: {"@_ooye_bot:cadence.moe": 100, "@example:matrix.org": 50}, events: {"m.room.tombstone": 100}}
+			},
+
+			async getStateEventOuter(roomID, type, key) {
+				t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				t.equal(type, "m.room.create")
+				t.equal(key, "")
+				return {
+					type: "m.room.create",
+					state_key: "",
+					sender: "@_ooye_bot:cadence.moe",
+					room_id: "!zTMspHVUBhFLLSdmnS:cadence.moe",
+					event_id: "$create",
+					origin_server_ts: 0,
+					content: {
+						room_version: "11"
+					}
+				}
+			},
+
+			async sendState(roomID, type, key, content) {
+				downgradedPowerLevel = true
+				t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				t.equal(type, "m.room.power_levels")
+				t.notOk(me in content.users, `got ${JSON.stringify(content)} but expected bot user to not be present`)
+				return ""
+			},
+
+			async leaveRoom(spaceID) {
+				leftRoom = true
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+			},
+		}
+	})
+
+	t.equal(unbridgedChannel, true)
+	t.equal(downgradedPowerLevel, true)
+	t.equal(leftRoom, true)
+
+	const missed_row = db.prepare(getLinkRowQuery).get()
+	t.equal(missed_row, undefined)
 })

src/web/server.js

@@ -4,13 +4,14 @@ const assert = require("assert")
 const fs = require("fs")
 const {join} = require("path")
 const h3 = require("h3")
-const {defineEventHandler, defaultContentType, getRequestHeader, setResponseHeader, handleCacheHeaders} = h3
+const mimeTypes = require("mime-types")
+const {defineEventHandler, defaultContentType, getRequestHeader, setResponseHeader, handleCacheHeaders, serveStatic} = h3
 const icons = require("@stackoverflow/stacks-icons")
 const DiscordTypes = require("discord-api-types/v10")
 const dUtils = require("../discord/utils")
 const reg = require("../matrix/read-registration")
 
-const {sync, discord, as, select} = require("../passthrough")
+const {sync, discord, as, select, from} = require("../passthrough")
 /** @type {import("./pug-sync")} */
 const pugSync = sync.require("./pug-sync")
 /** @type {import("../matrix/utils")} */
@@ -19,21 +20,7 @@ const {id} = require("../../addbot")
 
 // Pug
 
-pugSync.addGlobals({id, h3, discord, select, DiscordTypes, dUtils, mUtils, icons, reg: reg.reg})
-pugSync.createRoute(as.router, "/", "home.pug")
-pugSync.createRoute(as.router, "/ok", "ok.pug")
-
-// Routes
-
-sync.require("./routes/download-matrix")
-sync.require("./routes/download-discord")
-sync.require("./routes/guild-settings")
-sync.require("./routes/guild")
-sync.require("./routes/info")
-sync.require("./routes/link")
-sync.require("./routes/log-in-with-matrix")
-sync.require("./routes/oauth")
-sync.require("./routes/password")
+pugSync.addGlobals({id, h3, discord, select, from, DiscordTypes, dUtils, mUtils, icons, reg: reg.reg})
 
 // Files
 
@@ -65,12 +52,79 @@ as.router.get("/static/htmx.js", defineEventHandler({
 	}
 }))
 
-as.router.get("/icon.png", defineEventHandler(event => {
+as.router.get("/download/file/poll-star-avatar.png", defineEventHandler(event => {
+	handleCacheHeaders(event, {maxAge: 86400})
+	return fs.promises.readFile(join(__dirname, "../../docs/img/poll-star-avatar.png"))
+}))
+
+// Custom files
+
+const publicDir = "custom-webroot"
+
+/**
+ * @param {h3.H3Event} event
+ * @param {boolean} fallthrough
+ */
+function tryStatic(event, fallthrough) {
+	return serveStatic(event, {
+		indexNames: ["/index.html", "/index.pug"],
+		fallthrough,
+		getMeta: async id => {
+			// Check
+			const stats = await fs.promises.stat(join(publicDir, id)).catch(() => {});
+			if (!stats || !stats.isFile()) {
+				return
+			}
+			// Pug
+			if (id.match(/\.pug$/)) {
+				defaultContentType(event, "text/html; charset=utf-8")
+				return {}
+			}
+			// Everything else
+			else {
+				const mime = mimeTypes.lookup(id)
+				if (typeof mime === "string") defaultContentType(event, mime)
+				return {
+					size: stats.size
+				}
+			}
+		},
+		getContents: id => {
+			if (id.match(/\.pug$/)) {
+				const path = join(publicDir, id)
+				return pugSync.renderPath(event, path, {})
+			} else {
+				return fs.promises.readFile(join(publicDir, id))
+			}
+		}
+	})
+}
+
+as.router.get("/**", defineEventHandler(event => {
+	return tryStatic(event, false)
+}))
+
+as.router.get("/", defineEventHandler(async event => {
+	return (await tryStatic(event, true)) || pugSync.render(event, "home.pug", {})
+}))
+
+as.router.get("/icon.png", defineEventHandler(async event => {
+	const s = await tryStatic(event, true)
+	if (s) return s
 	handleCacheHeaders(event, {maxAge: 86400})
 	return fs.promises.readFile(join(__dirname, "../../docs/img/icon.png"))
 }))
 
-as.router.get("/discord/poll-star-avatar.png", defineEventHandler(event => {
-	handleCacheHeaders(event, {maxAge: 86400})
-	return fs.promises.readFile(join(__dirname, "../../docs/img/poll-star-avatar.png"))
-}))
+// Routes
+
+pugSync.createRoute(as.router, "/ok", "ok.pug")
+
+sync.require("./routes/download-matrix")
+sync.require("./routes/download-discord")
+sync.require("./routes/guild-settings")
+sync.require("./routes/guild")
+sync.require("./routes/info")
+sync.require("./routes/link")
+sync.require("./routes/log-in-with-matrix")
+sync.require("./routes/oauth")
+sync.require("./routes/password")

test/data.js

@@ -239,7 +239,7 @@ module.exports = {
 					unicode_emoji: null,
 					tags: {},
 					position: 0,
-					permissions: '559623605575360',
+					permissions: '1122573558996672',
 					name: '@everyone',
 					mentionable: false,
 					managed: false,
@@ -3219,6 +3219,37 @@ module.exports = {
 			flags: 0,
 			components: []
 		},
+		emojihax: {
+			id: "1126733830494093453",
+			type: 0,
+			content: "I only violate the don't modify our console part of terms of service [troll~1](https://cdn.discordapp.com/emojis/1254940125948022915.webp?size=48&name=troll%7E1&lossless=true)",
+			channel_id: "112760669178241024",
+			author: {
+				id: "111604486476181504",
+				username: "kyuugryphon",
+				avatar: "e4ce31267ca524d19be80e684d4cafa1",
+				discriminator: "0",
+				public_flags: 0,
+				flags: 0,
+				banner: null,
+				accent_color: null,
+				global_name: "KyuuGryphon",
+				avatar_decoration: null,
+				display_name: "KyuuGryphon",
+				banner_color: null
+			},
+			attachments: [],
+			embeds: [],
+			mentions: [],
+			mention_roles: [],
+			pinned: false,
+			mention_everyone: false,
+			tts: false,
+			timestamp: "2023-07-07T04:37:58.892000+00:00",
+			edited_timestamp: null,
+			flags: 0,
+			components: []
+		},
 		emoji_triple_long_name: {
 			id: "1156394116540805170",
 			type: 0,
@@ -4916,6 +4947,69 @@ module.exports = {
 			flags: 0,
 			components: []
 		},
+		klipy_gif: {
+			type: 0,
+			content: "https://klipy.com/gifs/cute-15",
+			mentions: [],
+			mention_roles: [],
+			attachments: [],
+			embeds: [
+				{
+					type: "gifv",
+					url: "https://klipy.com/gifs/cute-15",
+					title: "Cute Corgi Waddle",
+					provider: {
+						name: "Klipy",
+						url: "https://klipy.com"
+					},
+					thumbnail: {
+						url: "https://static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/xHVF6sVV.webp",
+						proxy_url: "https://images-ext-1.discordapp.net/external/Z54QmlQflPPb6NoXikflBHGmttgRm3_jhzmcILXHhcA/https/static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/xHVF6sVV.webp",
+						width: 277,
+						height: 498,
+						placeholder: "3gcGDAJV+WZYl3RpZ2gGeFBxBw==",
+						placeholder_version: 1,
+						flags: 0
+					},
+					video: {
+						url: "https://static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/7ndEhcilPNKJ8O.mp4",
+						proxy_url: "https://images-ext-1.discordapp.net/external/xZspzkQPUKBa74pBhJDpBf3v2d3d0lC943xaB9_JnoM/https/static.klipy.com/ii/d7aec6f6f171607374b2065c836f92f4/5b/5b/7ndEhcilPNKJ8O.mp4",
+						width: 356,
+						height: 640,
+						placeholder: "3gcGDAJV+WZYl3RpZ2gGeFBxBw==",
+						placeholder_version: 1,
+						flags: 0
+					},
+					content_scan_version: 4
+				}
+			],
+			timestamp: "2026-02-03T11:11:50.070000+00:00",
+			edited_timestamp: null,
+			flags: 0,
+			components: [],
+			id: "1468202316233707613",
+			channel_id: "1370776315266859131",
+			author: {
+				id: "304655299631906816",
+				username: "witterson",
+				avatar: "47ec94a1b2b4cc41ce0329b3575e9b66",
+				discriminator: "0",
+				public_flags: 0,
+				flags: 0,
+				banner: null,
+				accent_color: null,
+				global_name: "wit",
+				avatar_decoration_data: null,
+				collectibles: null,
+				display_name_styles: null,
+				banner_color: null,
+				clan: null,
+				primary_guild: null
+			},
+			pinned: false,
+			mention_everyone: false,
+			tts: false
+		},
 		tenor_gif: {
 			type: 0,
 			content: "<@&1182745800661540927> get real https://tenor.com/view/get-real-gif-26176788",
@@ -4975,6 +5069,183 @@ module.exports = {
 			tts: false
 		}
 	},
+	message_with_components: {
+		pk_question_mark_response: {
+			type: 0,
+			content: '',
+			mentions: [],
+			mention_roles: [],
+			attachments: [],
+			embeds: [],
+			timestamp: '2026-01-30T01:20:07.488000+00:00',
+			edited_timestamp: null,
+			flags: 32768,
+			author: {
+				id: '466378653216014359',
+				username: 'PluralKit',
+				avatar: '466df0c98b1af1e1388f595b4c1ad1b9',
+				discriminator: '0',
+				public_flags: 0,
+				flags: 0,
+				bot: true,
+				banner: null,
+				accent_color: null,
+				global_name: 'PluralKit',
+				avatar_decoration_data: null,
+				collectibles: null,
+				display_name_styles: null,
+				banner_color: null
+			},
+			components: [
+				{
+					type: 17,
+					id: 1,
+					accent_color: 1042150,
+					components: [
+						{
+							type: 9,
+							id: 2,
+							components: [
+								{ type: 10, id: 3, content: '### Lillith (INX)' },
+								{
+									type: 10,
+									id: 4,
+									content: '**Display name:** Lillith (she/her)\n' +
+									'**Pronouns:** She/Her\n' +
+									'**Message count:** 3091'
+								}
+							],
+							accessory: {
+								type: 11,
+								id: 5,
+								media: {
+									id: '1466603856149610687',
+									url: 'https://files.inx.moe/p/cdn/lillith.webp',
+									proxy_url: 'https://images-ext-1.discordapp.net/external/Kn5b32mM4o8AAQbq0k39KOzp9-fy6D1tWKvK_XI27LI/https/files.inx.moe/p/cdn/lillith.webp',
+									width: 256,
+									height: 256,
+									placeholder: 'KVoKJwSnt7lZl5ecj1mal5eGWjAHZXIA',
+									placeholder_version: 1,
+									content_scan_metadata: { version: 4, flags: 0 },
+									content_type: 'image/webp',
+									loading_state: 2,
+									flags: 0
+								},
+								description: null,
+								spoiler: false
+							}
+						},
+						{ type: 14, id: 6, spacing: 1, divider: true },
+						{
+							type: 10,
+							id: 7,
+							content: '**Proxy tags:**\n' +
+							'``l;text``\n' +
+							'``l:text``\n' +
+							'``l.text``\n' +
+							'``textl.``\n' +
+							'``textl;``\n' +
+							'``textl:``'
+						}
+					],
+					spoiler: false
+				},
+				{
+					type: 9,
+					id: 8,
+					components: [
+						{
+							type: 10,
+							id: 9,
+							content: '-# System ID: `xffgnx` ∙ Member ID: `pphhoh`\n' +
+							'-# Created: 2025-12-31 03:16:45 UTC'
+						}
+					],
+					accessory: {
+						type: 2,
+						id: 10,
+						style: 5,
+						label: 'View on dashboard',
+						url: 'https://dash.pluralkit.me/profile/m/pphhoh'
+					}
+				},
+				{ type: 14, id: 11, spacing: 1, divider: true },
+				{
+					type: 17,
+					id: 12,
+					accent_color: null,
+					components: [
+						{
+							type: 9,
+							id: 13,
+							components: [
+								{
+									type: 10,
+									id: 14,
+									content: '**System:** INX (`xffgnx`)\n' +
+									'**Member:** Lillith (`pphhoh`)\n' +
+									'**Sent by:** infinidoge1337 (<@197126718400626689>)\n' +
+									'\n' +
+									'**Account Roles (7)**\n' +
+									'§b, !, ‼, Ears Port Ping, Ears Update Ping, Yttr Ping, unsup Ping'
+								}
+							],
+							accessory: {
+								type: 11,
+								id: 15,
+								media: {
+									id: '1466603856149610689',
+									url: 'https://files.inx.moe/p/cdn/lillith.webp',
+									proxy_url: 'https://images-ext-1.discordapp.net/external/Kn5b32mM4o8AAQbq0k39KOzp9-fy6D1tWKvK_XI27LI/https/files.inx.moe/p/cdn/lillith.webp',
+									width: 256,
+									height: 256,
+									placeholder: 'KVoKJwSnt7lZl5ecj1mal5eGWjAHZXIA',
+									placeholder_version: 1,
+									content_scan_metadata: { version: 4, flags: 0 },
+									content_type: 'image/webp',
+									loading_state: 2,
+									flags: 0
+								},
+								description: null,
+								spoiler: false
+							}
+						},
+						{ type: 14, id: 16, spacing: 2, divider: true },
+						{ type: 10, id: 17, content: 'Same hat' },
+						{
+							type: 12,
+							id: 18,
+							items: [
+								{
+									media: {
+										id: '1466603856149610690',
+										url: 'https://cdn.discordapp.com/attachments/934955898965729280/1466556006527012987/image.png?ex=697d2c37&is=697bdab7&hm=09c5028be61ce01ebbdda5c79c42e4dc10d053ce0c4b12c9d84135a0708e9db6&',
+										proxy_url: 'https://media.discordapp.net/attachments/934955898965729280/1466556006527012987/image.png?ex=697d2c37&is=697bdab7&hm=09c5028be61ce01ebbdda5c79c42e4dc10d053ce0c4b12c9d84135a0708e9db6&',
+										width: 285,
+										height: 126,
+										placeholder: '0PcBA4BqSIl9t/dnn9f0rm0=',
+										placeholder_version: 1,
+										content_scan_metadata: { version: 4, flags: 0 },
+										content_type: 'image/png',
+										loading_state: 2,
+										flags: 0
+									},
+									description: null,
+									spoiler: false
+								}
+							]
+						}
+					],
+					spoiler: false
+				},
+				{
+					type: 10,
+					id: 19,
+					content: '-# Original Message ID: 1466556003645657118 · <t:1769724599:f>'
+				}
+			]
+		}
+	},
 	message_update: {
 		edit_by_webhook: {
 			application_id: "684280192553844747",
@@ -5234,7 +5505,6 @@ module.exports = {
 			mention_roles: [],
 			mentions: [],
 			pinned: false,
-			timestamp: "2023-08-16T22:38:38.641000+00:00",
 			tts: false,
 			type: 0
 		},
@@ -5308,7 +5578,6 @@ module.exports = {
 			mention_roles: [],
 			mentions: [],
 			pinned: false,
-			timestamp: "2023-08-16T22:38:38.641000+00:00",
 			tts: false,
 			type: 0
 		},
@@ -5343,7 +5612,6 @@ module.exports = {
 			pinned: false,
 			mention_everyone: false,
 			tts: false,
-			timestamp: "2023-05-11T23:44:09.690000+00:00",
 			edited_timestamp: "2023-05-11T23:44:19.690000+00:00",
 			flags: 0,
 			components: [],
@@ -5384,7 +5652,6 @@ module.exports = {
 			pinned: false,
 			mention_everyone: false,
 			tts: false,
-			timestamp: "2023-05-11T23:44:09.690000+00:00",
 			edited_timestamp: "2023-05-11T23:44:19.690000+00:00",
 			flags: 0,
 			components: [],
@@ -5425,7 +5692,6 @@ module.exports = {
 			pinned: false,
 			mention_everyone: false,
 			tts: false,
-			timestamp: "2023-05-11T23:44:09.690000+00:00",
 			edited_timestamp: "2023-05-11T23:44:19.690000+00:00",
 			flags: 0,
 			components: [],

test/ooye-test-data.sql

@@ -22,7 +22,7 @@ INSERT INTO channel_room (channel_id, room_id, name, nick, thread_parent, custom
 ('176333891320283136', '!qzDBLKlildpzrrOnFZ:cadence.moe', '🌈丨davids-horse_she-took-the-kids', 'wonderland', NULL, 'mxc://cadence.moe/EVvrSkKIRONHjtRJsMLmHWLS', '112760669178241024'),
 ('489237891895768942', '!tnedrGVYKFNUdnegvf:tchncs.de', 'ex-room-doesnt-exist-any-more', NULL, NULL, NULL, '66192955777486848'),
 ('1160894080998461480', '!TqlyQmifxGUggEmdBN:cadence.moe', 'ooyexperiment', NULL, NULL, NULL, '66192955777486848'),
-('1161864271370666075', '!mHmhQQPwXNananMUqq:cadence.moe', 'updates', NULL, NULL, NULL, '665289423482519565'),
+('1161864271370666075', '!mHmhQQPwXNananMUqq:cadence.moe', 'updates', NULL, NULL, NULL, '112760669178241024'),
 ('1438284564815548418', '!MHxNpwtgVqWOrmyoTn:cadence.moe', 'sin-cave', NULL, NULL, NULL, '665289423482519565'),
 ('598707048112193536', '!JBxeGYnzQwLnaooOLD:cadence.moe', 'winners', NULL, NULL, NULL, '1345641201902288987');
 
@@ -98,8 +98,8 @@ INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part
 ('$oLyUTyZ_7e_SUzGNWZKz880ll9amLZvXGbArJCKai2Q', 'm.room.message', 'm.text', '1128084748338741392', 0, 0, 1),
 ('$FchUVylsOfmmbj-VwEs5Z9kY49_dt2zd0vWfylzy5Yo', 'm.room.message', 'm.text', '1143121514925928541', 0, 0, 1),
 ('$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4', 'm.room.message', 'm.text', '1106366167788044450', 0, 1, 1),
-('$Ijf1MFCD39ktrNHxrA-i2aKoRWNYdAV2ZXYQeiZIgEU', 'm.room.message', 'm.image', '1106366167788044450', 1, 1, 0),
-('$f9cjKiacXI9qPF_nUAckzbiKnJEi0LM399kOkhdd8f8', 'm.sticker', NULL, '1106366167788044450', 1, 0, 0),
+('$Ijf1MFCD39ktrNHxrA-i2aKoRWNYdAV2ZXYQeiZIgEU', 'm.room.message', 'm.image', '1106366167788044450', 1, 1, 1),
+('$f9cjKiacXI9qPF_nUAckzbiKnJEi0LM399kOkhdd8f8', 'm.sticker', NULL, '1106366167788044450', 1, 0, 1),
 ('$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qd999', 'm.room.message', 'm.text', '1106366167788044451', 0, 0, 1),
 ('$Ijf1MFCD39ktrNHxrA-i2aKoRWNYdAV2ZXYQeiZI999', 'm.room.message', 'm.image', '1106366167788044451', 0, 0, 1),
 ('$f9cjKiacXI9qPF_nUAckzbiKnJEi0LM399kOkhdd999', 'm.sticker', NULL, '1106366167788044451', 0, 0, 1),
@@ -152,7 +152,8 @@ INSERT INTO file (discord_url, mxc_url) VALUES
 ('https://cdn.discordapp.com/attachments/1099031887500034088/1112476845502365786/voice-message.ogg', 'mxc://cadence.moe/MRRPDggXQMYkrUjTpxQbmcxB'),
 ('https://cdn.discordapp.com/attachments/122155380120748034/1174514575220158545/the.yml', 'mxc://cadence.moe/HnQIYQmmlIKwOQsbFsIGpzPP'),
 ('https://cdn.discordapp.com/attachments/112760669178241024/1296237494987133070/100km.gif', 'mxc://cadence.moe/qDAotmebTfEIfsAIVCEZptLh'),
-('https://cdn.discordapp.com/attachments/123/456/my_enemies.txt', 'mxc://cadence.moe/y89EOTRp2lbeOkgdsEleGOge');
+('https://cdn.discordapp.com/attachments/123/456/my_enemies.txt', 'mxc://cadence.moe/y89EOTRp2lbeOkgdsEleGOge'),
+('https://cdn.discordapp.com/emojis/1254940125948022915.webp', 'mxc://cadence.moe/bvVJFgOIyNcAknKCbmaHDktG');
 
 INSERT INTO emoji (emoji_id, name, animated, mxc_url) VALUES
 ('230201364309868544', 'hippo', 0, 'mxc://cadence.moe/qWmbXeRspZRLPcjseyLmeyXC'),

test/test.js

@@ -75,47 +75,45 @@ const file = sync.require("../src/matrix/file")
 file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not allowed to upload files during testing.\nURL: ${url}`) }
 
 ;(async () => {
-	/* c8 ignore start - maybe download some more test files in slow mode */
-	if (process.argv.includes("--slow")) {
-		test("test files: download", async t => {
-			/** @param {{url: string, to: string}[]} files */
-			async function allReporter(files) {
-				return new Promise(resolve => {
-					let resolved = 0
-					const report = files.map(file => file.to.split("/").slice(-1)[0][0])
-					files.map(download).forEach((p, i) => {
-						p.then(() => {
-							report[i] = green(".")
-							process.stderr.write("\r" + report.join(""))
-							if (++resolved === files.length) resolve(null)
-						})
+	/* c8 ignore start - download some more test files in slow mode */
+	test("test files: download", async t => {
+		/** @param {{url: string, to: string}[]} files */
+		async function allReporter(files) {
+			return new Promise(resolve => {
+				let resolved = 0
+				const report = files.map(file => file.to.split("/").slice(-1)[0][0])
+				files.map(download).forEach((p, i) => {
+					p.then(() => {
+						report[i] = green(".")
+						process.stderr.write("\r" + report.join(""))
+						if (++resolved === files.length) resolve(null)
 					})
 				})
-			}
-			async function download({url, to}) {
-				if (await fs.existsSync(to)) return
-				const res = await fetch(url)
-				// @ts-ignore
-				await res.body.pipeTo(Writable.toWeb(fs.createWriteStream(to, {encoding: "binary"})))
-			}
-			await allReporter([
-				{url: "https://cadence.moe/friends/ooye_test/RLMgJGfgTPjIQtvvWZsYjhjy.png", to: "test/res/RLMgJGfgTPjIQtvvWZsYjhjy.png"},
-				{url: "https://cadence.moe/friends/ooye_test/bZFuuUSEebJYXUMSxuuSuLTa.png", to: "test/res/bZFuuUSEebJYXUMSxuuSuLTa.png"},
-				{url: "https://cadence.moe/friends/ooye_test/qWmbXeRspZRLPcjseyLmeyXC.png", to: "test/res/qWmbXeRspZRLPcjseyLmeyXC.png"},
-				{url: "https://cadence.moe/friends/ooye_test/wcouHVjbKJJYajkhJLsyeJAA.png", to: "test/res/wcouHVjbKJJYajkhJLsyeJAA.png"},
-				{url: "https://cadence.moe/friends/ooye_test/WbYqNlACRuicynBfdnPYtmvc.gif", to: "test/res/WbYqNlACRuicynBfdnPYtmvc.gif"},
-				{url: "https://cadence.moe/friends/ooye_test/HYcztccFIPgevDvoaWNsEtGJ.png", to: "test/res/HYcztccFIPgevDvoaWNsEtGJ.png"},
-				{url: "https://cadence.moe/friends/ooye_test/lHfmJpzgoNyNtYHdAmBHxXix.png", to: "test/res/lHfmJpzgoNyNtYHdAmBHxXix.png"},
-				{url: "https://cadence.moe/friends/ooye_test/MtRdXixoKjKKOyHJGWLsWLNU.png", to: "test/res/MtRdXixoKjKKOyHJGWLsWLNU.png"},
-				{url: "https://cadence.moe/friends/ooye_test/HXfFuougamkURPPMflTJRxGc.png", to: "test/res/HXfFuougamkURPPMflTJRxGc.png"},
-				{url: "https://cadence.moe/friends/ooye_test/ikYKbkhGhMERAuPPbsnQzZiX.png", to: "test/res/ikYKbkhGhMERAuPPbsnQzZiX.png"},
-				{url: "https://cadence.moe/friends/ooye_test/AYPpqXzVJvZdzMQJGjioIQBZ.png", to: "test/res/AYPpqXzVJvZdzMQJGjioIQBZ.png"},
-				{url: "https://cadence.moe/friends/ooye_test/UVuzvpVUhqjiueMxYXJiFEAj.png", to: "test/res/UVuzvpVUhqjiueMxYXJiFEAj.png"},
-				{url: "https://ezgif.com/images/format-demo/butterfly.gif", to: "test/res/butterfly.gif"},
-				{url: "https://ezgif.com/images/format-demo/butterfly.png", to: "test/res/butterfly.png"},
-			])
-		}, {timeout: 60000})
-	}
+			})
+		}
+		async function download({url, to}) {
+			if (await fs.existsSync(to)) return
+			const res = await fetch(url)
+			// @ts-ignore
+			await res.body.pipeTo(Writable.toWeb(fs.createWriteStream(to, {encoding: "binary"})))
+		}
+		await allReporter([
+			{url: "https://cadence.moe/friends/ooye_test/RLMgJGfgTPjIQtvvWZsYjhjy.png", to: "test/res/RLMgJGfgTPjIQtvvWZsYjhjy.png"},
+			{url: "https://cadence.moe/friends/ooye_test/bZFuuUSEebJYXUMSxuuSuLTa.png", to: "test/res/bZFuuUSEebJYXUMSxuuSuLTa.png"},
+			{url: "https://cadence.moe/friends/ooye_test/qWmbXeRspZRLPcjseyLmeyXC.png", to: "test/res/qWmbXeRspZRLPcjseyLmeyXC.png"},
+			{url: "https://cadence.moe/friends/ooye_test/wcouHVjbKJJYajkhJLsyeJAA.png", to: "test/res/wcouHVjbKJJYajkhJLsyeJAA.png"},
+			{url: "https://cadence.moe/friends/ooye_test/WbYqNlACRuicynBfdnPYtmvc.gif", to: "test/res/WbYqNlACRuicynBfdnPYtmvc.gif"},
+			{url: "https://cadence.moe/friends/ooye_test/HYcztccFIPgevDvoaWNsEtGJ.png", to: "test/res/HYcztccFIPgevDvoaWNsEtGJ.png"},
+			{url: "https://cadence.moe/friends/ooye_test/lHfmJpzgoNyNtYHdAmBHxXix.png", to: "test/res/lHfmJpzgoNyNtYHdAmBHxXix.png"},
+			{url: "https://cadence.moe/friends/ooye_test/MtRdXixoKjKKOyHJGWLsWLNU.png", to: "test/res/MtRdXixoKjKKOyHJGWLsWLNU.png"},
+			{url: "https://cadence.moe/friends/ooye_test/HXfFuougamkURPPMflTJRxGc.png", to: "test/res/HXfFuougamkURPPMflTJRxGc.png"},
+			{url: "https://cadence.moe/friends/ooye_test/ikYKbkhGhMERAuPPbsnQzZiX.png", to: "test/res/ikYKbkhGhMERAuPPbsnQzZiX.png"},
+			{url: "https://cadence.moe/friends/ooye_test/AYPpqXzVJvZdzMQJGjioIQBZ.png", to: "test/res/AYPpqXzVJvZdzMQJGjioIQBZ.png"},
+			{url: "https://cadence.moe/friends/ooye_test/UVuzvpVUhqjiueMxYXJiFEAj.png", to: "test/res/UVuzvpVUhqjiueMxYXJiFEAj.png"},
+			{url: "https://ezgif.com/images/format-demo/butterfly.gif", to: "test/res/butterfly.gif"},
+			{url: "https://ezgif.com/images/format-demo/butterfly.png", to: "test/res/butterfly.png"},
+		])
+	}, {timeout: 60000})
 	/* c8 ignore stop */
 
 	const p = migrate.migrate(db)
@@ -135,15 +133,6 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 	require("./addbot.test")
 	require("../src/db/orm.test")
 	require("../src/web/server.test")
-	require("../src/web/routes/download-discord.test")
-	require("../src/web/routes/download-matrix.test")
-	require("../src/web/routes/guild.test")
-	require("../src/web/routes/guild-settings.test")
-	require("../src/web/routes/info.test")
-	require("../src/web/routes/link.test")
-	require("../src/web/routes/log-in-with-matrix.test")
-	require("../src/web/routes/oauth.test")
-	require("../src/web/routes/password.test")
 	require("../src/discord/utils.test")
 	require("../src/matrix/kstate.test")
 	require("../src/matrix/api.test")
@@ -158,10 +147,12 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 	require("../src/d2m/actions/register-user.test")
 	require("../src/d2m/converters/edit-to-changes.test")
 	require("../src/d2m/converters/emoji-to-key.test")
+	require("../src/d2m/converters/find-mentions.test")
 	require("../src/d2m/converters/lottie.test")
 	require("../src/d2m/converters/message-to-event.test")
-	require("../src/d2m/converters/message-to-event.embeds.test")
-	require("../src/d2m/converters/message-to-event.pk.test")
+	require("../src/d2m/converters/message-to-event.test.components")
+	require("../src/d2m/converters/message-to-event.test.embeds")
+	require("../src/d2m/converters/message-to-event.test.pk")
 	require("../src/d2m/converters/pins-to-list.test")
 	require("../src/d2m/converters/remove-reaction.test")
 	require("../src/d2m/converters/thread-to-announcement.test")
@@ -176,4 +167,13 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 	require("../src/discord/interactions/permissions.test")
 	require("../src/discord/interactions/privacy.test")
 	require("../src/discord/interactions/reactions.test")
+	require("../src/web/routes/download-discord.test")
+	require("../src/web/routes/download-matrix.test")
+	require("../src/web/routes/guild.test")
+	require("../src/web/routes/guild-settings.test")
+	require("../src/web/routes/info.test")
+	require("../src/web/routes/link.test")
+	require("../src/web/routes/log-in-with-matrix.test")
+	require("../src/web/routes/oauth.test")
+	require("../src/web/routes/password.test")
 })()

test/web.js

@@ -51,7 +51,7 @@ class Router {
 	/**
 	 * @param {string} method
 	 * @param {string} inputUrl
-	 * @param {{event?: any, params?: any, body?: any, sessionData?: any, getOauth2Token?: any, getClient?: (string) => {user: {getGuilds: () => Promise<DiscordTypes.RESTGetAPICurrentUserGuildsResult>}}, api?: Partial<import("../src/matrix/api")>, snow?: {[k in keyof SnowTransfer]?: Partial<SnowTransfer[k]>}, createRoom?: Partial<import("../src/d2m/actions/create-room")>, createSpace?: Partial<import("../src/d2m/actions/create-space")>, headers?: any}} [options]
+	 * @param {{event?: any, params?: any, body?: any, sessionData?: any, getOauth2Token?: any, getClient?: (string) => {user: {getGuilds: () => Promise<DiscordTypes.RESTGetAPICurrentUserGuildsResult>}}, api?: Partial<import("../src/matrix/api")>, snow?: {[k in keyof SnowTransfer]?: Partial<SnowTransfer[k]>}, createRoom?: Partial<import("../src/d2m/actions/create-room")>, createSpace?: Partial<import("../src/d2m/actions/create-space")>, mxcDownloader?: import("../src/m2d/actions/emoji-sheet")["getAndConvertEmoji"], headers?: any}} [options]
 	 */
 	async test(method, inputUrl, options = {}) {
 		const url = new URL(inputUrl, "http://a")
@@ -83,6 +83,7 @@ class Router {
 				},
 				context: {
 					api: options.api,
+					mxcDownloader: options.mxcDownloader,
 					params: options.params,
 					snow: options.snow,
 					createRoom: options.createRoom,