Add internals.md

internals.md

@@ -0,0 +1,57 @@
+# Discord's "stack"
+
+Technologies Discord uses to provide the service, gathered from
+[Discord's engineering articles](https://blog.discordapp.com/tagged/engineering) and some other texts:
+
+
+[Cassandra]: https://cassandra.apache.org
+[MongoDB]: https://www.mongodb.com
+[Flask]: http://flask.pocoo.org
+[Punt]: https://github.com/discordapp/punt
+[Loqui]: https://github.com/discordapp/punt
+
+
+## Storage:
+ - [Cassandra](http://cassandra.apache.org/) for storage
+ - They used [MongoDB] as primary storage, but moved to Cassandra, [source](https://blog.discordapp.com/how-discord-stores-billions-of-messages-7fa6ec7ee4c7)).
+ - It is still unknown what role MongoDB has on the stack, but it is still
+    a primary part of the service: [source](https://status.discordapp.com/incidents/rkj7rx8f4865)
+    and [source](https://status.discordapp.com/incidents/2cb5jc53jq28)
+
+
+## Programming languages:
+ - [Elixir](https://elixir-lang.org/) for any real time communication,
+    [source](https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b).
+ - [Python](https://www.python.org/) for HTTP/REST API,
+    The best guesses to which framework are [Flask].
+ - [Go](https://golang.org/) for the embed servers and one element of their logging(see [Punt])
+ - [Rust](https://www.rust-lang.org/) for many parts of the Discord Store, most commonly:
+   - [Game SDK](https://discordapp.com/developers/docs/game-sdk/sdk-starter-guide),
+        to make Discord integrations for any game.
+   - [Dispatch](https://discordapp.com/developers/docs/dispatch/dispatch-and-you),
+        which is a tool to send assets to Discord's servers.
+
+
+## Distribution:
+ - [Loqui] for node communication.
+
+
+## Logging:
+ - [Punt] in favour of [Logstash](https://github.com/elastic/logstash) for logging.
+ - [Elasticsearch](https://github.com/elastic/elasticsearch) that powers the search feature for users and powers logging.
+   - Sources about logging: Punt and [this issue](https://github.com/elastic/elasticsearch/issues/20354)
+   - [Source about message search](https://blog.discordapp.com/how-discord-indexes-billions-of-messages-e3d5e9be866f)
+
+
+## External services
+ - [Google's Cloud Platform](https://cloud.google.com/) for their infrastructure,
+     [source](https://status.discordapp.com/incidents/rhvp2tn7g0zc)
+ - [Cloudflare](https://www.cloudflare.com/) as a proxy to almost all services,
+     voice servers need to be direct connections, so they don't pass through CF.
+
+
+## Many other sources
+ - [Description of a full outage where they had to reboot everything](https://status.discordapp.com/incidents/dj3l6lw926kl)
+ - [`sessions` and `presence` clusters get rebooted due to a host error in a `guild` node](https://status.discordapp.com/incidents/ywdwttd6b0hg)
+ - [Repeating message sends due to errors in the `push` cluster](https://status.discordapp.com/incidents/93kyyctg0wf3)
+ - ["furiously" spinning an nginx cluster due to an error in GCP's load balancer](https://status.discordapp.com/incidents/rhvp2tn7g0zc)