# discord-json-2021-01

## Manual edits
* images -> placeholders
  * `accounts/avatar.png`
* manually scrub folder names
  * `account/applications/0000000000000`

## Notes about files
* `activity/`
  * All the .json are NDJSON so some json tools don't like them
  * _Massive_ files. They hang scrub.ts for a long long time (had to run these piecemeal)
  * These files also have an _incredible_ amount of shapes and variance.
    * Instead of outputing all the shapes I made a sort of "super-object" to capture the shape with `jq -n '[inputs] | add' events-2021-00000-of-00001.json.tmp > unique_shape.json` and then scrubbing `unique_shape.json`
* `messages/`
  * I hand did these to keep all the ids the same
  * There are multiple types of chats. DMs, guild channels, etc
  * I hand did the csvs as I have no scrubber for that
  * These are only **THE EXPORTING USERS MESSAGES**, no other user, just fyi
  * Ids in `messages.csv` are just the id of the message, not of any user
  * There is the potential to derive missing info from a channel via `@` tags sent or possibly via attachments. Maybe...
  * `11111111111111111`
    * This one has a shorter id (it's an older one)
    * Has `type: 0` but there's no guild information in `channel.json`
    * The user name was `null` in `index.json`
    * It's a really odd one
  * `222222222222222222`
    * This was a dm channel (said `direct message with xxx#7777` in index.json)
    * Has `type: 1` and there are two recipients (just the ids) in `channel.json`
    * Unfortunately that's all the info in the export
  * `333333333333333333`
    * This was a normal guild channel
    * `type: 0` and there's guild information in `channel.json`
    * I kept a good set of messages around from this one to show how attachements and other stuff works
    * The last message seemed to be a link not as an attachment. Links just seem to be normal text
* `programs/`
  * was empty...
* `servers/``
  * Info about _some_ of the guilds we have ids for
  * guild.json didn't really contain anything except the name
  * I kept around the only guild I noticed an audit-log.json with info in it