base-data-manager/test/fixtures
2026-03-02 08:42:47 -05:00
..
discord-chat-exporter-2026-02 Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
discord-json-2021-01 Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
facebook-json-2021-05-01 Added scrub and scrubbed facebook exports, and stub test 2026-02-20 01:28:05 -05:00
facebook-json-2025-11-29 Added scrub and scrubbed facebook exports, and stub test 2026-02-20 01:28:05 -05:00
fitbit-2026-02/FullHumanName Added fitbit test fixture 2026-03-02 08:42:47 -05:00
snapchat-2023-11 Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
discord-chat-exporter-2026-02.md Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
discord-json-2021-01.md Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
facebook-json-2021-05-01.zip Added zipped facebook export test 2026-02-21 06:06:50 -05:00
facebook-json.md Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
fitbit-2026-02.md Added fitbit test fixture 2026-03-02 08:42:47 -05:00
README.md Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00
snapchat-2023-11.md Added 3 more scrubbed fixtures for new exports, scrub added boolean and numeric key scrubbing 2026-02-27 03:39:42 -05:00

Addition process

  • Paste in a new folder
  • Manually scan through folders and delete any long lists of files. For example, facebook chats, we dont need 100 chat thread .json files
  • Run util/scrub.ts inside that folder
  • Fix any warnings that come out of the above for manual edits
  • Manually blank out any file/folder names with IDs or PII
  • Manually scan through each file for any remaining PII

Corpus