Open source ctcci

Closes #8

lol it took me 4 months

README.md

@@ -28,16 +28,38 @@ If you're affected by this, ensure that your F-Droid version is 1.7 or higher. A
 
 #### Toolchain setup
 
-- Get apktool (due to 2 bugs present in v2.3.4, you're strongly recommended to use v2.4.0)
+- Get apktool (due to 2 bugs present in v2.3.4, you're strongly recommended to use v2.4.0 or higher)
 - Get a keystore, see [here](https://stackoverflow.com/a/14994354/3286892), step 1.
 - If you want Mutant Standard emoji patches, get 72x72 PNG copies of latest version of mutant standard emojis with codepoints. I have a zip [here](https://mutant.lavatech.top/72x72.zip).
 - If you want Blob emoji patches, get 72x72 PNG copies of blobmojis with codepoints. I personally resized the png/128 folder in this [repo](https://github.com/C1710/blobmoji) (`find /home/ave/blobmoji/png/72 -iname '*.png' -exec convert \{\} -verbose -resize 72x72\> \{\} \;`).
 - Extract the emojis you got somewhere.
-- Clone this repo somewhere, edit `emojireplace.py` and set the `extracted_mutstd_path` folder to the folder you just extracted emojis to.
+- Clone this repo somewhere, edit `patches/mutant/emojireplace.py` and `patches/blobs/emojireplace.py`, set the `extracted_mutstd_path` and `extracted_blobmoji_path` folders respectively to the folder you just extracted emojis to.
+- Get a Discord apk (*cough* [apkmirror](https://www.apkmirror.com/apk/discord-inc/discord-chat-for-gamers/), [aptoide API](https://ws75.aptoide.com/api/7/app/getMeta?package_name=com.discord)).
+
+### Using CutTheCord CI
+
+CutTheCord CI (CTCCI) is what powers the official builds. It allows for extensive customization. It relies on distok files and state. While state is public, files aren't, so you'll have to create a similar file structure.
+
+CTCCI requires python3.6+, and also likely requires Linux (I haven't tried it on other OSes, and use of things like `patch` and pipes might limit it to Linux).
+
+#### Initial setup
+
+- Follow the "Toolchain setup" steps above in case you haven't already.
+- Install dependencies (`python3 -m pip install -Ur requirements.txt`)
+- Copy `ctcci/ctcconfig.example.py` to `ctcci/ctcconfig.py` and configure contents to your local data and your personal preferences.
+- If you don't want to always automatically be required to use the latest discord version, set `LOCAL_STATE` to `True` and point `STATE_FILE` to a `state.json` file that looks like this: `{"android": {"com.discord": {"version": 909}}}`.
+- Set up a local distok-style file storage for your APKs, and copy your APKs on it, and name them properly (the format is `$DISTOK_FOLDER/android/$PACKAGE_ID-$VERSION_NUMBER.apk`, example: `/home/ave/distok/android/com.discord-909.apk`).
+
+#### Running builds
+
+- Simply run the `ctcci.py` script, with first argument being branch name, and latter ones being name of patches you want to add in.
+
+Example: `python3.6 /home/ave/distokrepos/cutthecord/ctcci/ctcci.py ave mutant customfont slashcommands tokenlogin customtheme customdefaultemoji customring bettertm notrack noprofilestrip nonearby experiments noblocked squareavatars`
+
+### Manually
 
 #### Building a patched discord app
 
-- Get a Discord apk (*cough* [apkmirror](https://www.apkmirror.com/apk/discord-inc/discord-chat-for-gamers/), [aptoide API](https://ws75.aptoide.com/api/7/app/getMeta?package_name=com.discord)).
 - Extract it with apktool (`apktool d <apk path>`)
 - Get all the necessary patches for that version. Necessary patches are not available for all versions and are only required to get some versions to pack together correctly.
 - Get optional patches you want for your version. If the patch you want isn't available for your version, you'll have to port them yourself.

ctcci/.gitignore

@@ -0,0 +1 @@
+ctcconfig.py

ctcci/ctcci.py

@@ -0,0 +1,203 @@
+#!/usr/bin/env python3
+import os
+import sys
+import shutil
+import json
+import subprocess
+from ctcconfig import *
+
+print("Welcome to CutTheCord CI :)")
+
+BRANCH = sys.argv[1]
+PATCHES = sys.argv[2:]
+FORCE = False
+DEFAULT_PATCHES = ["necessary", "branding", "customversion"]
+OPTIONAL_PATCHES = ["necessary", "blobs"]
+
+
+def patch(patch_file, workdir, patch_name=""):
+    if not os.path.exists(patch_file):
+        # Allow missing optional patches
+        if patch_name in OPTIONAL_PATCHES:
+            return
+
+        raise FileNotFoundError(f"No patches with name \"{patch_file}\" :(")
+
+    subprocess.run(f"patch -p1 --no-backup-if-mismatch -i {patch_file}",
+                   shell=True, check=True, cwd=workdir)
+
+
+# Wipe and recreate the working folder
+if os.path.exists(WORK_APK_PATH):
+    shutil.rmtree(WORK_APK_PATH)
+
+os.makedirs(WORK_FOLDER, exist_ok=True)
+
+# Get the version of the APK from distok state file
+with open(STATE_FILE) as f:
+    STATE = json.load(f)
+    VERSION = STATE["android"][PACKAGE_ID]["version"]
+
+BASE_APK_PATH = os.path.join(WORK_FOLDER, f"discord-base-{VERSION}")
+
+# Prepare names of input and output APKs
+INPUT_FILE = os.path.join(DISTOK_FOLDER, "android",
+                          f"{PACKAGE_ID}-{VERSION}.apk")
+# OUTPUT_FILE = os.path.join(RESULT_FOLDER,
+#                            f"cutthecord-{VERSION}-{'_'.join(PATCHES)}.apk")
+OUTPUT_FILE = os.path.join(RESULT_FOLDER,
+                           f"cutthecord-{VERSION}-{BRANCH}.apk")
+
+# Add necessary patches to the list of patches that will be applied
+# Important to have this after the output file name generation
+# otherwise it'll include it, which is not wanted
+PATCHES += DEFAULT_PATCHES
+
+print(f"Branch: {BRANCH}, output name: {OUTPUT_FILE}")
+
+# Check if the version is already patched, if it is exit
+if not FORCE and os.path.exists(OUTPUT_FILE):
+    print("This version is already patched, bye!")
+    sys.exit(1)
+
+# Update cutthecord
+subprocess.run("git pull", shell=True, cwd=REPO_FOLDER)
+
+# Extract the APK if it's not already extracted to base cache
+if not os.path.exists(BASE_APK_PATH):
+    subprocess.run(f"{APKTOOL_BIN} d {INPUT_FILE} -o {BASE_APK_PATH} -f",
+                   shell=True,
+                   cwd=WORK_FOLDER)
+
+# Copy the base cache to work on it
+shutil.copytree(BASE_APK_PATH, WORK_APK_PATH)
+
+# Go through patches and apply every single one of them
+for patch_name in PATCHES:
+    # Apply custom emoji patches
+    if patch_name in ["mutant", "blobs"]:
+        print(f"Applying {patch_name} emoji patch")
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "emojireplace.py")
+        subprocess.run(f"{PYTHON_BIN} {patch_script}",
+                       shell=True,
+                       cwd=WORK_APK_PATH)
+
+    # Apply custom emoji patches
+    elif patch_name == "customtheme":
+        print(f"Applying splash patch")
+        splash = os.path.join(REPO_FOLDER, "patches",
+                              patch_name, "asset_loading.png")
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "fixsplash.sh")
+
+        subprocess.run(f"bash {patch_script} {splash}",
+                       shell=True,
+                       cwd=WORK_APK_PATH)
+
+    patch_file = os.path.join(REPO_FOLDER, "patches",
+                              patch_name, f"{VERSION}.patch")
+
+    # Apply custom version patches
+    if patch_name == "customversion":
+        print(f"Applying custom version")
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "addpatch.py")
+
+        subprocess.run(f"{PYTHON_BIN} {patch_script} {patch_file} "
+                       f"{' '.join(PATCHES)}",
+                       shell=True,
+                       cwd=WORK_APK_PATH)
+
+        patch(patch_file.replace(".patch", "-custom.patch"), WORK_APK_PATH)
+    # Apply branding patches
+    elif patch_name == "branding":
+        print(f"Applying branding icon patch")
+        if BRANCH in ICONS:
+            shutil.copyfile(ICONS[BRANCH],
+                            os.path.join(WORK_APK_PATH, "res",
+                                         "mipmap-xxxhdpi", "logo_debug.png"))
+        elif "default" in ICONS:
+            shutil.copyfile(ICONS["default"],
+                            os.path.join(WORK_APK_PATH, "res",
+                                         "mipmap-xxxhdpi", "logo_debug.png"))
+
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "customicon.sh")
+
+        subprocess.run(f"bash {patch_script}", shell=True, cwd=WORK_APK_PATH)
+
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "addpatch.py")
+
+        # Hell code
+        app_name = APP_NAMES.get(BRANCH, APP_NAMES.get("default", "CutTheCord"))
+
+        subprocess.run(f"{PYTHON_BIN} {patch_script} "
+                       f"{patch_file} {app_name} {BRANCH}",
+                       shell=True,
+                       cwd=WORK_APK_PATH)
+
+        patch(patch_file.replace(".patch", "-custom.patch"), WORK_APK_PATH)
+    elif patch_name in ["bettertm", "bettertmlight"]:
+        print(f"Applying bettertm emoji patch")
+        patch_dir = os.path.join(REPO_FOLDER, "patches",
+                                 patch_name)
+        patch_script = os.path.join(REPO_FOLDER, "patches",
+                                    patch_name, "bettertm.sh")
+        subprocess.run(f"bash {patch_script} {patch_dir}",
+                       shell=True,
+                       cwd=WORK_APK_PATH)
+    # Apply custom ringtone
+    elif patch_name == "customring":
+        print(f"Applying custom ringtone")
+        if BRANCH in RINGTONES:
+            CUSTOM_RINGTONE = RINGTONES[BRANCH]
+        else:
+            CUSTOM_RINGTONE = RINGTONES["default"]
+
+        shutil.copyfile(CUSTOM_RINGTONE,
+                        os.path.join(WORK_APK_PATH, "res",
+                                     "raw", "call_ringing.mp3"))
+    # Apply custom fonts
+    elif patch_name == "customfont":
+        print(f"Applying custom font")
+        if BRANCH in FONTS:
+            fonts = FONTS[BRANCH]
+        else:
+            fonts = FONTS["default"]
+
+        for font in fonts:
+            shutil.copyfile(fonts[font],
+                            os.path.join(WORK_APK_PATH, "res",
+                                         "font", font))
+    # Apply any other patches
+    else:
+        patch(patch_file, WORK_APK_PATH, patch_name)
+
+# Pack the APK
+subprocess.run(f"{APKTOOL_BIN} b discord",
+               shell=True,
+               cwd=WORK_FOLDER)
+
+APK_PATH = os.path.join(WORK_FOLDER, "discord",
+                        "dist", f"{PACKAGE_ID}-{VERSION}.apk")
+
+# Sign the APK
+subprocess.run(f"jarsigner -storepass {KEYSTORE_PASS} -keystore "
+               f"{KEYSTORE_FILE} {APK_PATH} {KEYSTORE_ALIAS}",
+               shell=True,
+               cwd=WORK_FOLDER)
+
+# Copy the result file
+shutil.copyfile(APK_PATH, OUTPUT_FILE)
+
+if DO_FDROID:
+    # Do fdroid magic
+    subprocess.run(f"fdroid update -c",
+                   shell=True,
+                   cwd=FDROID_FOLDER)
+
+# Wipe caches
+shutil.rmtree(WORK_APK_PATH)
+print("CutTheCord CI has fulfilled its purpose.")

ctcci/ctcconfig.example.py

@@ -0,0 +1,71 @@
+import os
+
+APKTOOL_BIN = "java -jar /home/ave/apktool-cli-all.jar"  # use latest
+
+# Set to false if you're going to use a local state.json file
+# state.json has information about the latest discord version
+LOCAL_STATE = False
+STATE_FILE = "/home/ave/distok/state.json"
+
+# APKs must be placed under $DISTOK_FOLDER/android/$PACKAGE_ID-$VERSION_NUMBER.apk
+# Example: /home/ave/distok/android/com.discord-909.apk
+DISTOK_FOLDER = "/home/ave/distok"
+
+# Set if F-Droid repo should be automatically updated or not
+# If set to False, you can leave FDROID_FOLDER empty
+DO_FDROID = True
+FDROID_FOLDER = "/var/www/fdroid/seabear"
+
+# This is where APKs will be placed
+RESULT_FOLDER = FDROID_FOLDER + "/repo"
+
+# Repo for CutTheCord (https://gitdab.com/distok/cutthecord)
+REPO_FOLDER = "/home/ave/distokrepos/cutthecord"
+
+# Keystore file, alias and pass. Required.
+KEYSTORE_FILE = "/home/ave/oof.keystore"
+KEYSTORE_ALIAS = "distok"
+KEYSTORE_PASS = "redacted"
+
+# Set this to the python version you want to use. Needs to be 3.6+.
+PYTHON_BIN = "python3.6"
+
+# Folder where the apk will be extracted to, patched in, and packed back on
+# You're not recommended to touch WORK_APK_PATH.
+WORK_FOLDER = "/tmp/cutthecord"
+WORK_APK_PATH = os.path.join(WORK_FOLDER, "discord")
+
+# Package ID for discord. You should probably not touch this.
+PACKAGE_ID = "com.discord"
+
+# Set to force if you want builds to be done
+# even if there's already a build for this version
+FORCE = False
+
+# Custom ringtones, default = applied to all the ones not explicitly stated
+RINGTONES = {"ave": "/home/ave/sans.mp3",
+             "default": "/home/ave/removeskype.mp3"}
+
+# Custom icons, default = applied to all the ones not explicitly stated
+ICONS = {"ave": "/home/ave/ctclogoave.png",
+         "default": "/home/ave/ctclogo.png"}
+
+# Custom app names, default = applied to all the ones not explicitly stated
+# Needs to be one word, __ gets replaced with space
+APP_NAMES = {"ave": "CutTheCord__Ave",
+             "litecord": "Litecord",
+             "test": "CTC__Testing",
+             "default": "CutTheCord"}
+
+# Custom fonts, default = applied to all the ones not explicitly stated
+FONTS = {"ellie": {"whitney_bold.ttf": "/home/ave/fonts/GoogleSans-Bold.ttf",
+                   "whitney_semibold.ttf": "/home/ave/fonts/GoogleSans-Medium.ttf",
+                   "whitney_medium.ttf": "/home/ave/fonts/GoogleSans-Regular.ttf"},
+        "dyslexic": {"whitney_bold.ttf": "/home/ave/fonts/OpenDyslexic3-Bold.ttf",
+                   "whitney_semibold.ttf": "/home/ave/fonts/OpenDyslexic3-Bold.ttf",
+                   "whitney_medium.ttf": "/home/ave/fonts/OpenDyslexic3-Regular.ttf"},
+        "murm": {"whitney_bold.ttf": "/home/ave/fonts/comicbd.ttf",
+                 "whitney_semibold.ttf": "/home/ave/fonts/comicbd.ttf",
+                 "whitney_medium.ttf": "/home/ave/fonts/comic.ttf"},
+        "ave": {"sourcecodepro_semibold.ttf": "/home/ave/fonts/comic.ttf"},
+        "default": {}}

ctcci/requirements.txt

@@ -0,0 +1 @@
+requests