CypherpunkSamurai
/
NewPipeExtractor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,959 Commits 2 Branches 0 Tags 95 MiB
Java 99.8%
JavaScript 0.2%
Go to file
TiA4f8R 05b7fee23b
[YouTube] Add the cpn param to playback requests and try to spoof better the Android client 
The cpn param, aka the content playback nonce param, is a parameter sent by YouTube web client in videoplayback requests, and for some of them, in the player request body. This PR adds it everywhere.

For the desktop/WEB client, some params were missing from the playbackContext object, which seemed (or not) to make YouTube throttle streams extracted from the WEB client. This PR adds them.

Fingerprinting on the WEB client basing on the client version used is not possible anymore, because the latest client version is extracted at the first time of a YouTube request on a session which require the extractor to fetch again the website (and this may come back the reCaptcha issues again unfortunately, but it seems there is no other way to get it).

For the Android client, the video id is now also sent as a query parameter, like a 12 characters string, in the t query parameter, in order to spoof better this client. Researches need to be done on this parameter, unique to each request, and how it is generated by clients.

This commit also fixes a small bug with the Android User-Agent string.

Some code improvements have been also made.
 		2022-03-27 20:51:38 +02:00
.github CI: Upload test reports when failure occurs 2022-03-26 19:57:17 +01:00
checkstyle Legitimately suppress remaining checkstyle warnings 2022-03-26 19:43:08 +01:00
extractor [YouTube] Add the cpn param to playback requests and try to spoof better the Android client 2022-03-27 20:51:38 +02:00
gradle/wrapper Update gradle wrapper to 7.1.1 2021-08-07 17:51:18 -04:00
timeago-parser Updated to JUnit 5 2021-12-27 21:08:08 +01:00
.gitignore ignore workspace and folder configs 2020-05-11 21:23:50 +02:00
LICENSE Update the license to the latest version of https://www.gnu.org/licenses/gpl-3.0.txt 2022-03-19 17:59:27 +01:00
README.md Update README.md 2021-12-11 16:06:29 +01:00
build.gradle Fixed typo 2022-03-26 19:43:09 +01:00
gradlew Update gradle wrapper to 7.1.1 2021-08-07 17:51:18 -04:00
gradlew.bat Update gradle wrapper to 7.1.1 2021-08-07 17:51:18 -04:00
settings.gradle Created gradle module and moved existing code to new one 2018-03-14 00:44:02 -03:00

README.md

NewPipe Extractor

CI JIT Pack Badge JDoc • Documentation

NewPipe Extractor is a library for extracting things from streaming sites. It is a core component of NewPipe, but could be used independently.

Usage

NewPipe Extractor is available at JitPack's Maven repo.

If you're using Gradle, you could add NewPipe Extractor as a dependency with the following steps:

  1. Add maven { url 'https://jitpack.io' } to the repositories in your build.gradle.
  2. Add implementation 'com.github.TeamNewPipe:NewPipeExtractor:INSERT_VERSION_HERE' to the dependencies in your build.gradle. Replace INSERT_VERSION_HERE with the latest release.

Note: To use NewPipe Extractor in projects with a minSdkVersion below 26, API desugaring is required.

Testing changes

To test changes quickly you can build the library locally. A good approach would be to add something like the following to your settings.gradle:

includeBuild('../NewPipeExtractor') {
    dependencySubstitution {
        substitute module('com.github.TeamNewPipe:NewPipeExtractor') with project(':extractor')
    }
}

Another approach would be to use the local Maven repository, here's a gist of how to use it:

  1. Add mavenLocal() in your project repositories list (usually as the first entry to give priority above the others).
  2. It's recommended that you change the version of this library (e.g. LOCAL_SNAPSHOT).
  3. Run gradle's ìnstall task to deploy this library to your local repository (using the wrapper, present in the root of this project: ./gradlew install)
  4. Change the dependency version used in your project to match the one you chose in step 2 (implementation 'com.github.TeamNewPipe:NewPipeExtractor:LOCAL_SNAPSHOT')

Tip for Android Studio users: After you make changes and run the install task, use the menu option File → "Sync with File System" to refresh the library in your project.

Supported sites

The following sites are currently supported:

  • YouTube
  • SoundCloud
  • media.ccc.de
  • PeerTube (no P2P)
  • Bandcamp

License

GNU GPLv3 Image

NewPipe is Free Software: You can use, study share and improve it at your will. Specifically you can redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.