CypherpunkSamurai
/
NewPipeExtractor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,715 Commits 2 Branches 0 Tags 95 MiB
Java 99.8%
JavaScript 0.2%
Go to file
FireMasterK 2eeb0a3403
Rebase + some code improvements + fix extraction of age-restricted videos + update clients version 
Here is now the requests which will be made by the `onFetchPage` method of `YoutubeStreamExtractor`:

- the desktop API is fetched.

If there is no streaming data, the desktop player API with the embed client screen will be fetched (and also the player code), then the Android mobile API.
- if there is no streaming data, a `ContentNotAvailableException` will be thrown by using the message provided in playability status

If the video is age restricted, a request to the next endpoint of the desktop player with the embed client screen will be sent.
Otherwise, the next endpoint will be fetched normally, if the content is available.

If the video is not age-restricted, a request to the player endpoint of the Android mobile API will be made.

We can get more streams by using the Android mobile API but some streams may be not available on this API, so the streaming data of the Android mobile API will be first used to get itags and then the streaming data of the desktop internal API will be used.
If the parsing of the Android mobile API went wrong, only the streams of the desktop API will be used.

Other code changes:

- `prepareJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareDesktopJsonBuilder`
- `prepareMobileJsonBuilder` in `YoutubeParsingHelper` was renamed to `prepareAndroidMobileJsonBuilder`
- two new methods in `YoutubeParsingHelper` were added: `prepareDesktopEmbedVideoJsonBuilder` and `prepareAndroidMobileEmbedVideoJsonBuilder`
- `createPlayerBodyWithSts` is now public and was moved to `YoutubeParsingHelper`
- a new method in `YoutubeJavaScriptExtractor` was added: `resetJavaScriptCode`, which was needed for the method `resetDebofuscationCode` of `YoutubeStreamExtractor`
- `areHardcodedClientVersionAndKeyValid` in `YoutubeParsingHelper` returns now a `boolean` instead of an `Optional<Boolean>`
- the `fetchVideoInfoPage` method of `YoutubeStreamExtractor` was removed because YouTube returns now 404 for every client with the `get_video_info` page
- some unused objects and some warnings in `YoutubeStreamExtractor` were removed and fixed

Co-authored-by: TiA4f8R <74829229+TiA4f8R@users.noreply.github.com>
 		2021-08-01 12:39:03 +02:00
.github Merge pull request #552 from FireMasterK/dependabot 2021-06-17 23:48:41 +02:00
extractor Rebase + some code improvements + fix extraction of age-restricted videos + update clients version 2021-08-01 12:39:03 +02:00
gradle/wrapper Update gradle wrapper. 2021-02-18 21:15:12 +05:30
timeago-parser Bump spotbugs-annotations from 4.0.2 to 4.2.3 2021-06-23 09:57:42 +00:00
.gitignore ignore workspace and folder configs 2020-05-11 21:23:50 +02:00
LICENSE initial commit 2017-03-01 18:47:52 +01:00
README.md Release version 0.21.7 2021-07-28 20:38:45 +02:00
build.gradle Release version 0.21.7 2021-07-28 20:38:45 +02:00
copyright initial commit 2017-03-01 18:47:52 +01:00
gradlew fix duration can not be paresd 2019-07-31 16:11:42 +02:00
gradlew.bat fix duration can not be paresd 2019-07-31 16:11:42 +02:00
settings.gradle Created gradle module and moved existing code to new one 2018-03-14 00:44:02 -03:00

README.md

NewPipe Extractor

CI JIT Pack Badge JDoc • Documentation

NewPipe Extractor is a library for extracting things from streaming sites. It is a core component of NewPipe, but could be used independently.

Usage

NewPipe Extractor is available at JitPack's Maven repo.

If you're using Gradle, you could add NewPipe Extractor as a dependency with the following steps:

  1. Add maven { url 'https://jitpack.io' } to the repositories in your build.gradle.
  2. Add implementation 'com.github.TeamNewPipe:NewPipeExtractor:v0.21.7'the dependencies in your build.gradle. Replace v0.21.7 with the latest release.

Note: To use NewPipe Extractor in projects with a minSdkVersion below 26, API desugaring is required.

Testing changes

To test changes quickly you can build the library locally. A good approach would be to add something like the following to your settings.gradle:

includeBuild('../NewPipeExtractor') {
    dependencySubstitution {
        substitute module('com.github.TeamNewPipe:NewPipeExtractor') with project(':extractor')
    }
}

Another approach would be to use the local Maven repository, here's a gist of how to use it:

  1. Add mavenLocal() in your project repositories list (usually as the first entry to give priority above the others).
  2. It's recommended that you change the version of this library (e.g. LOCAL_SNAPSHOT).
  3. Run gradle's ìnstall task to deploy this library to your local repository (using the wrapper, present in the root of this project: ./gradlew install)
  4. Change the dependency version used in your project to match the one you chose in step 2 (implementation 'com.github.TeamNewPipe:NewPipeExtractor:LOCAL_SNAPSHOT')

Tip for Android Studio users: After you make changes and run the install task, use the menu option File → "Sync with File System" to refresh the library in your project.

Supported sites

The following sites are currently supported:

  • YouTube
  • SoundCloud
  • media.ccc.de
  • PeerTube (no P2P)
  • Bandcamp

License

GNU GPLv3 Image

NewPipe is Free Software: You can use, study share and improve it at your will. Specifically you can redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.