TobiGr
62912ee834
Merge branch 'master' into dev
2020-10-28 14:28:06 +01:00
Scratch
947ce3ee10
Fix parsing Soundcloud tracks that contain the term 'sets'
2020-10-28 07:55:02 +11:00
6dc5ab4015
find playerUrl in another place when assetsPattern regex fails
2020-10-27 13:48:58 +01:00
db0ef83d6b
fix youtube decryption and three attemps bug
...
fixes teamnewpipe/newpipe#4572 fixes #439
2020-10-27 13:40:24 +01:00
c1e9857960
fix subscriber count when subscribe is disabled
...
fixes #305
2020-10-26 19:06:01 +01:00
a39a2cca82
fix redirect channels
2020-10-26 19:05:19 +01:00
0a12300c5e
polish tests
2020-10-26 16:57:37 +01:00
01f49e8f66
polish strings
2020-10-26 16:32:39 +01:00
Tobias Groza
14c0c37d64
Merge pull request #309 from Stypox/improve-stream-tests
...
Improve stream extractor tests & various fixes
2020-10-25 18:49:53 +01:00
Kiril Isakov
366ed71523
[YouTube] Ignore leading characters in video id
2020-10-25 14:34:20 +01:00
Stypox
8026304a0a
Merge pull request #409 from Bartoshr/bugfix/short-channel-urls
...
Support short custom YouTube channel URLs
2020-10-25 11:43:50 +01:00
Stypox
57e7994c9e
Add some missing finals, nullables and comments
2020-10-25 08:12:41 +01:00
Stypox
f11fe87688
[YouTube] Replace outdated PewDiePie video test with another one
...
The old video was made private, and this video (wedding) is probably never going to be removed.
2020-10-24 18:41:41 +02:00
Stypox
3c55ea9321
[PeerTube] Change age restricted video in tests
...
The old one wasn't available anymore
2020-10-24 18:41:41 +02:00
Stypox
880ff04a5c
[Test] Add stream dash mpd url test
2020-10-24 18:41:41 +02:00
Stypox
d0b14644bb
[YouTube/MediaCCC] Consider dates as GMT and not as local
2020-10-24 18:41:41 +02:00
Stypox
8ce711f40f
[Test] Add sub channel name, url and thumbnail tests
2020-10-24 18:41:41 +02:00
Stypox
6127826571
[Test] Add stream metadata tests
2020-10-24 18:41:41 +02:00
Stypox
a087b092b4
[Test] Improve code style and add final
2020-10-24 18:39:55 +02:00
Stypox
a4097d8d01
[MediaCCC] Return empty list of video-only streams instead of null
2020-10-24 18:39:55 +02:00
Stypox
8dc3f28618
[PeerTube] Test one channel url with api and one without
2020-10-24 18:39:55 +02:00
Stypox
68d23defba
[YouTube] Do not catch every exception on getErrorMessage
...
@B0pol suggestion
2020-10-24 18:39:55 +02:00
Stypox
3191bd6c70
[YouTube] Use final when possible
2020-10-24 18:39:55 +02:00
Stypox
55bc01d1ce
[SoundCloud] Use final when possible, ide refactorings
2020-10-24 18:39:55 +02:00
Stypox
06430c4749
[PeerTube] Use final when possible, ide refactorings
2020-10-24 18:39:06 +02:00
Stypox
fcb9b6f855
[MediaCCC] Use final when possible, ide refactorings
...
Refactorings on `throws` clause
2020-10-24 18:36:22 +02:00
Stypox
af5b8b1915
[MediaCCC] Add tests for stream and conference link handlers
2020-10-24 18:36:21 +02:00
Stypox
07a90d116a
[MediaCCC] Use regex to parse stream and conference urls
2020-10-24 18:36:21 +02:00
Stypox
0c980b2d64
[PeerTube] Improve channel and stream link handler tests
2020-10-24 18:36:21 +02:00
Stypox
d130fd79c3
[PeerTube] Prepend "accounts/" to channel id for backward compatibility
2020-10-24 18:36:21 +02:00
Stypox
492db83ccf
[MediaCCC] Return null instead of empty items collector
...
As per the documentation in the base getRelatedStreams()
2020-10-24 18:34:57 +02:00
Stypox
3b2a1829d4
[MediaCCC] Extract tags
2020-10-24 18:34:57 +02:00
Stypox
b461da792f
[MediaCCC] Fix link handler inconsistency providing API links
2020-10-24 18:34:34 +02:00
Stypox
aeeae87641
[PeerTube] Parse timestamp from url (previously unimplemented)
2020-10-24 18:33:42 +02:00
Stypox
7ae3cb6d07
[PeerTube] Fix link handler inconsistency providing API links
2020-10-24 18:33:11 +02:00
Stypox
4349be13af
[PeerTube] Return empty audio stream list instead of null
2020-10-24 18:25:56 +02:00
Stypox
3b2cfb4ca2
[SoundCloud] Return empty video stream list instead of null
...
Also replace every instance of `return new ArrayList<>();` with `return Collections.emptyList();`
2020-10-24 18:23:46 +02:00
Stypox
072bae321f
[YouTube] Fix frame extraction for livestreams
...
Use saved playerResponse instead of parsing json every time
2020-10-24 18:23:46 +02:00
Stypox
7cd410f3fc
[YouTube] Return 0 when there is no timestamp, not -2, as per javadoc
2020-10-24 18:23:45 +02:00
Stypox
7fb867c166
[YouTube] Fix error message obtaining when there is none
2020-10-24 18:22:44 +02:00
Stypox
1d7a86e664
[Test] Add base classes for stream extractor tests
...
Refactor all stream extractor tests to use new base class.
Remove check if upload date is in the past: this does not have to hold true: youtube premieres turn up in search results even though they are in the future
2020-10-24 18:22:44 +02:00
Bartosz Rumiński
29695aed0a
Small field refactor
2020-10-23 16:42:30 +02:00
Bartosz Rumiński
0e67d820bc
Use static regex pattern for excluded path segments
2020-10-22 20:13:29 +02:00
Bartosz Rumiński
d3f80d1538
Exlude links which are not channels
2020-10-20 20:06:06 +02:00
Stypox
6887d59570
[YouTube] Handle urls for Shorts
2020-10-18 12:03:01 +02:00
Scratch
e945f711c1
Fix SoundCloud test artist account name ( #416 )
2020-10-16 23:42:46 +02:00
wb9688
be9a6f931c
Fix parsing new ytInitialData
2020-10-16 20:27:40 +02:00
Tobias Groza
1a9bc8caf1
Merge pull request #414 from Scrxtchy/soundcloud-slash
...
Remove tailing slash from SoundCloud URLs
2020-10-16 09:48:44 +02:00
Scratch
6a70cb9d50
Remove tailing slash from SoundCloud URLs
...
Fixes #412
2020-10-16 17:17:40 +11:00
Tobias Groza
9a785f9a7e
Merge pull request #413 from Scrxtchy/soundcloud-test-update
...
Fix SoundCloud test artist account name
2020-10-16 08:05:44 +02:00
Scratch
538f5d3973
Fix SoundCloud test artist account name
2020-10-16 13:45:05 +11:00
Stypox
19e862657a
[YouTube] Fix some decryption exceptions by retrying
2020-10-15 20:16:44 +02:00
Bartosz Rumiński
e3f996e014
Exlude links which are not channels
2020-10-12 20:59:56 +02:00
Bartosz Rumiński
5ab1b053d2
Update youtube channel link handler tests
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 20:11:28 +02:00
Bartosz Rumiński
7abb4b3713
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/linkHandler/YoutubeChannelLinkHandlerFactory.java
...
Fix typos
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:57:45 +02:00
Bartosz Rumiński
9d63211a66
Fix typos
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-10-12 19:56:53 +02:00
Bartosz Rumiński
8c38a5509e
Prevent attribution_link urls to be accepted by channel links handler
2020-10-09 19:07:38 +02:00
Bartosz Rumiński
4e04991762
Support short custom youtube channel urls
2020-10-09 17:06:59 +02:00
wb9688
ebbfe7f6d4
Skip YouTube's OTF streams
2020-09-29 10:48:02 +02:00
Tobias Groza
6633f26ec5
Merge pull request #386 from mhmdanas/remove-subtitles-stream-get-url
...
Remove SubtitlesStream#getURL()
2020-08-26 21:32:36 +02:00
wb9688
7657c2ed1a
Use initSafeStandardsObjects()
2020-08-15 17:08:07 +02:00
mhmdanas
1a63dcb355
Remove overriden field url
2020-08-12 22:37:36 +03:00
mhmdanas
f0f1c009b2
Remove SubtitlesStream#getURL()
2020-08-10 19:28:56 +03:00
TobiGr
0fb73301e3
[YouTube] Fix crash on empty comment
...
Closes #380
2020-08-05 18:25:40 +02:00
TobiGr
5ac80624a4
Fix detection of YouTube's decryption function
...
Closes TeamNewPipe/NewPipe#3951
2020-07-28 01:01:10 +02:00
Tobias Groza
32dff1541f
Merge pull request #370 from wb9688/fix-yt-age-restricted
...
Fix age restricted YouTube videos
2020-07-26 14:17:19 +02:00
TobiGr
ec778200d9
Add comments to the code
2020-07-26 14:15:13 +02:00
wb9688
667dce034c
Only use fallback methods for clientVersion when clientVersion hasn't been successfully extracted yet
2020-07-26 13:14:25 +02:00
wb9688
f345f667e2
Extract YouTube's key
2020-07-26 12:01:59 +02:00
wb9688
4c987a5302
Support YouTube's new continuations for search
2020-07-26 10:01:03 +02:00
wb9688
81459e289f
Fix age restricted YouTube videos
2020-07-18 19:52:54 +02:00
wb9688
fc3a63fec5
Update Javadoc for getRelatedStreams()
2020-07-15 18:48:37 +02:00
wb9688
ff560e907a
Remove getNextStream() from StreamExtractor
2020-07-15 18:48:37 +02:00
Tobias Groza
a70cb0283f
Merge pull request #314 from wb9688/remove-getnextpageurl
...
Next page stuff
2020-07-07 20:49:38 +02:00
wb9688
0a5a905bc7
Add final at more places
2020-07-07 20:45:47 +02:00
wb9688
17ba8a57fa
Clean up the code
2020-07-06 20:19:31 +02:00
wb9688
9b6fe1dea6
Throw IllegalArgumentException when Page is invalid
2020-07-06 20:19:31 +02:00
wb9688
4cc312086a
Introduce Page class
2020-07-06 20:19:31 +02:00
wb9688
e3bfdba135
Remove getNextPageUrl() function from ListExtractor
2020-07-06 20:11:40 +02:00
wb9688
156a26b64b
Support start= timestamps for YouTube
2020-07-06 14:08:40 +02:00
261471e7f4
remove invidiou.sh from tests
...
dead instance
2020-07-02 21:33:45 +02:00
eb3901acee
update invidious instances
2020-07-02 21:31:05 +02:00
Tobias Groza
c0ceb5cb27
Merge pull request #340 from wb9688/peertube-deleted-comments
...
Handle isDeleted for PeerTube comments
2020-06-28 10:21:37 +02:00
Tobias Groza
339f1d9c43
Merge pull request #349 from wb9688/okhttp
...
Use OkHttp for tests like in NewPipe
2020-06-27 20:08:02 +02:00
Tobias Groza
970bc7f69d
Merge pull request #359 from mauriciocolli/fix-client-id-bug
...
[SoundCloud] Fix concurrency issue when getting the client id
2020-06-27 17:36:24 +02:00
Robin
576754982e
Update extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java
...
Co-authored-by: Tobias Groza <TobiGr@users.noreply.github.com>
2020-06-27 11:35:01 +02:00
Mauricio Colli
7505e828cc
[SoundCloud] Update client id
2020-06-24 01:50:09 -03:00
Mauricio Colli
d2f1c0f40d
[SoundCloud] Fix concurrency issue when getting the client id
2020-06-24 01:50:08 -03:00
Robin
b63ae93495
Added a comment
...
and fixed a "typo" in the method parseDurationString replacing Long with Integer.
These are not the droids you are looking for wb9688 😁
2020-06-15 11:27:44 +02:00
Tobias Groza
92f6754f0f
Merge pull request #338 from wb9688/parse-duration-string-test
...
Add tests for parseDurationString()
2020-06-15 11:01:21 +02:00
Stypox
5808439c3f
[PeerTube] Change age restricted video in tests
...
The old one wasn't available anymore
2020-06-14 19:57:39 +02:00
wb9688
3239aa84f2
Handle isDeleted for PeerTube comments
2020-06-14 19:27:20 +02:00
wb9688
a65e46e4b1
Use OkHttp for tests like in NewPipe
2020-06-11 15:06:08 +02:00
063f4be766
For #342 : support PeerTube embeds
2020-06-04 14:26:42 +02:00
wb9688
33b1121fc7
Add tests for parseDurationString()
2020-05-30 17:20:54 +02:00
wb9688
e39147202c
Fix YouTube video duration parsing
2020-05-30 09:31:08 +02:00
wb9688
98055a3c3c
Revert "Add more decryption function regexes"
...
This reverts commit 384d6acd5d
2020-05-29 18:41:40 +02:00
Tobias Groza
bdb0f2ae6b
Merge pull request #332 from wb9688/learning-playlist
...
Support YouTube's learning playlists
2020-05-29 10:11:08 +02:00
Tobias Groza
cf18cdb2f5
Merge pull request #329 from wb9688/decryption-regexes
...
Add more decryption function regexes
2020-05-29 10:10:50 +02:00
wb9688
ab77961b0a
Extract trailer thumbnail in YT learning playlists
2020-05-29 09:55:24 +02:00
wb9688
4d683e7655
Support YouTube's learning playlists
2020-05-29 09:17:03 +02:00
Tobias Groza
0b4977bb0c
Merge pull request #333 from wb9688/playerresponse-dashmanifesturl
...
Extract dashManifestUrl in playerResponse
2020-05-28 21:17:07 +02:00
wb9688
384d6acd5d
Add more decryption function regexes
2020-05-28 11:40:38 +02:00
wb9688
8f71bde776
Fix bug for YT videos longer than 1000h
2020-05-28 11:34:42 +02:00
wb9688
aa277e1b6a
Upgrade other dependencies
2020-05-28 11:34:42 +02:00
wb9688
036cc7e26e
Upgrade jsoup
2020-05-28 11:34:41 +02:00
wb9688
7ef3e80493
Extract dashManifestUrl in playerResponse
2020-05-15 12:07:26 +02:00
wb9688
efafbbb88a
Merge pull request #331 from wb9688/playlist-subchannel
...
Add sub-channel stuff to PlaylistInfo
2020-05-12 10:49:13 +02:00
wb9688
baf5dd3e8f
Merge pull request #311 from B0pol/fixSearchSuggestion
...
Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSea…
2020-05-12 09:29:15 +02:00
wb9688
102f04e297
Add sub-channel stuff to PlaylistInfo
2020-05-12 09:14:29 +02:00
Tobias Groza
b40ccb5075
Merge pull request #307 from B0pol/playlists_peertube
...
[PeerTube] playlist support
2020-05-11 20:53:03 +02:00
adaf196c99
remove unused import
2020-05-11 12:54:33 +02:00
de26e00079
changed all == null || isEmpty() to isNullOrEmpty()
2020-05-11 11:40:24 +02:00
202a73516c
refactor: add Utils.isNullOrEmpty()
2020-05-11 11:26:18 +02:00
wb9688
1de0876fb4
Fix #327
2020-05-09 20:06:50 +02:00
7cae95bf09
add tests for subchannel stuff in playlists
2020-05-09 10:15:35 +02:00
1a6e92ebf6
add getSubChannel Name, Url and AvatarUrl for playlists
2020-05-09 09:52:24 +02:00
cf99107745
address yausername change
2020-05-09 00:52:04 +02:00
5bab9d9fc0
[PeerTube] playlist support & refactoring
2020-05-09 00:52:04 +02:00
Tobias Groza
f3913e241e
Merge pull request #324 from Stypox/music-detector-url
...
[YouTube] Support stream urls in `vnd.youtube://ID` form
2020-05-08 17:43:04 +02:00
wb9688
7320108c66
Rename SubChannel in channels back to ParentChannel
2020-05-08 16:02:59 +02:00
Roy Yosef
a408661771
Fix PR review
...
Rename "ParentChannel" to "SubChannel"
Add Tests for sub-channel info fetching
2020-05-07 20:20:12 +02:00
Roy Yosef
1de1f97cf1
Add parent channel info to StreamExtractor: name, url & avatar url
2020-05-07 20:20:12 +02:00
Roy Yosef
4234740baa
Add parent channel info to ChannelExtractor: name, url & avatar url
2020-05-07 20:20:12 +02:00
6cff5de400
renaming methods for consistency
2020-05-07 15:40:41 +02:00
5760366d70
use JsonObject#isEmpty instead of JsonObject#equals(JsonUtils.EMPTY_OBJECT)
2020-05-07 15:40:41 +02:00
93e868634e
clarified javadoc for isCorrectedSearch and getSearchSuggestions in SearchExtractor
2020-05-07 15:40:41 +02:00
3757541a9c
adapt new nanojson version
2020-05-07 15:40:41 +02:00
79483a6dc0
«Use this in SearchInfo, for consistency's sake» @Stypox order xD
2020-05-07 15:40:41 +02:00
e8d58e09c9
getSearchSuggestion now returns the corrected query if isCorrectedSearch() is true
2020-05-07 15:40:41 +02:00
5d1c3b3fa2
Use setters in SearchInfo
2020-05-07 15:39:57 +02:00
29b639b454
Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSearchSuggestion()
...
isCorrectedSearch: see the javadoc
getSearchSuggestion: in YoutubeSearchExtractor, it was giving the corrected search query. It now gives the suggested query, as it should
2020-05-07 15:39:57 +02:00
Stypox
a02bfbeae5
[YouTube] Support stream urls in vnd.youtube://ID form
2020-05-07 15:31:31 +02:00
bc13e0c616
Merge remote-tracking branch 'upstream/dev' into soundcloudComments
2020-05-05 14:49:21 +02:00
wb9688
6aae99f832
Fix NPE in defaultAlertsCheck()
2020-05-03 10:28:45 +02:00
wb9688
a1eabc7589
Return null instead of "" in getTextFromObject()
2020-05-02 08:02:25 +02:00
Tobias Groza
49157fcf0d
Merge pull request #317 from wb9688/tnp-nanojson
...
Use our fork of nanojson
2020-04-24 21:54:35 +02:00
2564bcf399
refactor comments
...
rename methods in CommentsInfoItemExtractor interface to match the other method names across NewPipeExtractor
remove getName in (Youtube|SoundCloud|Peertube)CommentsExtractor and move it up in CommentsExtractor, return "Comments" instead
2020-04-21 20:05:54 +02:00
wb9688
b51699a20e
Implement @TobiGr's suggestions
2020-04-20 14:27:33 +02:00
wb9688
74f0ab1792
Merge pull request #301 from wb9688/media-ccc-non-api-urls
...
Use media.ccc.de instead of api.media.ccc.de
2020-04-19 17:06:57 +02:00
0cbbc2a1f9
remove unused import
2020-04-19 14:16:38 +02:00
wb9688
979c5a7502
Use our fork of nanojson
2020-04-17 19:36:42 +02:00
d4352f9b84
support comments for SoundCloud
2020-04-10 20:41:41 +02:00
Roy Yosef
4afe657f6f
Add tests for getUrlFromId and test for video-channels in acceptUrlTest
2020-04-10 19:05:38 +03:00
Roy Yosef
2c9f1260eb
Fix wb9688 review comments
...
* Rename PeertubeUserExtractor to PeertubeAccountExtractor
* Add test for video-channels in PeertubeChannelLinkHandlerFactoryTest
* Compatibility support for older versions (use "accounts/" as default)
2020-04-10 16:40:53 +03:00
636c430743
refactor: create extractors and linkHandler packages for SoundCloud, move YoutubeParsingHelper to youtube package (not linkhandler anymore)
2020-04-10 10:51:05 +02:00
Roy Yosef
b6e6f403a8
add support for PeerTube channels extraction
2020-04-09 20:37:49 +03:00
a20d53dd98
remove the mention of Opus on wb9688 advise when
...
ContentNotSupportedException is thrown
2020-04-08 15:31:11 +02:00
9a7c6b7ab0
use ContentNotSupportedException for channels without tabs
2020-04-07 14:33:43 +02:00
a22104cbda
add tests for ContentNotSupportedException
2020-04-07 14:33:43 +02:00
