Commit graph

314 commits

Author SHA1 Message Date
Tobias Groza
a70cb0283f
Merge pull request #314 from wb9688/remove-getnextpageurl
Next page stuff
2020-07-07 20:49:38 +02:00
wb9688
0a5a905bc7 Add final at more places 2020-07-07 20:45:47 +02:00
wb9688
17ba8a57fa Clean up the code 2020-07-06 20:19:31 +02:00
wb9688
4cc312086a Introduce Page class 2020-07-06 20:19:31 +02:00
wb9688
e3bfdba135 Remove getNextPageUrl() function from ListExtractor 2020-07-06 20:11:40 +02:00
wb9688
156a26b64b Support start= timestamps for YouTube 2020-07-06 14:08:40 +02:00
261471e7f4 remove invidiou.sh from tests
dead instance
2020-07-02 21:33:45 +02:00
Tobias Groza
c0ceb5cb27
Merge pull request #340 from wb9688/peertube-deleted-comments
Handle isDeleted for PeerTube comments
2020-06-28 10:21:37 +02:00
Tobias Groza
339f1d9c43
Merge pull request #349 from wb9688/okhttp
Use OkHttp for tests like in NewPipe
2020-06-27 20:08:02 +02:00
Tobias Groza
92f6754f0f
Merge pull request #338 from wb9688/parse-duration-string-test
Add tests for parseDurationString()
2020-06-15 11:01:21 +02:00
Stypox
5808439c3f
[PeerTube] Change age restricted video in tests
The old one wasn't available anymore
2020-06-14 19:57:39 +02:00
wb9688
3239aa84f2 Handle isDeleted for PeerTube comments 2020-06-14 19:27:20 +02:00
wb9688
a65e46e4b1 Use OkHttp for tests like in NewPipe 2020-06-11 15:06:08 +02:00
063f4be766 For #342: support PeerTube embeds 2020-06-04 14:26:42 +02:00
wb9688
33b1121fc7 Add tests for parseDurationString() 2020-05-30 17:20:54 +02:00
wb9688
4d683e7655 Support YouTube's learning playlists 2020-05-29 09:17:03 +02:00
Tobias Groza
0b4977bb0c
Merge pull request #333 from wb9688/playerresponse-dashmanifesturl
Extract dashManifestUrl in playerResponse
2020-05-28 21:17:07 +02:00
wb9688
036cc7e26e Upgrade jsoup 2020-05-28 11:34:41 +02:00
wb9688
7ef3e80493 Extract dashManifestUrl in playerResponse 2020-05-15 12:07:26 +02:00
wb9688
baf5dd3e8f
Merge pull request #311 from B0pol/fixSearchSuggestion
Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSea…
2020-05-12 09:29:15 +02:00
Tobias Groza
b40ccb5075
Merge pull request #307 from B0pol/playlists_peertube
[PeerTube] playlist support
2020-05-11 20:53:03 +02:00
de26e00079 changed all == null || isEmpty() to isNullOrEmpty() 2020-05-11 11:40:24 +02:00
202a73516c refactor: add Utils.isNullOrEmpty() 2020-05-11 11:26:18 +02:00
7cae95bf09 add tests for subchannel stuff in playlists 2020-05-09 10:15:35 +02:00
5bab9d9fc0 [PeerTube] playlist support & refactoring 2020-05-09 00:52:04 +02:00
Tobias Groza
f3913e241e
Merge pull request #324 from Stypox/music-detector-url
[YouTube] Support stream urls in `vnd.youtube://ID` form
2020-05-08 17:43:04 +02:00
wb9688
7320108c66 Rename SubChannel in channels back to ParentChannel 2020-05-08 16:02:59 +02:00
Roy Yosef
a408661771 Fix PR review
Rename "ParentChannel" to "SubChannel"
Add Tests for sub-channel info fetching
2020-05-07 20:20:12 +02:00
Roy Yosef
4234740baa Add parent channel info to ChannelExtractor: name, url & avatar url 2020-05-07 20:20:12 +02:00
3757541a9c adapt new nanojson version 2020-05-07 15:40:41 +02:00
e8d58e09c9 getSearchSuggestion now returns the corrected query if isCorrectedSearch() is true 2020-05-07 15:40:41 +02:00
29b639b454 Search: add isCorrectedSearch() and fix YoutubeSearchExtractor#getSearchSuggestion()
isCorrectedSearch: see the javadoc
getSearchSuggestion: in YoutubeSearchExtractor, it was giving the corrected search query. It now gives the suggested query, as it should
2020-05-07 15:39:57 +02:00
Stypox
a02bfbeae5
[YouTube] Support stream urls in vnd.youtube://ID form 2020-05-07 15:31:31 +02:00
bc13e0c616 Merge remote-tracking branch 'upstream/dev' into soundcloudComments 2020-05-05 14:49:21 +02:00
wb9688
a1eabc7589 Return null instead of "" in getTextFromObject() 2020-05-02 08:02:25 +02:00
2564bcf399 refactor comments
rename methods in CommentsInfoItemExtractor interface to match the other method names across NewPipeExtractor
remove getName in (Youtube|SoundCloud|Peertube)CommentsExtractor and move it up in CommentsExtractor, return "Comments" instead
2020-04-21 20:05:54 +02:00
wb9688
74f0ab1792
Merge pull request #301 from wb9688/media-ccc-non-api-urls
Use media.ccc.de instead of api.media.ccc.de
2020-04-19 17:06:57 +02:00
Roy Yosef
4afe657f6f Add tests for getUrlFromId and test for video-channels in acceptUrlTest 2020-04-10 19:05:38 +03:00
Roy Yosef
2c9f1260eb Fix wb9688 review comments
* Rename PeertubeUserExtractor to PeertubeAccountExtractor
* Add test for video-channels in PeertubeChannelLinkHandlerFactoryTest
* Compatibility support for older versions (use "accounts/" as default)
2020-04-10 16:40:53 +03:00
636c430743 refactor: create extractors and linkHandler packages for SoundCloud, move YoutubeParsingHelper to youtube package (not linkhandler anymore) 2020-04-10 10:51:05 +02:00
Roy Yosef
b6e6f403a8 add support for PeerTube channels extraction 2020-04-09 20:37:49 +03:00
9a7c6b7ab0 use ContentNotSupportedException for channels without tabs 2020-04-07 14:33:43 +02:00
a22104cbda add tests for ContentNotSupportedException 2020-04-07 14:33:43 +02:00
wb9688
bf24caa5f2 Use media.ccc.de instead of api.media.ccc.de 2020-04-01 13:25:10 +02:00
wb9688
2af610e3e7 Add tests for other YT Music search types 2020-04-01 10:30:51 +02:00
wb9688
cf0f2aff3e Extract uploader url from certain YouTube Music videos 2020-04-01 10:30:51 +02:00
wb9688
aa8cea47f3 Refactor YouTube Music search tests 2020-04-01 10:30:51 +02:00
wb9688
d58c0f230d Improve code for YouTube Music search 2020-04-01 10:30:51 +02:00
wb9688
dc29d87962 Extract YouTube search suggestions 2020-04-01 10:30:51 +02:00
wb9688
eb48524411 Add tests for YouTube Music search 2020-04-01 10:30:51 +02:00
Tobias Groza
69e0624e35
Merge pull request #298 from B0pol/feed_peertube
[peertube] add feed url in channels
2020-03-28 12:53:47 +01:00
0ae53c1e50 [peertube] adapt test for channel feed url 2020-03-28 11:48:24 +01:00
TobiGr
647e7cd450 Accept YouTube Music playlists but not YouTube Mixes 2020-03-25 22:24:18 +01:00
Tobias Groza
e4874d554e
Merge pull request #296 from mauriciocolli/fix-search-errors-detection
Fix search errors detection and refactor search tests
2020-03-24 21:53:32 +01:00
Mauricio Colli
0a20c53f1a
Use easier-to-read asserts statements in tests 2020-03-23 18:08:59 -03:00
Tobias Groza
6fd9b38ad9
Merge pull request #287 from mauriciocolli/fix-channel-redirect
[YouTube] Fix channel with redirects directly in the response
2020-03-23 18:41:56 +01:00
Mauricio Colli
04bfa0ec6a
[PeerTube] Update video used in age limit test (was 404 not found) 2020-03-21 15:55:40 -03:00
Mauricio Colli
9704fc9952
Improve search extractor tests for services 2020-03-21 15:55:36 -03:00
TobiGr
2cf8cbf0df Merge branch 'master' into dev 2020-03-18 22:29:10 +01:00
Stypox
265cfb61f3
[SoundCloud] Add test for small playlist, use defaultTestGetPageInNewExtractor 2020-03-17 21:03:50 +01:00
Stypox
ae47c9587c
[SoundCloud] Optimize imports in edited files 2020-03-17 20:32:04 +01:00
Stypox
4389fd3b7b
[SoundCloud] Migrate StreamExtractor to api-v2 2020-03-17 18:06:13 +01:00
Stypox
5e4ddb368f
[SoundCloud] Fix extractors built from next playlist pages
They didn't have the information to calculate another next page url. So now `nextPageUrl` contains a full link with all video ids, and `getPage` takes the first part of the url (containing 15 streams) and produces another `nextPageUrl` with the remaining streams.
Also add a test for this.
2020-03-17 18:04:40 +01:00
Stypox
0e1b4bbf17
[SoundCloud] Test playlists banner: it should not exist 2020-03-17 15:53:25 +01:00
Stypox
65bdb3bc9d
[SoundCloud] Unignore ignored playlist tests 2020-03-17 15:49:58 +01:00
Stypox
1558da6f6b
[SoundCloud] Fix playlist next page generation 2020-03-17 15:46:35 +01:00
Stypox
ca8bf53b61
[SoundCloud] Fix playlist test: number of streams changed 2020-03-17 15:35:33 +01:00
Stypox
4b1121aac7
[SoundCloud] Add tests for api-v2 channel urls 2020-03-17 15:23:13 +01:00
TobiGr
efad0be99f Test if the extractor can get AudiStreams with the hard-coded cliend_id
SoundCloud client_id are changed on a regular basis now. We might want to remove the hardcoded id completely on a later point of time when client_id is changed to often
2020-03-16 19:42:35 +01:00
Tobias Groza
65a7eda446
Merge pull request #256 from B0pol/mediaccc
Support for non-api mediaccc links, invidious shortened links, refactor mediaccc tests
2020-03-15 17:04:09 +01:00
TobiGr
40b3bc138f Remove hack to fix YoutubeStreamLinkHandlerFactoryTest of vnd.youtube videos 2020-03-14 21:54:54 +01:00
Stypox
957b78ed9b
[YouTube] Add test for an unlisted video, which has no next stream 2020-03-14 14:04:01 +01:00
Mauricio Colli
00d1ed439b
[YouTube] Fix channel extraction when redirects are in the response
Some redirects were embed directly into the response as instructions
for the page, instead of the usual http redirects.
2020-03-14 02:39:48 -03:00
66518ec444 check wether mediaccc streams url are secure 2020-03-13 20:07:36 +01:00
0cd5e05b7b MediaCCCLH: use substring instead of replace; improve a bit tests, return Collections.emptyList(); instead of null where it's annotated @NonNull 2020-03-13 20:00:49 +01:00
f742a6bd3e code optimization 2020-03-13 20:00:49 +01:00
e8e535b815 mediaccc: update linkhandlers & refactor tests 2020-03-13 20:00:49 +01:00
Mauricio Colli
6aa4d59b91
[PeerTube] Update video used in age limit test (the previous was 404) 2020-03-01 10:29:30 -03:00
Mauricio Colli
90ae5fbea2 Improve kiosk tests across services 2020-03-01 12:57:18 +01:00
Mauricio Colli
e590417cc4 Test if services recognizes their own items urls 2020-03-01 12:57:18 +01:00
Mauricio Colli
5686a6f562 [YouTube] Detect when a stream is deleted or doesn't exist
Added a test case as well.
2020-03-01 12:49:13 +01:00
Mauricio Colli
e65333c3ce [YouTube] Detect deleted/nonexistent/invalid channels and playlists
- Added tests for these cases.
2020-03-01 12:49:13 +01:00
Mauricio Colli
408f042127 [YouTube] Fix bug when url isn't present in the browseEndpoint object 2020-03-01 12:49:13 +01:00
Mauricio Colli
342bdbb852 [YouTube] Avoid crashing by letting exceptions bubble up 2020-03-01 12:49:13 +01:00
Mauricio Colli
5edd774fc4 Add latest url to the response to make detection of a redirect possible
Will be latest one in this commit because there's need to check the
history of redirects as of now.
2020-03-01 12:49:13 +01:00
Mauricio Colli
3441946bea Make test downloader return a response instead of throwing an exception
The test implementation was throwing an exception instead of just
returning the response and letting the caller handle it.
2020-03-01 12:49:13 +01:00
wb9688
3e4d2fd64e Improve testChannelUrl() 2020-02-29 18:20:35 +01:00
wb9688
bfe3eb1409 Use suggested try-if code style 2020-02-29 17:18:50 +01:00
wb9688
beb07a4852 Add test to check whether hardcoded client version is still valid 2020-02-28 16:36:14 +01:00
wb9688
365b0329f3 Implement getTextFromObject() function 2020-02-27 17:39:23 +01:00
wb9688
8d503407ef Parse browseEndpoint and watchEndpoint in descriptions 2020-02-26 19:37:58 +01:00
wb9688
8dc39d517a Fix tests
Except testGetFullLinksInDescription() in DescriptionTestUnboxing, since that requires other changes first.
2020-02-26 17:31:01 +01:00
TobiGr
9efcc61ca6 Adapt some tests to match the new URLs 2020-02-25 21:51:30 +01:00
TobiGr
f33f9466ce Fix testDescription() in ChannelExtractor 2020-02-25 21:51:30 +01:00
TobiGr
21253abd9b Use Firefox ESR User-Agent: 68 2020-02-25 21:51:30 +01:00
TobiGr
39bf1ff1eb Add ytInitialData regex 2020-02-25 21:51:30 +01:00
TobiGr
030465b5d4 Improve code formatting and optimise imports 2020-02-15 13:36:54 +01:00
TobiGr
008b6f505c [PeerTube] Fix subscriber count in ChannelExtractorTest 2020-02-08 23:08:37 +01:00
TobiGr
0e33249c85 Fix SoundCloud description test 2020-02-08 22:20:20 +01:00
0f8a7f9df2 fix testGetUploadDate for PeerTubeStreamExtractor 2020-02-07 13:33:50 +01:00