Commit graph

12547 commits

Author SHA1 Message Date
Xiao Di Guan
c64c03be35
[twitch] Add basic support for two-factor authentication 2017-03-05 01:06:27 +07:00
Lars Vierbergen
a3ba8a7acf [vier] Add support for vijf.be
vier.be and vijf.be run on the same CMS and are property of the same company,
so the same extractor can be used for both of them.
2017-03-05 00:47:19 +08:00
Sergey M․
054a587de8
[redbulltv] Improve extraction (closes #11948, closes #3919) 2017-03-04 23:28:21 +07:00
Juanjo Benages
64b7ccef3e
[redbulltv] Add extractor 2017-03-04 23:26:15 +07:00
Yen Chi Hsuan
6f4e4132d8
[douyutv] Switch to the PC API to escape the 5-min limitation
Thanks @spacemeowx2 for the algo.

Ref: https://gist.github.com/spacemeowx2/629b1d131bd7e240a7d28742048e80fc

Closes #12316
2017-03-04 23:23:18 +08:00
Sergey M․
eb3079b6ce
[generic] Add support for rutube embeds 2017-03-04 00:46:33 +07:00
Sergey M․
bc82f22879
[rutube] Relax _VALID_URL 2017-03-04 00:42:51 +07:00
Sergey M․
4d058c9862
[vrak] Improve and update test (closes #11452) 2017-03-03 23:58:16 +07:00
Sergey M․
d16f27ca27
[brightcove:new] Add ability to smuggle geo_countries into URL 2017-03-03 23:58:03 +07:00
Olivier Bilodeau
cbb127568a
[vrak] Add extractor 2017-03-03 23:54:21 +07:00
Sergey M․
d02d4fa0a9
[brightcove:new] Raise GeoRestrictedError 2017-03-03 22:49:48 +07:00
Sergey M․
692fa200ca
[go] Relax _VALID_URL (closes #12341) 2017-03-03 22:28:34 +07:00
Sergey M․
9bae185ba6
[24video] Use original host for requests (closes #12339) 2017-03-03 22:16:00 +07:00
Sergey M․
4d345bf17b
[ruutu] Disable DASH formats (closes #12322)
Due to causing out of sync issue
2017-03-02 23:53:46 +07:00
Sergey M․
250eea6821
release 2017.03.02 2017-03-02 22:33:22 +07:00
Yen Chi Hsuan
11bb6ad1a5
[facebook] Fix extraction (closes #12323)
Almost all videos now use the pagelet type 'permalink_video_pagelet'
2017-03-02 20:51:24 +08:00
Yen Chi Hsuan
c9612c0487
[youtube] Mark errors about rental videos as expected
Closes #12324
2017-03-02 16:59:53 +08:00
Yen Chi Hsuan
af5049f128
[adobepass] Add Charter Spectrum (#11465)
Thanks @tv21 for the fix!
2017-03-02 02:15:51 +08:00
Sergey M․
158af5242e
[utils] Carry long doc string 2017-03-01 23:04:02 +07:00
Sergey M․
40df485f55
[YoutubeDL] Don't sanitize identifiers (closes #12317) 2017-03-01 23:03:36 +07:00
Sergey M․
4b8a984c67
[npo] Add support for audio 2017-03-01 22:21:13 +07:00
Sergey M․
83e8fce628
[npo] Improve extraction and update tests 2017-03-01 22:14:46 +07:00
Sergey M․
aa9cc2ecbf
[npo] Adapt to app.php API (closes #12311) 2017-03-01 05:03:35 +07:00
Sergey M․
1dc24093f8
release 2017.02.28 2017-02-28 23:59:22 +07:00
Sergey M․
43b38424a9
[azmedien:showplaylist] Improve (closes #12160) 2017-02-28 23:37:54 +07:00
Alex Seiler
948519b35d
[azmedien:showplaylist] Add support for all episodes playlists 2017-02-28 23:36:05 +07:00
Sergey M․
87dadd456a
[youtube:playlist] Recognize another playlist pattern (closes #11928, closes #12286) 2017-02-28 23:06:47 +07:00
Yen Chi Hsuan
7c4aa6fd6f
[daisuki] Add subtitles (#4738) 2017-02-28 22:29:01 +08:00
Yen Chi Hsuan
9bd05b5a18 [daisuki] Add new extractor (closes #4738) 2017-02-28 22:19:26 +08:00
Yen Chi Hsuan
0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes()
Used in daisuki.net (#4738)

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
2017-02-28 22:10:31 +08:00
Yen Chi Hsuan
f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
Yen Chi Hsuan
c9619f0a17 [aes] Add aes_cbc_encrypt
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
Yen Chi Hsuan
f4c68ba372
[douyu] Fix extraction and update _TESTS
They've switched from flv to hls

Closes #12301
2017-02-28 21:41:03 +08:00
Sergey M․
ef48a1175d
release 2017.02.27 2017-02-27 23:26:07 +07:00
Sergey M․
18abb74376
[npo] Relax _VALID_URL for zapp.nl 2017-02-27 23:13:51 +07:00
Sergey M․
dbc01fdb6f
[hetklokhuis] Fix IE_NAME 2017-02-27 23:10:29 +07:00
Sergey M․
f264c62334
[npo] Add support for zapp.nl 2017-02-27 23:10:00 +07:00
Sergey M․
0dc5a86a32
[npo] Add support for hetklokhuis.nl (closes #12293) 2017-02-27 22:43:19 +07:00
Sergey M․
0e879f432a
[youtube:channel] Remove duplicate test 2017-02-27 22:22:43 +07:00
Yen Chi Hsuan
892b47ab6c
[scivee] Remove extractor (#9315)
The Wikipedia page is changed from active to down:
https://en.wikipedia.org/w/index.php?title=SciVee&diff=prev&oldid=723161154

Some other interesting bits:

$ nslookup www.scivee.tv
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
www.scivee.tv   canonical name = scivee.rcsb.org.
Name:   scivee.rcsb.org
Address: 132.249.231.211

$ nslookup rcsb.org
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   rcsb.org
Address: 132.249.231.77

Both IPs are from UCSD. I guess it's maintained by a lab and they don't
maintain it anymore.
2017-02-27 21:34:33 +08:00
Alex Seiler
fdeea72611 [cda] Decode URL (fixes #12255) 2017-02-26 22:05:52 +08:00
xbe
7fd4655256 [crunchyroll] Extract uploader name that's not a link
Provide the Crunchyroll extractor with the ability to extract uploader
names that aren't links. Add a test for this new functionality.
This fixes #12267.
2017-02-26 19:08:10 +08:00
Sergey M․
fd5c4aab59
[youtube] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
Sergey M․
8878789f11
[dailymotion] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
Yen Chi Hsuan
a5cf17989b
[MDR] Relax _VALID_URL and playerURL matching and update _TESTS
Ref: #12169
2017-02-26 17:24:54 +08:00
Sergey M․
b3aec47665
[tvigle] Raise GeoRestrictedError 2017-02-25 23:27:45 +07:00
Yen Chi Hsuan
9d0c08a02c
[vevo] Fix videos with the new streams/streamsV3 format (closes #11719) 2017-02-26 00:15:49 +08:00
Sergey M․
e498758b9c
[freshlive] Fix issues and improve (closes #12175) 2017-02-25 22:56:42 +07:00
Ricardo Constantino
5fc8d89361
[freshlive] Add extractor 2017-02-25 22:55:17 +07:00
Pratyush Singh
d374d943f3
[downloader/common] Limit displaying 2 digits after decimal point in sleep interval message 2017-02-25 20:59:04 +07:00
Sergey M․
103f8c8d36
[xhamster] Capture and output videoClosed error (#12263) 2017-02-25 20:38:21 +07:00
Sergey M․
922ab7840b
[etonline] Add extractor (closes #12236) 2017-02-25 20:16:40 +07:00
Sergey M․
831217291a
[compat] Use try except for compat_numeric_types 2017-02-25 19:44:50 +07:00
Yen Chi Hsuan
db182c63fb
[njpwworld] Add new extractor (closes #11561) 2017-02-25 18:44:39 +08:00
Yen Chi Hsuan
eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries
Some websites, like NJPWorld, put different qualities on different
player pages.
2017-02-25 18:40:05 +08:00
Sergey M․
231bcd0b6b
[amcnetworks] Relax _VALID_URL (#12127) 2017-02-25 02:51:53 +07:00
Sergey M․
204efc8509
release 2017.02.24.1 2017-02-24 21:59:39 +07:00
Sergey M․
ad3033037c
[noco] Modernize 2017-02-24 21:51:56 +07:00
Sergey M․
f3bc281239
[noco] Swtich login URL to https (closes #12246) 2017-02-24 21:48:34 +07:00
Sergey M․
441d7a32e5
[thescene] Extract more metadata 2017-02-24 21:22:29 +07:00
Thomas Christlieb
51ed496307 [thescene] Fix extraction (closes #12235) 2017-02-24 22:08:45 +08:00
Remita Amine
68f17a9c2d [tubitv] use geo bypass mechanism 2017-02-24 12:27:56 +01:00
Remita Amine
39e7277ed1 [openload] fix extraction(closes #10408) 2017-02-24 11:21:58 +01:00
Sergey M․
42dcdbe11c
[ivi] Raise GeoRestrictedError 2017-02-24 10:54:39 +07:00
Sergey M․
6b097cff27
release 2017.02.24 2017-02-24 06:09:15 +07:00
Sergey M․
be5df5ee31
Suppress help for all deprecated options and print warning when used 2017-02-24 06:04:27 +07:00
Sergey M․
f2980fddeb
[lynda:course] Add webpage extraction fallback (closes #12238) 2017-02-24 05:01:31 +07:00
Sergey M․
0f57447de7
[postprocessor/ffmpeg] Add mising space (closes #12232) 2017-02-24 04:56:58 +07:00
Remita Amine
8e1409fd80 [go] sign all uplynk urls and use geo bypass only for free videos(closes #12087)(closes #12210) 2017-02-23 18:42:06 +01:00
Sergey M․
fafc2bf5a9
[options] Deprecate --autonumber-size 2017-02-23 22:11:16 +07:00
Sergey M․
b3175982c3
[YoutubeDL] Add more numeric fields for NA substitution in outtmpl 2017-02-23 22:01:57 +07:00
Sergey M․
89db639dfe
[YoutubeDL] Rewrite outtmpl for playlist_index and autonumber for backward compatibility 2017-02-23 22:01:09 +07:00
Sergey M․
d0d9ade486 [YoutubeDL] Add support for string formatting operations in output template 2017-02-23 22:57:53 +08:00
Sergey M․
28572a1a0b [compat] Add compat_numeric_types 2017-02-23 22:57:53 +08:00
Sergey M․
d5fd9a3be3
[skylinewebcams] Add extractor (closes #12221) 2017-02-23 18:45:38 +07:00
Sergey M․
ada77fa544
[instagram] Add support for multi video posts (closes #12226) 2017-02-23 18:02:04 +07:00
Remita Amine
9e03aa75c7 [crunchyroll] extract playlist entries ids 2017-02-23 11:57:18 +01:00
Remita Amine
30eaa3a702 [mgtv] fix extraction 2017-02-23 11:57:05 +01:00
Remita Amine
c59f703610 [sohu] raise GeoRestrictedError 2017-02-23 11:56:55 +01:00
Remita Amine
bc61c80c14 [leeco] raise GeoRestrictedError and use geo bypass mechanism 2017-02-23 11:56:45 +01:00
Sergey M․
345b24538b
release 2017.02.22 2017-02-22 23:50:42 +07:00
Yen Chi Hsuan
b5869560a4
[crunchyroll] Fix descriptions with double quotes (closes #12124) 2017-02-23 00:08:45 +08:00
Sergey M․
527ef85fe9
[dailymotion] Make comment count optional (closes #12209)
Not served anymore
2017-02-22 21:49:30 +07:00
Sergey M․
58ad6995cd
[vidzi] Add test for #12213 2017-02-22 21:29:53 +07:00
Tobias Florek
a86e416088 [vidzi] Add support for vidzi.cc 2017-02-22 22:28:09 +08:00
Sergey M․
71e9577b94
[24video] Add support for 24video.tube (closes #12217) 2017-02-22 21:19:52 +07:00
Remita Amine
abd29a2ced [crackle] use geo bypass mechanism 2017-02-21 19:37:26 +01:00
Remita Amine
31615ac279 [viewster] use geo verifcation headers 2017-02-21 19:36:39 +01:00
Remita Amine
fc320a40d9 Revert "[cbc] use geo bypass mechanism"
This reverts commit 86466a8b6f.
2017-02-21 18:14:55 +01:00
Remita Amine
7345d6d465 [tfo] Improve geo restriction detection and use geo bypass mechanism 2017-02-21 17:52:50 +01:00
Remita Amine
86466a8b6f [cbc] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
Remita Amine
33dc173cdc [telequebec] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
Remita Amine
3444844b04 [limelight] extract PlaylistService errors 2017-02-21 17:52:50 +01:00
Sergey M․
8c6c88c7da
release 2017.02.21 2017-02-21 23:48:24 +07:00
Sergey M․
eea0716cae
[extractor/common] Print origin country for fake IP 2017-02-21 23:14:33 +07:00
Sergey M․
336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries 2017-02-21 23:09:41 +07:00
Sergey M․
dc0a869e5e
[extractor/common] Fix typo 2017-02-21 23:05:31 +07:00
Sergey M․
e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors (#11970) 2017-02-21 23:00:43 +07:00
Remita Amine
e469ab2528 [ninecninemedia] use geo bypass mechanism 2017-02-21 14:38:00 +01:00
Remita Amine
890d44b005 [adobepass] add support for Time Warner Cable(closes #12191) 2017-02-20 19:00:40 +01:00
Sergey M․
6926304472
[spankbang] Make uploader optional (closes #12193) 2017-02-21 00:54:43 +07:00
Sergey M․
3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental 2017-02-20 23:21:15 +07:00
Sergey M․
da42ff0668
[iprima] Improve geo restriction detection and disable geo bypass 2017-02-20 23:17:19 +07:00
Sergey M․
82f662182b
[iprima] Modernize 2017-02-20 23:16:14 +07:00
Sergey M․
2cc7fcd338
[commonmistakes] Disable UnicodeBOM extractor test for python 3.2 2017-02-20 03:06:52 +07:00
Sergey M․
6d4c259765
[svt] PEP 8 2017-02-20 02:25:55 +07:00
Sergey M․
c78dd35491
[nrk] PEP 8 2017-02-20 02:25:39 +07:00
Sergey M․
8ffb8e63fe
[prosiebensat1] Throw ExtractionError on unsupported page type (closes #12180) 2017-02-20 01:00:53 +07:00
Sergey M․
983e9b7746
[nrk] Update _API_HOST and relax _VALID_URL 2017-02-20 00:59:31 +07:00
Sergey M․
c58b7ffef4
[tv4] Bypass geo restriction and improve detection 2017-02-19 06:25:59 +07:00
Sergey M․
f1a78ee4ef
[tv4] Switch to hls3 protocol (closes #12177) 2017-02-19 06:16:00 +07:00
Sergey M․
de64e23c56
[downloader/ism] Honor HTTP headers when downloading fragments 2017-02-19 04:18:36 +07:00
Sergey M․
553f6dbac7
[downloader/dash] Honor HTTP headers when downloading fragments
For example, https://www.oppetarkiv.se/video/1196142/natten-ar-dagens-mor
2017-02-19 04:18:22 +07:00
Sergey M․
0aa10994f4 [options] Move geo restriction related options to separate section 2017-02-19 05:10:08 +08:00
Sergey M․
4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
2017-02-19 05:10:08 +08:00
Sergey M․
0a840f584c Rename bypass geo restriction options 2017-02-19 05:10:08 +08:00
Sergey M․
0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers 2017-02-19 05:10:08 +08:00
Sergey M․
18a0defab0 [utils] Make random_ipv4 return unicode string 2017-02-19 05:10:08 +08:00
Sergey M․
5d3fbf77d9 [viki] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
Sergey M․
80b59020e0 [vgtv] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
Sergey M․
71631862f4 [srgssr] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
Sergey M․
89cc7fe770 [vbox7] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
04d906eae3 [svt] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
8ab8066cf0 [pbs] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
01b1aa9ff4 [ondemandkorea] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
ff4007891f [nrk] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
28200e654b [itv] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
e633f21a96 [go] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
d392005a79 [dramafever] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
Sergey M․
773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
2017-02-19 05:10:08 +08:00
Pierre Mdawar
bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions 2017-02-19 05:10:08 +08:00
Remita Amine
049a0f4d6d [brightcove:legacy] restrict videoPlayer value(closes #12040) 2017-02-18 21:08:40 +01:00
Sergey M․
ac33accd96
[options] Mention quoted string literals for --match-filter 2017-02-18 23:59:26 +07:00
Sergey M․
e84888b432
[tvn24] Improve extraction (closes #11679) 2017-02-18 23:34:09 +07:00
Jakub Wilk
02d9b82a23
[tvn24] Add extractor 2017-02-18 23:33:49 +07:00
Sergey M․
a2e3286676
[thisav] Add support for html5 media (closes #11771) 2017-02-18 20:21:53 +07:00
Sergey M․
f75caf059e
[metacafe] Improve (closes #10371) 2017-02-18 19:58:25 +07:00
Alex Monk
bdabbc220c
[metacafe] Bypass family filter
If you don't send this user=ffilter: false cookie, it will 301 redirect you to a page asking about it, and then the title check will fail.
2017-02-18 19:47:33 +07:00
Remita Amine
70bcc444a9 [viceland] improve info extraction and update test 2017-02-18 09:52:43 +01:00
Sergey M․
28e35f5070
release 2017.02.17 2017-02-17 23:59:56 +07:00
Sergey M․
2c1f442c2b
[options] Add missing spaces 2017-02-17 23:18:26 +07:00
Sergey M․
bad4ccdb5d
[heise] Improve (closes #9725) 2017-02-17 23:09:40 +07:00
Tobias Gruetzmacher
db76c30c6e
[heise] Support videos embedded in any article. 2017-02-17 22:55:53 +07:00
Déstin Reed
c2bde5d081
[ellentv] Improve 2017-02-17 22:45:51 +07:00
Sergey M․
90fad0e74c
[openload] Fix extraction (closes #12002) 2017-02-17 22:31:16 +07:00
Vijay Singh
d94badc755
[openload] Semifix extraction (closes #10408)
just updated the code. i don't do much python still i tried to convert my code. lemme know if there is any prob with it
2017-02-17 22:30:05 +07:00
Yen Chi Hsuan
fef51645d6
[theplatform] Recognize URLs with whitespaces (closes #12044) 2017-02-17 23:13:51 +08:00
Sergey M․
4cead6a614
[einthusan] Relax _VALID_URL (closes #12141, closes #12159) 2017-02-17 22:02:01 +07:00
Yen Chi Hsuan
a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes #12030) 2017-02-16 23:44:03 +08:00
Sergey M․
b898f0a173
[elpais] Fix typo and improve extraction (closes #12139) 2017-02-16 04:57:42 +07:00