Commit Graph

420 Commits

Author SHA1 Message Date
Kavin dfd7580374
Use right join on channels. 2022-11-17 01:09:12 +00:00
Kavin 0fd90ab1c9
Implement better playlist querying. 2022-11-17 00:03:41 +00:00
Kavin 005bbfd1e5
Implement channel data federation. 2022-11-16 23:04:45 +00:00
Kavin 92cee87822
Merge pull request #439 from TeamPiped/stream-timeout
Add timeout for streams response.
2022-11-16 19:45:28 +00:00
Kavin f757c439af
Add timeout for streams response. 2022-11-16 19:44:53 +00:00
Kavin 32f30e42f4
Fix NPE check for channel uploader. 2022-11-16 19:32:39 +00:00
Kavin 98628aff98
Implement support for audio track info. 2022-11-15 19:24:52 +00:00
Kavin 6ad27aea2a
Update NPE. 2022-11-15 19:21:36 +00:00
Kavin f85b9d095d
Attempt to warm classes. 2022-11-15 09:54:13 +00:00
Kavin fd6a6b3083
Start matrix sync thread asynchronously. 2022-11-15 01:41:15 +00:00
Kavin 79621ac562
Federate new video content. 2022-11-13 01:00:26 +00:00
Kavin 9d8117239f
Fix incorrect field. 2022-11-13 00:48:12 +00:00
Kavin cd6e63130b
Implement federation of video data. 2022-11-13 00:27:13 +00:00
Kavin ea67f60689
Update piped.kavin.rocks to piped.video 2022-11-11 19:52:51 +00:00
Kavin 22dcf7a88e
When parameter is not provided, throw an ErrorResponse. 2022-11-04 19:42:05 +00:00
Kavin d5585738da
Throw 400 response for invalid parameters messages from extractor. 2022-10-31 12:00:37 +00:00
Kavin afd213b4c8
Handle potential bad responses for RYD. 2022-10-31 11:49:22 +00:00
Kavin fe87d2cf4d
Don't log error response. 2022-10-31 11:09:57 +00:00
Kavin a1170677b4
Add more parameter validation. 2022-10-31 10:44:19 +00:00
Kavin 322769b0e4
Use streaming api directly from query. 2022-10-31 10:44:00 +00:00
Kavin d4d9c71279
Remove org.json and fix occasional json exception. 2022-10-31 10:43:34 +00:00
Kavin 405e0d3bb3
Add some sentry extra data.
And fix potential NPE in RydHelper.
2022-10-31 01:15:46 +00:00
Kavin f4cb18d102
Add videoId scope and fix sentry transaction trace. 2022-10-30 23:53:41 +00:00
Kavin 08b61a1d20
Ignore ErrorResponse and implement tracing. 2022-10-30 22:26:20 +00:00
Kavin 4a8fb80411
Implement status code error errors. 2022-10-30 21:41:02 +00:00
Kavin 5e531e19d3
Add better validation for some parameters. 2022-10-30 20:29:57 +00:00
Kavin 8c55b85fc2
Allow using sentry to log errors. 2022-10-30 19:56:04 +00:00
Kavin 3b6f983fdc
Change max length to 200 characters. 2022-10-30 14:11:14 +00:00
Kavin 1c73f852b9
Implement support for uploaderUrl and uploaderVerified for playlists. 2022-10-29 23:30:47 +01:00
Kavin 2afecb3f39
Refactor server code into multiple classes. 2022-10-29 22:00:55 +01:00
Kavin 2c79ddb299
Extract item collectors to a new class. 2022-10-29 21:29:09 +01:00
Kavin 82ac80df97
Add channel name updates when loading. 2022-10-29 21:16:52 +01:00
Kavin aacafb8ab4
Fix channel tab nextpage. 2022-10-27 12:14:53 +01:00
Kavin 00a6d7da93
Implement support for channel tabs. 2022-10-27 11:59:10 +01:00
Kavin 2b8039ff87
Add isShort field to StreamInfo. 2022-10-27 10:24:36 +01:00
Kavin b15f90bbd3
Add support to configure sponsorblock servers. 2022-10-23 19:17:04 +01:00
Kavin 4d745b8a84
Add support for reply count and bump deps. 2022-10-17 14:36:43 +01:00
Kavin c10ff6e3c5
Add video count to playlists response. 2022-10-11 01:34:58 +01:00
Kavin 3c00bfae89
Query all videos at once and batch update. 2022-10-06 11:05:42 +01:00
Ellie 4e155fc70a
Get PubSub Hub url from constants 2022-10-04 07:46:19 +00:00
Ellie a10d2f6edf
Add PUBSUB_HUB_URL constant 2022-10-04 07:44:53 +00:00
Kavin 202df96442
Merge pull request #374 from Bnyro/config
add config for the frontend
2022-10-02 14:53:51 +01:00
Kavin 51185f2558
Some small changes. 2022-09-25 22:33:29 +01:00
Kavin dcc4b6e807
Use and query for where clause. 2022-09-25 22:19:37 +01:00
Kavin 9edfd133f9
Fix unauthenticated subscriptions where clause. 2022-09-25 21:49:12 +01:00
Bnyro 52370036e3 add config for the frontend 2022-09-18 15:15:58 +02:00
Kavin 383c017f5c
Optimize pubsub subscription query. 2022-09-17 20:29:26 +05:30
Kavin ab0540392f
Let NewPipeExtractor use same executor. 2022-09-11 01:35:10 +05:30
Kavin 13710d24d8
Update NPE fork and parallelize StreamExtraction requests. 2022-09-07 12:59:11 +05:30
Bnyro 178dbefb52 route for renaming playlists 2022-08-27 19:09:21 +02:00
Kavin adbd2b347f
Small Java 17 improvements. (#337) 2022-08-18 08:43:40 +05:30
Kavin 42d8951863
Remove debug code. 2022-08-12 20:04:01 +05:30
Kavin 48c780ed90
Use AsyncLoadingCache for Downloader requests. 2022-08-12 19:50:53 +05:30
Kavin eae003e422
Fix authenticated RSS feeds. 2022-08-06 15:25:41 +05:30
Kavin 81fa92361d
Fix and optimize pubsub timer query. 2022-08-05 00:03:47 +05:30
Kavin 30a7b43c5e
Fix cloning of playlist with duplicate videos. 2022-08-03 21:13:21 +05:30
Kavin d93f2f360b
Optimize feed query by reducing join on users. 2022-08-03 21:01:49 +05:30
Kavin 48efbe66ca
Unauthenticated channel info endpoint. 2022-08-01 17:39:40 +05:30
Kavin 9a45e33fdb
Fix bugs and implement more of unauthenticated subs. 2022-07-31 23:17:42 +05:30
Kavin 02067518b0
Implement part of unauthenticated subscription api. 2022-07-31 20:50:01 +05:30
Kavin 6c6c8c021f
Add option to disable LBRY. 2022-07-22 09:22:48 +05:30
Kavin a36279cf79
Allow increasing the RYD proxy timeout. 2022-07-21 09:40:13 +05:30
Kavin 12c4890587
Implement logging out of session. 2022-07-20 20:16:48 +05:30
Kavin 4283c3fee0
Make the api url redirect to the instance's frontend. 2022-07-19 22:07:07 +05:30
Kavin c0f51c6fb6
Allow disabling server on a node. 2022-07-19 14:15:42 +05:30
Kavin 0e8b17698d
Fix imports and make database schema more strict. 2022-07-18 21:22:53 +05:30
Kavin f37682cf4e Revert "Revert distinct from criteria. (#308)"
This reverts commit bcb0d32daa.
2022-07-17 19:44:56 +05:30
Kavin c7a0535039
Attempt to log import errors. (#312) 2022-07-08 10:41:49 +01:00
Kavin c85cbf6034
Use a timer task instead for deleting playlist videos not in any playlists. (#311) 2022-07-08 00:06:39 +01:00
Kavin 78b52d70f5
Set statement timeout to 20s for feed. (#309) 2022-07-07 20:05:34 +01:00
Kavin bcb0d32daa
Revert distinct from criteria. (#308) 2022-07-07 18:43:47 +01:00
Kavin d29d9c415d
Attempt to optimize session handling. (#307) 2022-07-05 14:18:33 +01:00
Kavin e6c0f7c0f4
Cleanup transaction code. (#306) 2022-07-04 18:44:16 +01:00
Kavin 7c087d82b3
Fetch user table as left join. (#304) 2022-07-04 08:37:52 +01:00
Kavin c812ace4f6
Fix errors being logged due to invalid requests. (#303) 2022-07-04 07:09:29 +01:00
Kavin 606cbf5ca6
Use Criteria API for deleting old videos. (#302) 2022-07-04 07:09:20 +01:00
Kavin 17b9a974de
Attempt to use stateless sessions in read-only operations. (#301) 2022-07-04 06:42:01 +01:00
Kavin cdd8a962d7
Avoid duplicate video errors. (#300) 2022-07-04 06:02:06 +01:00
Kavin c8628dcf05
Simplify session handling for new videos. (#299) 2022-07-02 21:30:19 +01:00
Kavin 0158744506
Hold pubsub session for lesser time. (#298) 2022-07-02 18:13:58 +01:00
Kavin 3a009407ad
Run server on new thread. (#297) 2022-07-02 13:32:20 +01:00
Kavin 943e4a3de8
Add some parameter checking to search and feed routes. (#294) 2022-06-28 18:13:29 +01:00
Kavin 374d067a95
Implement RYD-Proxy API implementation. (#293) 2022-06-27 17:11:22 +01:00
Kavin b7a91036f9
Cleanup Transactions. (#275)
- use hibernate 6 methods
- Don't use session.getTransaction()
- regex to validate channelId for saving channels
2022-06-26 10:03:02 +01:00
Kavin e8e055a676
Case insensitive sort for subscriptions. (#292) 2022-06-26 03:44:07 +01:00
Kavin f5e881e0e4
Add length check for username. 2022-06-26 03:01:19 +01:00
Kavin c558572953
Import mapper statically. 2022-06-26 03:00:45 +01:00
Kavin e085d4c8a3
Allow having a different url for pubsub. (#290)
This would be useful for onion instances.
2022-06-25 10:22:45 +01:00
Kavin 160e6933d3
Allow disabling pubsub timer to lower cpu usage. (#289) 2022-06-25 10:19:06 +01:00
Kavin 18211a3eed
Add route to import playlist from YouTube. (#282) 2022-06-18 19:10:26 +01:00
Kavin c23805288f
Update to Hibernate 6. 2022-06-17 17:05:59 +01:00
Kavin 87917f0da2
Use parallel streams for some operations. (#264) 2022-05-12 11:54:28 +01:00
Kavin ce70bd037b
Limit RSS feed size to 100 entries (#263) 2022-05-10 09:40:26 +01:00
the_4n0nym0u53 d3d1ee420b
Feature: Implement account deletion (#248)
* Add backend code for user deletion

* Add endpoint and delete user test

* Delete all user content from db

* Delete from playlist_videos table on user deletion

* Avoid raw SQL in unsubscribeResponse()

* Fix unsubscribeResponse()

* Don't delete PlaylistVideos from other users' playlists

* Fix pruneUnusedPlaylistVideos()

* Remove unused commented-out code

* Fix oopsie

* Proper type declaration due to false error-reporting by VSCode

* Use delete query for better performance

* Cleanup and add OneToMany relationship.

* Revert unsubscribe logic.

Co-authored-by: Kavin <20838718+FireMasterK@users.noreply.github.com>
2022-05-05 19:51:51 +01:00
Kavin a5296d6e5a
Add support for a different address for proxying images. (#252) 2022-04-25 07:42:59 +01:00
Kavin 11b53e8c00
Update NewPipeExtractor. (#249) 2022-04-24 18:01:00 +01:00
Kavin 9c1c165fb2
Significantly speed up import process. (#242) 2022-04-11 00:02:25 +01:00
Kavin ce50b7523f
Handle lbryId timeout gracefully. (#241) 2022-04-09 23:22:16 +01:00
Kavin 561a4f44e1
Add index for playlist owner. (#240) 2022-04-09 22:04:16 +01:00
Kavin cef4b1dcb7
Handle JSON data better. 2022-04-07 18:40:34 +01:00
Kavin dc0e4c1bbb
Cache based on playlistId length. 2022-04-07 02:43:01 +01:00
Kavin ad89de49a0
Change table declaration so unique unique constraints aren't generated. 2022-04-06 17:34:20 +01:00
Kavin 46d9dd1a2e
Add videos size to custom playlist response. 2022-04-06 16:29:30 +01:00
Kavin 76c5fe9273
Fix rss for empty playlist. 2022-04-06 16:27:04 +01:00
Kavin 9e320c914e
Set playlist thumbnail when adding first thumbnail. 2022-04-06 16:15:44 +01:00
Kavin f7a5f5051a
Add route to delete playlist. 2022-04-06 16:11:28 +01:00
Kavin 1135e20c3d
Add support for Piped playlists in RSS. 2022-04-06 15:05:18 +01:00
Kavin 3d6aefdd0e
Add route to remove from playlist. 2022-04-06 14:31:16 +01:00
Kavin 00e32dc992
Add route to add video to playlist. 2022-04-06 13:07:50 +01:00
Kavin 797a621b51
Add tests for playlist. 2022-04-06 04:23:31 +01:00
Kavin b049848e79
Add playlist creation route. 2022-04-06 03:46:40 +01:00
FireMaskterK 056d61f2d2
Add playlist object. 2022-04-05 21:59:56 +01:00
Kavin 11845f26fc
Fix ClassCastException. (#235) 2022-03-30 17:47:05 +01:00
Kavin 0cea0a50f5
Change future code and add uploaderSubscriberCount. (#234) 2022-03-30 17:23:56 +01:00
Kavin 5b6dc2b097
Add check for database session in healthcheck. (#232)
* Add check for database session in healthcheck.

* Add curl test for healthcheck.
2022-03-29 09:49:42 +01:00
Kavin b1e875bcf5
Fix subscription imports and improve session handling. (#228) 2022-03-19 18:57:00 +00:00
Kavin a4b6e1f804
Check if username and password is empty during registration. (#225) 2022-03-16 15:08:18 +00:00
Kavin 43d6dc93c0
Use try with for sessions (#222) 2022-03-16 03:21:24 +00:00
Kavin 6e88cf3050
Log and clear throttling cache every hour. (#220) 2022-03-14 14:54:47 +00:00
Kavin 8fb1871dce
Reduce response cache size. (#214) 2022-03-10 10:14:05 +00:00
Kavin 02ca78ee7a
Manually close okhttp responses. (#210)
Ensure response is closed properly.
2022-03-03 12:27:43 +00:00
Kavin cec135933e
Update NPE and remove the caching of CommentsInfo instances. (#209) 2022-03-03 12:03:45 +00:00
Kavin 72666ecf83
Change no-cache to no-store. (#208) 2022-03-02 20:38:22 +00:00
Kavin 3d8d592768
Fix for bug where subscriptions can't be fetched on fresh accounts. (#205) 2022-03-01 21:02:35 +00:00
Kavin 597e861159
Replace in to equals. (#203) 2022-02-28 22:46:15 +00:00
Kavin b0b651f44d
Don't perform authentication checks in subscribed route. (#201)
If the sessionId is invalid, false should be returned for subscribed, we make it the client's responsibility to ensure the sessionId is valid.
2022-02-24 19:06:38 +00:00
Kavin 014c9533c8
Optimize subscriptions query, by using Criteria API. (#200) 2022-02-24 12:32:39 +00:00
Kavin 78082742ad
Replace left joins with inner joins (#199)
* Replace left joins with inner joins.

* Temporarily disable YugabyteDB CI
2022-02-24 12:27:23 +00:00
Kavin 94f5fa37ac
Add route to display version information. (#195)
* Add version information route.

* Use build action for tests.

* Add version test to script.

* Add testing directory to dockerignore.

* Don't unnecessarily set permission for gradle wrapper script.
2022-02-22 09:16:45 +00:00
Kavin a03ae434be
Replace native query with criteria api for PubSub subscriptions (#191) 2022-02-15 11:55:08 +00:00
Kavin 498849ec6b
Replace native query with criteria api for feeds (#190) 2022-02-15 11:37:21 +00:00
Kavin 166e14cf71
Add dedicated route for health check (#189) 2022-02-14 02:38:02 +00:00
FireMasterK f28d40dffe
Add support for extracting videoId from clipId. 2022-02-10 14:57:08 +00:00
Kavin 7eeff6eec4
Refractor codebase and improve video data update handling. (#185) 2022-02-09 03:06:52 +00:00
Kavin ec3b69b7a7
Add shortDescription parameter to StreamItem. (#184) 2022-02-08 14:43:45 +00:00
Kavin cf32a60ed1
Add uploaded field to StreamItem. (#183) 2022-02-06 15:33:46 +00:00
Kavin b54248b47f
Remove casts by specifying type. (#182) 2022-02-03 11:57:07 +00:00
Kavin 8ba39000b1
Simplify comparators (#181) 2022-02-03 11:29:59 +00:00
Kavin 3748afdd72
Fix for empty lbry streams. (#180) 2022-02-02 21:23:37 +00:00
Kavin c07cf5fd1f
Refactor Utility methods and migrate to OkHttp (#179) 2022-02-02 21:05:22 +00:00
Kavin 9c926d3183
Force use HTTP 1.1 temporarily. (#177) 2022-02-02 08:05:43 +00:00
Kavin 8880a884c3
Add feed retention configuration option. (#174) 2022-01-31 04:17:16 +00:00
Kavin 9aa16b3299
Improvements to caffeine caching. (#171) 2022-01-26 04:00:56 +00:00
Kavin 322610dbbb
Rewrite chapter image URLs. (#167) 2022-01-13 04:22:09 +00:00
Kavin c22e8d878b
Add proper mime for rss feeds, and preconnect header for proxy (#157) 2021-12-27 08:54:48 +00:00
Kavin 2923e1ec76
Fix feed validation errors (#155) 2021-12-25 16:46:39 +00:00
Kavin 49f439e761
Add null-check for SponsorBlock endpoint (#154) 2021-12-24 14:17:02 +00:00
Kavin 4c317a6158
Change rss entry from id to link (#150)
Uri -> Link
2021-12-20 14:36:20 +00:00
Kavin b0872ce414
Update default frontend uri (#149) 2021-12-20 14:21:45 +00:00
FireMaskterK d34e22790f
Add hibernate properties from env variables. 2021-11-23 01:50:06 +00:00
FireMaskterK 749cf48f4b
Read properties from environment if available.
Closes https://github.com/TeamPiped/Piped/issues/638
2021-11-23 00:21:18 +00:00
FireMaskterK 6ec517e0ef
Add index and improve pubsub db query. 2021-11-18 23:32:19 +00:00
FireMaskterK 376e068264
Add DownloaderImpl caching. 2021-11-17 09:54:56 +00:00
FireMaskterK e1e5382959
Add registered users badge. 2021-11-11 11:35:50 +00:00
FireMaskterK 1c8e8b190e
Fix LBRY timeouts. 2021-11-09 08:06:13 +00:00
FireMaskterK 3de3b55d3c
Revert "Fix LBRY timeouts."
This reverts commit a7b02e9566.
2021-11-09 08:03:48 +00:00
FireMaskterK a7b02e9566
Fix LBRY timeouts. 2021-11-09 08:03:21 +00:00
FireMaskterK ced7b0ee9e
Add configuration property to disable registration. 2021-11-09 06:27:29 +00:00
FireMaskterK 2a87b7946d
Add index for session_id. 2021-11-08 23:46:56 +00:00
FireMaskterK 6012c1cd82
Attempt to fix index performance issues. 2021-11-08 21:28:08 +00:00
FireMaskterK 0d3b167e8a
Add index for video uploaded. 2021-11-08 20:53:48 +00:00
FireMaskterK e629121c25
Log path only when exception is logged. 2021-11-08 20:35:02 +00:00
FireMaskterK a7885f1ca1
Update privacy.resistFingerprinting UA. 2021-11-08 15:20:41 +00:00
FireMaskterK 98127b8c0e
Implement chapter segment support.
Closes https://github.com/TeamPiped/Piped/issues/600
2021-11-04 19:21:26 +00:00
FireMaskterK 5feec94220
Fix url rewrite issues
Closes https://github.com/TeamPiped/Piped/issues/598
2021-11-04 17:45:39 +00:00
FireMaskterK 759a6445f0
Fix for Comment repliesPage being "null" when not present. 2021-11-01 19:58:44 +00:00
FireMaskterK 86154a1a29
Allow Avatar URL to change in channel. 2021-10-31 17:48:34 +00:00
FireMaskterK ceced61e60
Fix for Chromium depreciation warning. 2021-10-26 18:22:31 +01:00
FireMaskterK f731308dcc
Implement OpenSearch api. 2021-10-25 23:55:29 +01:00
FireMaskterK b63786a15e
Cache videos for a maximum of 30 seconds on the client side. 2021-10-23 23:13:54 +01:00
FireMaskterK 1c1dd05455
Add timeout for LBRY calls. 2021-10-12 11:44:21 +01:00
FireMaskterK 022a0af007
Include search corrections in API. 2021-10-10 00:24:29 +01:00
FireMaskterK 2c7baa0710
Don't fail hard when LBRY throws errors. 2021-10-06 16:37:47 +01:00
FireMaskterK a1a3e717b2 Add option to disable HIBP password check.
Usage of this option is discouraged for password security reason.
2021-10-04 19:57:56 +01:00
FireMasterK d03c677a7f
Fix for possible NPE. 2021-10-03 09:55:13 +01:00
FireMasterK 08aeefa099
Add support for legacy bcrypt hashes from Invidious 2021-10-03 09:02:43 +01:00
FireMasterK cc3976b454
Improve livestream detection. 2021-10-01 18:53:54 +01:00
FireMasterK 29fb9b02a2
Add support for extracting replies and update NPE. 2021-09-22 15:18:24 +01:00
FireMasterK 84bd3aa164
Improve feed query performance. 2021-09-21 22:35:12 +01:00
FireMasterK aae627a423
Remove debug constant. 2021-09-21 22:34:25 +01:00
FireMasterK d6a2d92eeb
Refractor repeated code. 2021-09-10 00:55:16 +05:30
FireMasterK baa84e9403
Refractor and improve log handling. 2021-09-10 00:19:46 +05:30
FireMasterK e38d989a5e
Add FRONTEND_URL configuration property. 2021-09-09 13:35:19 +05:30
FireMasterK cdcae82c4d
Check if passwords are compromised in HIBP. 2021-09-06 00:49:54 +05:30
FireMasterK 721f197bd8
Use substringAfter to cleanup code and fix YouTube music URLs. 2021-09-06 00:14:24 +05:30
FireMasterK c7338f85f3
Fix NPE in YouTube Music search. 2021-09-05 03:31:48 +05:30
FireMasterK 0fe215ea67
Unify video/StreamInfoItem in Search Results. 2021-09-04 01:02:51 +05:30
FireMasterK 1524085577
Rename field to fix consistency issues. 2021-09-03 23:05:58 +05:30
FireMasterK b2c9ffa789
Update NewPipeExtractor and extract more info. 2021-09-03 22:01:26 +05:30
FireMasterK f4b0884978
Commit missing import and fix build. 2021-08-28 13:36:09 +05:30
FireMasterK 3e99d1d806
Add support for overriding subscriptions when importing. 2021-08-28 13:27:56 +05:30
FireMasterK a15ad70817
Remove unnecessary methods. 2021-08-21 01:55:45 +05:30
FireMasterK 15f7eef82c
Cleanup PubSub query. 2021-08-21 01:52:39 +05:30
FireMasterK f9f4dd8e74
Add verified info in more places. 2021-08-18 01:37:49 +05:30
FireMasterK 90d04f9e4f
Implement uploaderVerified for feed response. 2021-08-17 22:59:15 +05:30
FireMasterK e4f071b72a
Add Dash URL to streams response. 2021-08-16 01:21:48 +05:30
3nprob aed1da4aae
Make listening port configurable (#95)
Co-authored-by: 3nprob <3nprob@3nprob>
2021-08-14 16:12:55 +05:30
ChunkyProgrammer b28356d0ed
get channel subscriber count (#94)
update
2021-08-14 09:58:52 +05:30
3nprob ce5881bad1
Add support for outbound http proxy (#93)
* Add support for outbound http proxy

* Simplify logic and fix typos.

Co-authored-by: 3nprob <3nprob@3nprob>
Co-authored-by: FireMasterK <20838718+FireMasterK@users.noreply.github.com>
2021-08-14 00:08:15 +05:30
FireMasterK a0c5898f4d
Fix CommentorURL NPE and wrong substring. 2021-08-12 22:57:44 +05:30