Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor test_stream to a separate function #434

Open
wants to merge 237 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
237 commits
Select commit Hold shift + click to select a range
25d254d
fembed additional domain (#292)
Gujal00 Mar 12, 2020
37e1814
Fix streamz (#295)
Mar 18, 2020
b2016dd
Create streamwire.py (#297)
lukkie112 Mar 18, 2020
b8fbdaf
Move head request in the conditional (#296)
Mar 18, 2020
4a4170a
Update waaw (#294)
mbebe Mar 18, 2020
24105a3
Fix mystream.to (#298)
Gujal00 Mar 19, 2020
63c112f
Update mystream.py
Gujal00 Mar 19, 2020
accdd0f
New Resolver pkspeed (#299)
Gujal00 Mar 20, 2020
efaf864
Delete megamp4.py
Gujal00 Mar 20, 2020
f4c6381
Delete dailynotion.py
Gujal00 Mar 20, 2020
517c22e
Fix Jetload resolver (#302)
Gujal00 Mar 24, 2020
e88bad6
Entervideo resolver fix (#303)
Gujal00 Mar 24, 2020
60b6683
Fix Dailymotion resolver
Gujal00 Mar 24, 2020
59973ab
Add additional embed url pattern
Gujal00 Mar 24, 2020
5b99bce
Merge pull request #305 from Gujal00:update-streamwire
Gujal00 Mar 24, 2020
be85dc3
fix Uptobox resolver (#306)
Gujal00 Mar 24, 2020
0f4e04a
Update changelog.txt
Gujal00 Mar 24, 2020
948faf9
Bump to 5.0.39
Gujal00 Mar 24, 2020
be657d6
Alternate code for PR307
Gujal00 Mar 25, 2020
3db0cf1
New resolver Vidfast (#309)
Gujal00 Mar 25, 2020
cb9372c
Delete putvid.py (#311)
azzy9 Mar 26, 2020
5836cda
Delete fileholic.py (#310)
azzy9 Mar 26, 2020
8006964
Fix Videa resolver
Gujal00 Mar 27, 2020
e8eaa8c
Saruch resolver
Gujal00 Mar 27, 2020
859dcf8
Fix Gamovideo
Gujal00 Mar 27, 2020
d23fdf9
New resolver Sibnet.ru
Gujal00 Mar 27, 2020
56afcf6
Fix CDA
Gujal00 Mar 28, 2020
b71d421
CDA additional domain
Gujal00 Mar 28, 2020
f33c8ae
Fix CDA
Gujal00 Mar 28, 2020
ce8ebb4
Fix Uptobox resolver
Gujal00 Mar 29, 2020
d5b6799
Merge branch 'fix-uptobox'
Gujal00 Mar 29, 2020
1875213
Bump version 5.0.40
Gujal00 Mar 29, 2020
15f735c
Fix Aliez resolver
Gujal00 Mar 30, 2020
d1d354c
update Chromecast resolver
Gujal00 Mar 30, 2020
747ed83
Clicknupload update domain
Gujal00 Mar 30, 2020
4e2de14
update clipwatching resolver
Gujal00 Mar 30, 2020
1f3474a
Update CloudB resolver
Gujal00 Mar 30, 2020
7ba107a
Fix cloud.mail.ru resolver
Gujal00 Mar 30, 2020
7838e2c
Cleanup datemule resolver
Gujal00 Mar 30, 2020
8d8c518
Update README.md
Gujal00 Mar 30, 2020
8cd1a1e
Delete watchshare.py (#315)
azzy9 Mar 31, 2020
892b17c
Merge branch 'master' of github.com:jsergio123/script.module.resolveurl
Gujal00 Mar 31, 2020
5f9a227
update Readme
Gujal00 Mar 31, 2020
5d91253
remove defunct hosts
Gujal00 Mar 31, 2020
c944fac
New resolver Primevideos
Gujal00 Mar 31, 2020
9630f98
update waaw to check for failure
Gujal00 Apr 1, 2020
e16746e
Fix GoUnlimited resolver
Gujal00 Apr 1, 2020
e1bde1e
Fix mystream resolver
Gujal00 Apr 2, 2020
6878f21
Bump Version
Gujal00 Apr 2, 2020
1052a99
Fix upstream
Gujal00 Apr 2, 2020
e18ed0d
fix Upstream resolver
Gujal00 Apr 2, 2020
cb47297
New Resolver Vidcloud9
Gujal00 Apr 2, 2020
b1be31d
Fix VKPrime resolver
Gujal00 Apr 3, 2020
e5460f9
Alldebrid API update to v4
Gujal00 Apr 5, 2020
0a6e730
Update alldebrid.py
Gujal00 Apr 5, 2020
c507895
Fix CDA Resolver
Gujal00 Apr 9, 2020
12bb165
Squashed commit of the following:
Gujal00 Apr 10, 2020
4f0fc40
Delete settings.xml.org
Gujal00 Apr 10, 2020
917352a
Fix Premiumize pick final file
Gujal00 Apr 10, 2020
8656b89
Add anavids & liivideo
Gujal00 Apr 10, 2020
f4aa25e
Alldebrid api V4 updates
Gujal00 Apr 11, 2020
060e320
Bump version
Gujal00 Apr 11, 2020
c60a7a7
Fix CDA resolver
Gujal00 Apr 12, 2020
c5ab0f4
Add DoodStream resolver
Gujal00 Apr 13, 2020
4ae2dda
Add torrent file support
Gujal00 Apr 14, 2020
5450e0f
Add wstream.video
Gujal00 Apr 16, 2020
18263e1
Debrid-link improve torrent transfer check
Gujal00 Apr 19, 2020
886c549
cleanup comments
Gujal00 Apr 19, 2020
20ba8f2
Bump version
Gujal00 Apr 25, 2020
35932cd
Add Videomega and youdbox resolvers
Gujal00 Apr 27, 2020
1e72fc8
Bump version
Gujal00 Apr 27, 2020
6896edb
Fix clipwatching
Gujal00 Apr 28, 2020
219fb3a
Add Vidmoly resolver
Gujal00 Apr 30, 2020
31987c6
Bump version
Gujal00 May 1, 2020
67a5b08
Add EasyLoad.io resolver
Gujal00 May 1, 2020
27eaa98
hxload domain change
Gujal00 May 11, 2020
b5574de
update user agents
Gujal00 May 12, 2020
c46b001
bump version
Gujal00 May 12, 2020
507b575
MyStream additional domain
Gujal00 May 15, 2020
896fa3a
New resolver abcvideo
Gujal00 May 16, 2020
2b10915
vidmoly additional match pattern
Gujal00 May 16, 2020
178a348
Add viduplayer resolver
Gujal00 May 17, 2020
f585d67
bump version
Gujal00 May 17, 2020
2770e68
wstream addtional embed pattern
Gujal00 May 20, 2020
7569ee0
credits
Gujal00 May 21, 2020
dbfd6c8
Vivo addito0inal embed pattern
Gujal00 May 21, 2020
ca3154b
Fix Anavid Resolver
Gujal00 Jun 1, 2020
936d4e7
Bump version
Gujal00 Jun 1, 2020
9e2f8e2
Merge PY2/PY3 compliant code
Gujal00 Jun 7, 2020
946caee
Bump version
Gujal00 Jun 7, 2020
2ac644c
change init logging to debug level
Gujal00 Jun 11, 2020
b21c3e8
Add daxab resolver
Gujal00 Jun 14, 2020
e87a38e
helper function
Gujal00 Jun 15, 2020
0ae3452
fembed additional domain
Gujal00 Jun 16, 2020
0eebd29
wstream additional pattern
Gujal00 Jun 17, 2020
df56b56
reset auth on refresh token failure
Gujal00 Jun 18, 2020
a275f88
Bump version
Gujal00 Jun 21, 2020
effabe8
Additional pattern for wstream resolver
Gujal00 Jun 26, 2020
72a34c5
Fix settings.xml writing on non english systems
Gujal00 Jun 29, 2020
6fb4768
Add Vudeo resolver
Gujal00 Jul 5, 2020
5e74322
Bump Version
Gujal00 Jul 6, 2020
69b099f
Add new resolvers
Gujal00 Jul 9, 2020
c10fbd2
Add megogo resolver
Gujal00 Jul 10, 2020
4a79538
Possible fix for settings encoding in XBox
Gujal00 Jul 10, 2020
c3b44f5
bump version
Gujal00 Jul 10, 2020
83941bf
handle non english characters in settings.xml
Gujal00 Jul 16, 2020
63b062d
additional nxload url pattern
Gujal00 Jul 19, 2020
dd1a089
fix mixdrop
Gujal00 Jul 24, 2020
d112369
add voe.sx resolver
Gujal00 Jul 24, 2020
391d540
bump version
Gujal00 Jul 25, 2020
513ca9e
Add streamtape resolver
Gujal00 Jul 26, 2020
002364a
Add cloud9 resolver
Gujal00 Aug 2, 2020
f432f8e
NXLoad additional pattern
Gujal00 Aug 2, 2020
2178879
Add Aparat resolver
Gujal00 Aug 7, 2020
09295d8
Additional pattern for Playtube
Gujal00 Aug 9, 2020
affeff4
bump version
Gujal00 Aug 9, 2020
2f23298
Add Doodstream resolver
Gujal00 Aug 12, 2020
783b95f
Add Vidup resolver
Gujal00 Aug 12, 2020
c4198dc
Fix Vidstreaming.IO
Gujal00 Aug 15, 2020
2ec3973
Add Vidbom resolver
Gujal00 Aug 16, 2020
f14dc8c
remove debug
Gujal00 Aug 16, 2020
71e1810
bump version
Gujal00 Aug 16, 2020
399c3f4
Cloudb resolver additoinal embed method
Gujal00 Aug 19, 2020
0bc0572
FEmbed additional domain
Gujal00 Aug 21, 2020
40d43f8
fix (#362)
Aug 22, 2020
f5daf40
anistream flake8 linting
Gujal00 Aug 22, 2020
28e7e7d
Fix Aparat resolver
Gujal00 Aug 22, 2020
658e260
add domain vidoza.co (#366)
Aug 22, 2020
ccf1b54
Addon.xml el_GR tweaks and the addition of disclaimer within it. (#367)
Aug 22, 2020
aa0f0ed
linting
Gujal00 Aug 29, 2020
00cee01
Add underscore to url pattern (#368)
LowerDecks Sep 2, 2020
eb41f4d
GoUnlimited, remove unnecessary code
Gujal00 Sep 2, 2020
68a016a
Bump Version
Gujal00 Sep 2, 2020
80bee18
Premiumize option to keep finished torrents
Gujal00 Sep 7, 2020
c689c70
Premiumize remove unneeded delete_transfer
Gujal00 Sep 7, 2020
a3622b9
Bump Version
Gujal00 Sep 7, 2020
07b2249
Fix handling embed links without referer
Gujal00 Sep 13, 2020
9c6485a
Linksnappy fixes (plugin version 3)
Sep 15, 2020
08ccb72
Bump version
Gujal00 Sep 18, 2020
39f0edd
kodi 19 fix: deprecated LOGNOTICE (#374)
host505 Sep 23, 2020
0920150
Linksnappy, version 4 (#375)
Sep 24, 2020
d72c0cd
Bump version
Gujal00 Sep 25, 2020
adec3b9
Fix Gounlimited
Gujal00 Sep 25, 2020
9ea0230
xbmc.translatePath being deprecated in Matrix
Gujal00 Sep 27, 2020
26348a1
fix upstream (#378)
Sep 29, 2020
630498a
Add newtube plugin (#379)
Oct 6, 2020
7ab8b6b
Fix Streamtape
Gujal00 Oct 15, 2020
e357f63
Fembed additional domain
Gujal00 Oct 18, 2020
1e46569
Bump Version
Gujal00 Oct 18, 2020
4df9e2f
Fix MP4Upload
Gujal00 Oct 18, 2020
7d52660
Update captcha_lib (#384)
frankiek3 Oct 24, 2020
4982220
Revert "Update captcha_lib (#384)" (#385)
Gujal00 Oct 25, 2020
502477b
Fix vidia
Gujal00 Oct 25, 2020
bb60300
Add elupload
Gujal00 Oct 25, 2020
3aad194
Bump Version
Gujal00 Oct 25, 2020
81e1350
fix vidoza (#386)
Oct 28, 2020
1c829c8
fix UnicodeWarning (#387)
Oct 29, 2020
c6fb0ec
* Update captcha_lib (#391)
frankiek3 Oct 30, 2020
31a6c90
gdrive fix
Gujal00 Nov 1, 2020
59b67ab
use googledrive addon
Gujal00 Nov 6, 2020
1f059b1
Add streamsb resolver (#394)
groggyegg Nov 6, 2020
8f0fd8f
Bump version
Gujal00 Nov 6, 2020
ed51aea
Add letsupload
Gujal00 Nov 8, 2020
a1f3421
Additional streamz domain
Gujal00 Nov 9, 2020
98d126f
FEmbed additional domain
Gujal00 Nov 10, 2020
4586b13
googlevideo change in import
Gujal00 Nov 14, 2020
0b049c6
Fix Streamtape resolver
Gujal00 Nov 17, 2020
dcaf724
bump
Gujal00 Nov 17, 2020
1fa453f
Add VideoSeyred resolver
Gujal00 Nov 20, 2020
17e7709
use six.iteritems
Gujal00 Nov 21, 2020
01953a4
fix mystream (#400)
Nov 22, 2020
b04d0ca
merge vidnode and vidcloud9 resolvers
Gujal00 Nov 26, 2020
dac740b
Fix cached source checking
Gujal00 Nov 29, 2020
33836a8
add ninjastream
Gujal00 Nov 30, 2020
e3b4b0f
bump version
Gujal00 Nov 30, 2020
a21fb4f
Fix class name for letsupload
Gujal00 Dec 3, 2020
60f2659
code cleanup
Gujal00 Dec 4, 2020
cbe16cb
support label blacklisting
Gujal00 Dec 4, 2020
3c62651
Bump
Gujal00 Dec 6, 2020
c062d2b
fix text encoding issue under py2 (#403)
host505 Dec 6, 2020
e7d5e96
bump
Gujal00 Dec 6, 2020
5877949
allow mp4 in vidlox
Gujal00 Dec 10, 2020
6ce8b0b
Fembed additional domain
Gujal00 Dec 14, 2020
af956ee
Allow dailymotion live streams (#405)
Twilight0 Dec 14, 2020
5582976
youdbox fix
Gujal00 Dec 16, 2020
f6fdb15
Fix StreamZ resolver
Gujal00 Dec 19, 2020
031c2c9
Fix Upstream Resolver
Gujal00 Dec 19, 2020
d18dc25
bump
Gujal00 Dec 19, 2020
d0c26a5
Fix Playtube
Gujal00 Dec 21, 2020
e8436b7
Fix hdvid playback
Gujal00 Dec 24, 2020
edb0ef6
fix streamz
Gujal00 Dec 26, 2020
a44de70
Fix streamtape
Gujal00 Dec 29, 2020
04279a8
Bump
Gujal00 Jan 3, 2021
27fa768
Fix Streamtape
Gujal00 Jan 5, 2021
2e9d90b
Add json POST support
Gujal00 Jan 5, 2021
d370a9b
Add k2s resolver
Gujal00 Jan 5, 2021
b8234bd
bump
Gujal00 Jan 7, 2021
902fe68
fix tvlogy
Gujal00 Jan 7, 2021
d2a2598
Fix Speedwatch resolver
Gujal00 Jan 11, 2021
c98737f
Add evoload resolver
Gujal00 Jan 12, 2021
4b1cc82
add abcvideo resolver
Gujal00 Jan 12, 2021
dad2db8
restore gamovideo resolver
Gujal00 Jan 12, 2021
330bf50
Add videos.sh resolver
Gujal00 Jan 12, 2021
169d774
remove vidup resolver
Gujal00 Jan 12, 2021
0db1c0a
remove jetload
Gujal00 Jan 12, 2021
2d48feb
remove hardcoded user-agent
Gujal00 Jan 12, 2021
e04c605
bump
Gujal00 Jan 12, 2021
a17da7d
Update videossh.py (#415)
addon-lab Jan 14, 2021
967e734
restore jetload
Gujal00 Jan 14, 2021
cada20a
fix Voe.sx
Gujal00 Jan 15, 2021
09fdbb6
update evoload.io (#416)
addon-lab Jan 15, 2021
05d052e
doodstream addl domain
Gujal00 Jan 17, 2021
507b47e
bump
Gujal00 Jan 17, 2021
8b0c05a
add speedostream
Gujal00 Jan 19, 2021
d35e97b
add vidcloud
Gujal00 Jan 19, 2021
7b0e2aa
fix evoload
Gujal00 Jan 21, 2021
ab81e02
add file deleted check to streamz
Gujal00 Jan 27, 2021
fa2eef5
save dl client_id in settings
Gujal00 Jan 28, 2021
827a564
bump
Gujal00 Jan 28, 2021
2e11bd4
Add MegaupNet resolver
Gujal00 Jan 31, 2021
713b180
Fix TVLogy
Gujal00 Feb 7, 2021
e0df2d1
fix GoFile
Gujal00 Feb 7, 2021
4e1b0af
bump
Gujal00 Feb 9, 2021
a9d6f1a
fix tvlogy
Gujal00 Feb 11, 2021
b4d7075
update pattern fembed (#424)
addon-lab Feb 18, 2021
8a7c8a9
bump
Gujal00 Feb 20, 2021
0868d66
Add cocoscope support (#425)
azzy9 Feb 24, 2021
7e2b348
Add lbry.tv & odysee.com support (#428)
azzy9 Mar 2, 2021
e94b22e
Update README.md (#429)
azzy9 Mar 2, 2021
8e3566e
Fix Ninjastream
Gujal00 Mar 2, 2021
5792f26
update youtube
Gujal00 Mar 3, 2021
7ecfce6
fix voe (#430)
anis3 Mar 5, 2021
e5cba46
fix TypeError (#432)
anis3 Mar 12, 2021
415c50e
bump
Gujal00 Mar 12, 2021
d97aaa9
add vidmojo resolver
Gujal00 Mar 14, 2021
71f2f92
bump
Gujal00 Mar 14, 2021
9307962
refactor test_stream to a separate function
soraxas Mar 14, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
refactor test_stream to a separate function
Signed-off-by: Tin Lai <[email protected]>
  • Loading branch information
soraxas committed Mar 14, 2021
commit 93079625993a024a747133be68148cd9d71b39eb
131 changes: 66 additions & 65 deletions lib/resolveurl/hmf.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def resolve(self, include_universal=True, allow_popups=True):
stream_url = resolver.get_media_url(self._host, self._media_id)
if stream_url.startswith("//"):
stream_url = 'http:%s' % stream_url
if stream_url and self.__test_stream(stream_url):
if stream_url and test_stream(stream_url):
self.__resolvers = [resolver] # Found a working resolver, throw out the others
self._valid_url = True
return stream_url
Expand Down Expand Up @@ -230,70 +230,6 @@ def valid_url(self):
self._valid_url = True if resolvers else False
return self._valid_url

def __test_stream(self, stream_url):
"""
Returns True if the stream_url gets a non-failure http status (i.e. <400) back from the server
otherwise return False

Intended to catch stream urls returned by resolvers that would fail to playback
"""
# parse_qsl doesn't work because it splits elements by ';' which can be in a non-quoted UA
try:
headers = dict([item.split('=') for item in (stream_url.split('|')[1]).split('&')])
except:
headers = {}
for header in headers:
headers[header] = urllib_parse.unquote_plus(headers[header])
common.logger.log_debug('Setting Headers on UrlOpen: %s' % headers)

try:
import ssl
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
opener = urllib_request.build_opener(urllib_request.HTTPSHandler(context=ssl_context))
urllib_request.install_opener(opener)
except:
pass

try:
msg = ''
request = urllib_request.Request(stream_url.split('|')[0], headers=headers)
# only do a HEAD request. gujal
request.get_method = lambda: 'HEAD'
# set urlopen timeout to 15 seconds
http_code = urllib_request.urlopen(request, timeout=15).getcode()
except urllib_error.HTTPError as e:
if isinstance(e, urllib_error.HTTPError):
http_code = e.code
if http_code == 405:
http_code = 200
else:
http_code = 600
except urllib_error.URLError as e:
http_code = 500
if hasattr(e, 'reason'):
# treat an unhandled url type as success
if 'unknown url type' in str(e.reason).lower():
return True
else:
msg = e.reason
if not msg:
msg = str(e)

except Exception as e:
http_code = 601
msg = str(e)
if msg == "''":
http_code = 504

# added this log line for now so that we can catch any logs on streams that are rejected due to test_stream failures
# we can remove it once we are sure this works reliably
if int(http_code) >= 400 and int(http_code) != 504:
common.logger.log_warning('Stream UrlOpen Failed: Url: %s HTTP Code: %s Msg: %s' % (stream_url, http_code, msg))

return int(http_code) < 400 or int(http_code) == 504

def __nonzero__(self):
if self._valid_url is None:
return self.valid_url()
Expand All @@ -305,3 +241,68 @@ def __str__(self):

def __repr__(self):
return self.__str__()


def test_stream(stream_url):
"""
Returns True if the stream_url gets a non-failure http status (i.e. <400) back from the server
otherwise return False

Intended to catch stream urls returned by resolvers that would fail to playback
"""
# parse_qsl doesn't work because it splits elements by ';' which can be in a non-quoted UA
try:
headers = dict([item.split('=') for item in (stream_url.split('|')[1]).split('&')])
except:
headers = {}
for header in headers:
headers[header] = urllib_parse.unquote_plus(headers[header])
common.logger.log_debug('Setting Headers on UrlOpen: %s' % headers)

try:
import ssl
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
opener = urllib_request.build_opener(urllib_request.HTTPSHandler(context=ssl_context))
urllib_request.install_opener(opener)
except:
pass

try:
msg = ''
request = urllib_request.Request(stream_url.split('|')[0], headers=headers)
# only do a HEAD request. gujal
request.get_method = lambda: 'HEAD'
# set urlopen timeout to 15 seconds
http_code = urllib_request.urlopen(request, timeout=15).getcode()
except urllib_error.HTTPError as e:
if isinstance(e, urllib_error.HTTPError):
http_code = e.code
if http_code == 405:
http_code = 200
else:
http_code = 600
except urllib_error.URLError as e:
http_code = 500
if hasattr(e, 'reason'):
# treat an unhandled url type as success
if 'unknown url type' in str(e.reason).lower():
return True
else:
msg = e.reason
if not msg:
msg = str(e)

except Exception as e:
http_code = 601
msg = str(e)
if msg == "''":
http_code = 504

# added this log line for now so that we can catch any logs on streams that are rejected due to test_stream failures
# we can remove it once we are sure this works reliably
if int(http_code) >= 400 and int(http_code) != 504:
common.logger.log_warning('Stream UrlOpen Failed: Url: %s HTTP Code: %s Msg: %s' % (stream_url, http_code, msg))

return int(http_code) < 400 or int(http_code) == 504