plugins.piaulizaportal: new plugin #5508
Conversation
https://ulizaportal.jp/ https://t.pia.jp/streaming For video platform "ULIZA" operated by the PIA Corporation. Tickets purchased at "PIA LIVE STREAM" are also used for "ULIZA".
There was a problem hiding this comment.
Thanks for the PR, @pzhlkj6612
the second one, however, makes VLC player end immediately.
#EXT-X-ENDLIST
See --player-no-close.
Once the stream has been fully read by Streamlink and the ringbuffer been drained, Streamlink CLI will close the player process and terminate by default. WIth --player-no-close, it'll wait until the player process terminates (e.g. when the user closes it or when it closes on its own).
Regarding the plugin itself, it looks similar to plugins/streamingproviders which are already implemented, so I guess merging it would be ok. Opening a plugin request first before submitting a pull request would be much preferred in the future though, because it avoids situations like this, where the validity of such a plugin needs to be checked after the PR was already submitted.
| schema=validate.Schema( | ||
| re.compile(r"""https://vms-api.p.uliza.jp/v1/prog-index.m3u8[^"]+"""), | ||
| validate.get(0), | ||
| validate.url(), | ||
| ), |
There was a problem hiding this comment.
re.Pattern schemas result in either None or re.Match, so the following validate.get() call could be made on a NoneType, which will lead to an AttributeError.
There was a problem hiding this comment.
My bad, I forgot validate.none_or_all.
Fixed in 663e5fc . More checks are added as well.
|
|
||
| expires = self.match.group("expires") | ||
| if expires and int(expires) <= time.time(): | ||
| raise FatalPluginError("The link is expired") |
There was a problem hiding this comment.
Invalid / expired input URLs should just return None and log an error message. PluginError and FatalPluginError are reserved for other use cases.
There was a problem hiding this comment.
Invalid / expired input URLs should just return
Noneand log an error message.
So, --retry-streams and --retry-max will retry expired links...? I think it's in vain.
PluginErrorandFatalPluginErrorare reserved for other use cases.
OK. The reason I chose FatalPluginError is that it can stop the retry loop.
streamlink/src/streamlink_cli/main.py
Lines 464 to 472 in 54136f4
Changing FatalPluginError to error messages does not hurt, though.
- Invalid / expired input URLs should just return `None` and log an error message. `PluginError` and `FatalPluginError` are reserved for other use cases. - `re.Pattern` schemas result in either `None` or `re.Match`, so the following `validate.get()` call could be made on a `NoneType`, which will lead to an `AttributeError`. Co-Authored-By: bastimeyer <mail@bastimeyer.de>
|
Thanks for your review, @bastimeyer .
OK, now I understand.
Do you mean "plugins/streamingvideoprovider.py"? I didn't know it before, and I found this plugin has been removed by #3843.
Well, I just searched the domain name of that platform, checked "Plugin requests § CONTRIBUTING.md" and then submitted this PR. I made a disallowed plugin several months ago, so I'm being more careful than before. |
There was a problem hiding this comment.
Are there any actual free live streams that can be tested?
Do you mean "plugins/streamingvideoprovider.py"?
No, I was talking about plugins for streaming sites which provide similar services like this one, e.g. UStreamTV and others.
Some further code cleanups:
- max line length
- the
Refererheader with the current URL should be set on the player JS request - the order of query string parameters can be arbitrary, so matching
expiresin the pluginmatcher is not the way to go
diff --git a/src/streamlink/plugins/piaulizaportal.py b/src/streamlink/plugins/piaulizaportal.py
index bdaa7fbe..48640d49 100644
--- a/src/streamlink/plugins/piaulizaportal.py
+++ b/src/streamlink/plugins/piaulizaportal.py
@@ -9,6 +9,7 @@ $notes Tickets purchased at "PIA LIVE STREAM" are used for this platform.
import logging
import re
import time
+from urllib.parse import parse_qsl, urlparse
from streamlink.plugin import Plugin, pluginmatcher
from streamlink.plugin.api import validate
@@ -18,36 +19,34 @@ from streamlink.stream.hls import HLSStream
log = logging.getLogger(__name__)
-@pluginmatcher(
- re.compile(
- r"https://ulizaportal\.jp/pages/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})(\?expires=(?P<expires>\d+).*)?",
- ),
-)
+@pluginmatcher(re.compile(
+ r"https://ulizaportal\.jp/pages/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})",
+))
class PIAULIZAPortal(Plugin):
- def __init__(self, *args, **kwargs):
- super().__init__(*args, **kwargs)
- self.session.http.headers.update({"Referer": "https://ulizaportal.jp/"})
+ _URL_PLAYER_DATA = "https://player-api.p.uliza.jp/v1/players/"
+ _URL_PLAYLIST = "https://vms-api.p.uliza.jp/v1/prog-index.m3u8"
def _get_streams(self):
self.id = self.match.group("id")
- expires = self.match.group("expires")
- if expires and int(expires) <= time.time():
- log.error("The link is expired")
+ try:
+ expires = int(dict(parse_qsl(urlparse(self.url).query)).get("expires", 0))
+ except ValueError:
+ expires = 0
+ if 0 < expires <= time.time():
+ log.error("The URL has expired")
return None
self.title, player_data_url = self.session.http.get(
self.url,
schema=validate.Schema(
validate.parse_html(),
- validate.union(
- (
- validate.xml_xpath_string(".//head/title[1]/text()"),
- validate.xml_xpath_string(
- ".//script[@type='text/javascript'][contains(@src,'https://player-api.p.uliza.jp/v1/players/')]/@src",
- ),
+ validate.union((
+ validate.xml_xpath_string(".//head/title[1]/text()"),
+ validate.xml_xpath_string(
+ f".//script[@type='text/javascript'][contains(@src,'{self._URL_PLAYER_DATA}')][1]/@src",
),
- ),
+ )),
),
)
if not player_data_url:
@@ -56,8 +55,11 @@ class PIAULIZAPortal(Plugin):
m3u8_url = self.session.http.get(
player_data_url,
+ headers={
+ "Referer": self.url,
+ },
schema=validate.Schema(
- re.compile(r"""https://vms-api.p.uliza.jp/v1/prog-index.m3u8[^"]+"""),
+ re.compile(rf"""{re.escape(self._URL_PLAYLIST)}[^"']+"""),
validate.none_or_all(
validate.get(0),
validate.url(),
diff --git a/tests/plugins/test_piaulizaportal.py b/tests/plugins/test_piaulizaportal.py
index 6f9b6293..dd084910 100644
--- a/tests/plugins/test_piaulizaportal.py
+++ b/tests/plugins/test_piaulizaportal.py
@@ -5,7 +5,19 @@ from tests.plugins import PluginCanHandleUrl
class TestPluginCanHandleUrlPIAULIZAPortal(PluginCanHandleUrl):
__plugin__ = PIAULIZAPortal
- should_match = [
- "https://ulizaportal.jp/pages/005f18b7-e810-5618-cb82-0987c5755d44",
- "https://ulizaportal.jp/pages/005e1b23-fe93-5780-19a0-98e917cc4b7d?expires=4102412400&signature=f422a993b683e1068f946caf406d211c17d1ef17da8bef3df4a519502155aa91&version=1",
+ should_match_groups = [
+ (
+ "https://ulizaportal.jp/pages/005f18b7-e810-5618-cb82-0987c5755d44",
+ {"id": "005f18b7-e810-5618-cb82-0987c5755d44"},
+ ),
+ (
+ "https://ulizaportal.jp/pages/005e1b23-fe93-5780-19a0-98e917cc4b7d"
+ + "?expires=4102412400&signature=f422a993b683e1068f946caf406d211c17d1ef17da8bef3df4a519502155aa91&version=1",
+ {"id": "005e1b23-fe93-5780-19a0-98e917cc4b7d"},
+ ),
+ ]
+
+ should_not_match = [
+ "https://ulizaportal.jp/pages/",
+ "https://ulizaportal.jp/pages/invalid-id",
]- max line length - the `Referer` header with the current URL should be set on the player JS request - the order of query string parameters can be arbitrary, so matching `expires` in the pluginmatcher is not the way to go Co-Authored-By: bastimeyer <mail@bastimeyer.de>
OK.
It seems that our linter (ruff) does not conform to the code style, or there are some gaps in the configuration?
I searched but cannot find any free live streams. Here I'm providing some info about a real event. The "Save All Resources" Chrome extension helped me download it: downloaded infoPlayer dataBefore live stream starts: {"src":{"video":"https://uliza-cover-images.p.uliza.jp/.../....png"live streams and archives available for viewing during the day of the live stream: {"src":{"video":"https://vms-api.p.uliza.jp/v1/prog-index.m3u8?expires=...&signature=...&ss=...&version=1&live=true"Tomorrow's archives: {"src":{"video":"https://vms-api.p.uliza.jp/v1/prog-index.m3u8?expires=...&signature=...&ss=...&version=1"Note: the player data is not always valid JavaScript. プレゼンテーションプレイヤーのサンプル gives me this: ...{"authDomain":[]}}},
src: {"video":"https://vms-api.p.uliza.jp/v1/prog-index.m3u8?expires=...&signature=...&ss=...&version=1"...M3U8 playlistsLive streams: #EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=4000000
https://<string_1>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_4000/chunklist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000
https://<string_1>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_2000/chunklist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000
https://<string_1>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_1000/chunklist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4000000
https://<string_2>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_4000/chunklist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000
https://<string_2>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_2000/chunklist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000
https://<string_2>.cloudfront.net/hls/video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_1000/chunklist.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:****
#EXT-X-KEY:METHOD=AES-128,URI="/hls/key/ipblite/key.php"
#EXTINF:4.004,
media-\w{9,}_****.ts
#...
Archives available for viewing during the day of the live stream: #EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=4000000
https://<string_1>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_4000/chunklist_DVR.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000
https://<string_1>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_2000/chunklist_DVR.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000
https://<string_1>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_1000/chunklist_DVR.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4000000
https://<string_2>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_4000/chunklist_DVR.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000
https://<string_2>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_2000/chunklist_DVR.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000
https://<string_2>.cloudfront.net/hls/dvr/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_1000/chunklist_DVR.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-KEY:METHOD=AES-128,URI="/hls/key/ipblite/key.php?{encKeySessionid}"
#EXTINF:10.01,
../../../video/ipblite\d{2}/<numbers_1>_livestream<numbers_2>_4000/media-\w{9,}_DVR_0.ts
#...
Tomorrow's archives: #EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4600000
https://<string_3>.cloudfront.net/<numbers_1>/<YYYYmmddHHMMSS>_[a-z\d]{64}_1.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2300000
https://<string_3>.cloudfront.net/<numbers_1>/<YYYYmmddHHMMSS>_[a-z\d]{64}_2.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1200000
https://<string_3>.cloudfront.net/<numbers_1>/<YYYYmmddHHMMSS>_[a-z\d]{64}_3.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXT-X-KEY:METHOD=AES-128,URI="https://vms-api.p.uliza.jp/v1/keys?st=...&s=0"
#EXTINF:10.033333,
<YYYYmmddHHMMSS>_[a-z\d]{64}_1/mediafile_0.ts
#...EOF |
No, ruff ignores lines with single "words" without whitespace which exceed the max line length. |
|
|
||
|
|
||
| @pluginmatcher(re.compile( | ||
| r"https://ulizaportal\.jp/pages/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})", |
There was a problem hiding this comment.
| r"https://ulizaportal\.jp/pages/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})", | |
| r"https?://ulizaportal\.jp/pages/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})", | |
Hi, @bastimeyer , could you please help me make the RegEx a little more generic by committing a quick fix?
There was a problem hiding this comment.
I don't think it's necessary. https:// is applied to scheme-less input URLs automatically. The plugin expects URLs to be copied by users with the provided UUIDs (nobody will manually type the UUID), so unless stream URLs get shared with http:// schemes explicitly (which I don't think they will), this is not needed. Apart from that, nobody can push to master directly, so another PR would be needed anyway.
There was a problem hiding this comment.
I don't think it's necessary. ...
OK, I agree.
I found that almost all pluginmatchers match both "http" and "https", so I was wondering if I forgot to do so to comply with any rules.
Apart from that, nobody can push to master directly, so another PR would be needed anyway.
Got it. It's necessary protection.
Summary
This is a very simple extractor for video platform "ULIZA" operated by the PIA Corporation.
Tickets purchased at "PIA LIVE STREAM" are used for "ULIZA".
Bug
There are two public videos:
The first one works well with this plugin; the second one, however, makes VLC player end immediately.
M3U8 playlists and logs are as follows:
M3U8 playlists and logs
The master M3U8 is:
The
BANDWIDTH=1200000M3U8 is:Log:
Can anyone help me with this?