Skip to content

plugin: new matchers API #3821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 29, 2021
Merged

Conversation

bastimeyer
Copy link
Member

Opening this as a draft, see #3814.

The interesting commits are the first two and the last one, the rest are plugin updates, which are not that interesting and bloat up this diff a lot.

I've split up the plugin changes into multiple commits, based on the type of changes. Since the URL regexes had to be moved, I immediately applied fixes, as it would be duplicate work otherwise.

I'm not sure if I'm convinced with the regex pattern compilation in the decorator and the language injection annotations. After thinking about it again, it's probably better to just call re.compile in the decorator patameter.

Apart from that, the only thing that's missing is documentation.

@bastimeyer bastimeyer force-pushed the plugin/matchers branch 6 times, most recently from 8e7ad26 to 9709153 Compare June 27, 2021 13:07
- drop `Plugin.can_handle_url` and `Plugin.priority`
- introduce `Plugin.matchers`
- implement `pluginmatcher` decorator
- automatically match url (on change) and store result
- update `Session.resolve_url`
- add and fix related tests
@bastimeyer
Copy link
Member Author

I've replaced @pluginmatcher(pattern, flags, priority) with @pluginmatcher(re.compile(pattern, flags), priority) now, as language injection annotations will cause issues in editors/IDEs other than PyCharm and are too verbose. This is much better I think.

Documentation still needs to be written, but that can be done later on to make things not more complicated here.

@bastimeyer bastimeyer marked this pull request as ready for review June 27, 2021 14:52
- update protocol plugins: akamaihd, dash, hds, hls, http and rtmp
- remove `plugin.parse_url_params`, update `plugin.parse_params`
- refactor DASH tests
Basic plugins do only have one URL regex, don't reference capture
groups, don't translate URLs and their `can_handle_url` is simple.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Plugins with only one URL regex which reference capture groups, but
don't translate URLs and their `can_handle_url` is simple.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Update plugins which have specific checks in their `can_handle_url`
classmethod. Make those plugins return no streams instead when the
specific checks would have returned False otherwise.

Also update plugins calling other plugins' `can_handle_url`.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Plugins that use more than one URL matcher pattern.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Resolve deprecated plugins which are using the old
`can_handle_url(url)` and `priority(url)` methods.

Since the `Plugin` class doesn't implement the `priority` classmethod
with the NORMAL_PRIORITY return value anymore, the NORMAL_PRIORITY
gets set by the resolver if the method is missing on plugins which
implement `can_handle_url` and don't have the `matchers` list.
@back-to back-to merged commit 518598f into streamlink:master Jun 29, 2021
@bastimeyer bastimeyer deleted the plugin/matchers branch June 29, 2021 15:02
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
- drop `Plugin.can_handle_url` and `Plugin.priority`
- introduce `Plugin.matchers`
- implement `pluginmatcher` decorator
- automatically match url (on change) and store result
- update `Session.resolve_url`
- add and fix related tests
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
- update protocol plugins: akamaihd, dash, hds, hls, http and rtmp
- remove `plugin.parse_url_params`, update `plugin.parse_params`
- refactor DASH tests
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
Basic plugins do only have one URL regex, don't reference capture
groups, don't translate URLs and their `can_handle_url` is simple.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
)

Plugins with only one URL regex which reference capture groups, but
don't translate URLs and their `can_handle_url` is simple.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
…#3821)

Update plugins which have specific checks in their `can_handle_url`
classmethod. Make those plugins return no streams instead when the
specific checks would have returned False otherwise.

Also update plugins calling other plugins' `can_handle_url`.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
…k#3821)

Plugins that use more than one URL matcher pattern.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
Resolve deprecated plugins which are using the old
`can_handle_url(url)` and `priority(url)` methods.

Since the `Plugin` class doesn't implement the `priority` classmethod
with the NORMAL_PRIORITY return value anymore, the NORMAL_PRIORITY
gets set by the resolver if the method is missing on plugins which
implement `can_handle_url` and don't have the `matchers` list.
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
)

Plugins with only one URL regex which reference capture groups, but
don't translate URLs and their `can_handle_url` is simple.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
…#3821)

Update plugins which have specific checks in their `can_handle_url`
classmethod. Make those plugins return no streams instead when the
specific checks would have returned False otherwise.

Also update plugins calling other plugins' `can_handle_url`.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
…k#3821)

Plugins that use more than one URL matcher pattern.

Update and fix URL regex patterns:
- remove unnecessary anchors and optional trailing wildcards
- optionally match https (scheme update is plugin responsibility)
- escape actual dots in hostnames and path file extensions
- fix subdomain (non-)capture groups (don't match anything)
- fix unnecessarily escaped characters
- simplify character sets
- improve readability (verbose and non-verbose patterns)
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
Billy2011 added a commit to Billy2011/streamlink-27 that referenced this pull request Jul 1, 2021
Resolve deprecated plugins which are using the old
`can_handle_url(url)` and `priority(url)` methods.

Since the `Plugin` class doesn't implement the `priority` classmethod
with the NORMAL_PRIORITY return value anymore, the NORMAL_PRIORITY
gets set by the resolver if the method is missing on plugins which
implement `can_handle_url` and don't have the `matchers` list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants