Skip to content

[hls] New option --hls-segment-ignore-names #1432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 7, 2018
Merged

[hls] New option --hls-segment-ignore-names #1432

merged 1 commit into from
Feb 7, 2018

Conversation

back-to
Copy link
Collaborator

@back-to back-to commented Jan 13, 2018

You can ignore segment names that might corrupt or lag your output

  • this will freeze the player until there is a valid segment name
  • this will not add the segment to an output file,
    so it won't be damaged and can be edited.

Note: The --hls-timeout must be increased, to a time that is longer
than the ignored break.

Example:
streamlink URL best --hls-segment-ignore-names 000,001,002 --hls-timeout 1200

This will ignore every segment that ends with 000.ts, 001.ts and 002.ts


some names/text can be changed, if someone has a better idea.


Hey @RedPenguin2

it would be nice if you could test this,
if you don't know how to, i can help you.

streamlink http://www.tbd.com best -l debug --hls-segment-ignore-names 0000,0001,0002 -o test.ts --hls-timeout 2400
change the url with your hls link

this PR would fix #687

You can ignore segment names that might corrupt or lag your output

- this will freeze the player until there is a valid segment name
- this will not add the segment to an output file,
  so it won't be damaged and can be edited.

Note: The `--hls-timeout` must be increased, to a time that is longer
than the ignored break.

Example:
`streamlink URL best --hls-segment-ignore-names 000,001,002
--hls-timeout 1200`

This will ignore every segment that ends with 000.ts, 001.ts and 002.ts
@codecov
Copy link

codecov bot commented Jan 13, 2018

Codecov Report

Merging #1432 into master will increase coverage by <.01%.
The diff coverage is 57.14%.

@@            Coverage Diff             @@
##           master    #1432      +/-   ##
==========================================
+ Coverage   50.48%   50.48%   +<.01%     
==========================================
  Files         104      104              
  Lines        7472     7485      +13     
==========================================
+ Hits         3772     3779       +7     
- Misses       3700     3706       +6

@RedPenguin2
Copy link

RedPenguin2 commented Jan 13, 2018

@back-to I got streamlink setup with the pull request but now I seem to have found a completely different issue now.

[stream.hls][debug] Segments in this playlist are encrypted

I don't remember getting that before at all.

Though I am still playing with ffplay just fine.

It seems like streamlink just keeps saying the encrypted message and the normal adding segments to queue and skipping segments.

I don't seem to actually be getting any output ts file though.

Though I tested really quick with out the new option and it did write a file that was playable, so it appears it's not "encrypted" in a sense like DRM or something that can't be recorded.

@back-to
Copy link
Collaborator Author

back-to commented Jan 13, 2018

the
[stream.hls][debug] Segments in this playlist are encrypted
is
#EXT-X-KEY:METHOD=AES-128,URI=...
it's supported by streamlink


the skipping segments are the corrupt mp4 files that won't get added,
so you can edit the output file with a video editor, what did not work before.

there are maybe dead minutes in the output file, but you can just cut it.


you need to run the command longer than the Pause break,
so you will get some data to the output file.

i only tested tbd, the other channels might have different filenames

@RedPenguin2
Copy link

Oh thanks for the info, going to try it again.

I really wish they really had a better commercial break method.

Watching a screen just saying "We'll Be Right Back" for 3 minutes straight with no music is even more boring than actually watching a normal commercial break. LoL

@RedPenguin2
Copy link

RedPenguin2 commented Jan 14, 2018

So far it appears to work for TBD.

Originally I had an outdated URL, that just gave me a commerical break the entire time.

I noticed on the website I guess it supplements with ads.

The only thing I noticed is, I still had to do a quick ffmpeg copy audio/video to fix the header, because even though the file looks great being just the program with no commercials, it oddly wouldn't rewind correctly in ffplay at first.

Also VLC won't play the original ts until the header is fixed.

@amurzeau
Copy link
Contributor

VLC 3.0 handle TBD stream good enough. VLC 3.0 is at RC stage as of now (downloadable in nightly builds) (see also my comment just added on the linked issue)

@gravyboat
Copy link
Member

@bastimeyer, @beardypig Any concerns over this new option?

@beardypig
Copy link
Member

Is it a work around for discontinuity issues? Seems generally useful though :)

@gravyboat
Copy link
Member

Is there any other feedback here from anyone? If not I want to get this merged since it's a good addition.

Copy link
Member

@bastimeyer bastimeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a reasonable addition to me. 👍

# this will be used to ignore segments.
self.ignore_names = list(set(self.ignore_names))
self.ignore_names = "|".join(list(map(re.escape, self.ignore_names)))
self.ignore_names_re = re.compile(r"(?:{blacklist})\.ts".format(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will ignore every segment that ends with ...

Just a minor suggestion: you should add the dollar anchor to the end of the regex, just to be sure that a segment doesn't get matched by accident if a URL somehow contains another .ts, you'll never know 😃

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ends with

i search for the end of the filename not for the uri end,
if there are params after 00.ts?foo=bar it's not the end of the filename

$ won't work for this


if i would parse the uri and just work with the path, then sure
but using urlparse might be to advanced.

Copy link
Member

@bastimeyer bastimeyer Feb 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True... I had static segment URLs in mind without query data.
\.ts(?:\?.*)?$ should work, but we could also just leave it as it is, it's good enough. :)


Default is None.

Note: The --hls-timeout must be increased, to a time that is longer than the ignored break.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this can't be automated, right? Segment lengths are dependent on the streaming service (may even be variable) and there could also be multiple segments in a row which get removed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i could change timeout=self.timeout to a bigger value
if this command is used

but that's not really automated

@gravyboat
Copy link
Member

@bastimeyer Any other feedback? If not I would like to get this merged.

@bastimeyer
Copy link
Member

No, let's merge this... Further improvements, if needed, can be added in another PR

@gravyboat gravyboat merged commit 97ec633 into streamlink:master Feb 7, 2018
@back-to back-to deleted the hls_skip_ts branch February 7, 2018 09:39
mkbloke pushed a commit to mkbloke/streamlink that referenced this pull request Aug 18, 2020
[hls] New option --hls-segment-ignore-names
back-to added a commit to back-to/streamlink that referenced this pull request Dec 13, 2020
It has some downsides such as `--hls-timeout`,
Plugin specified FilteredHLSStream should be used instead.

streamlink#3187

---

Afreeca Plugin as an Example

streamlink#3408

---

Revert streamlink#1432
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sinclair Broadcast Channels (Charge!, Comet, and TBD).
6 participants