Skip to content

cli.console: refactor and move into sub-package #6496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

bastimeyer
Copy link
Member

@bastimeyer bastimeyer commented Apr 2, 2025

Some more changes in preparation for #6449. Doing this in a sequence of PRs instead of just one giant one...

  1. Split the streamlink_cli.console module into a sub-package and move the ConsoleUserInputRequester into its own module.
  2. Refactor the ConsoleOutput._write() method(s), so adding the msg_status() method in the future is easier (used for the download progress status line at the bottom)
  3. Manually/gracefully close console-output and file-output streams on process exit
  4. Add ConsoleOutputStream, which wraps sys.stdout/sys.stderr and simply passes through all attribute lookups. This is necessary so we can intercept writes to stdout/stderr which we don't control via the ConsoleOutput, e.g. in Streamlink's dependencies. Otherwise, random writes to these text streams would break the status line at the bottom.
  5. Move streamlink_cli.progress into the streamlink_cli.console sub-package
  6. Refactor terminal-width / message-width logic out of the progress module (again, this has already been moved twice in the past)

Uncommitted and unfinished changes on top of this branch that will come in follow-up PRs:

  • Refactor Progress and send messages to ConsoleOutput.msg_status() instead of writing to a stream object (sys.stderr) directly and adding a carriage-return character with white-space padding at the end of the line
  • Refactor StreamRunner and its setup due to the Progress setup changes (rewriting tests is a bit annoying)
  • Add additional line-buffering and status message logic to ConsoleOutputStream (with subclasses) which handle regular messages and status messages, depending on available terminal features (ANSI control sequences, Windows console, or neither).
  • The ConsoleOutputStreamWindows class including its detection is still unfinished and currently falls back to the default writer implementation which does not support status messages
  • --progress=force doesn't make sense anymore when the required writer features are not supported, as we're now merging the regular and status messages into one stream. An idea I have is making it write the status messages as regular messages, but this would require a slower time interval by the Progress class in that case, as it updates four times per second. Basically another (minor) Progress refactor.

@bastimeyer bastimeyer force-pushed the cli/console/subpkg-and-wrapper branch 4 times, most recently from 8194ee6 to 47ffc78 Compare April 3, 2025 22:16
@bastimeyer
Copy link
Member Author

bastimeyer commented Apr 3, 2025

Been struggling with this for a bit, but it's fine now (going to merge tomorrow once the other stuff is finished). To recap, as said, the stdout/stderr stream objects need to be wrapped.

Initially, the wrapper class didn't subclass from io.TextIOWrapper (1) or typing.TextIO (2), because this required some additional setup, either for runtime stuff (1) or for static typing analysis (2), and I was unhappy about this, because there was no proper type inheritance, which required updating other typing annotations referencing the wrapper object(s).

Then I decided to fix this by first subclassing from TextIOWrapper and passing the buffer and config attributes from the given stream object (a TextIOWrapper) to its super class constructor. No issues were expected and test ran fine locally on Linux and Windows, but the Windows CI runners failed. For some reason, LF chars were translated to CRLF, which didn't make any sense considering that no newline-translation is being done when re-using the wrapped stream's config attributes.

After that, I decided to subclass from TextIO instead and define all missing methods+properties from the abstract base class, so mypy is happy. This meant that __getattr__() could not be used by the wrapper and every single method+property needed to be declared with an appropriate self._stream.XYZ redirect. Annoying. But for some reason, the issue persisted, despite all stream methods+properties simply being redirected to the wrapped stream object. I have no idea why.

So now I've reverted back to the first approach but added a respective .pyi file with all the typing annotations for the wrapper class, so it can incorrectly declare inheritance from TextIOWrapper, fixing the other typing annotations.

@bastimeyer bastimeyer force-pushed the cli/console/subpkg-and-wrapper branch 2 times, most recently from 1fb4387 to 901e69d Compare April 9, 2025 22:26
- Flush and close console/file output streams manually on process exit
- Simplify the `BrokenPipeError` fix, as streams are already flushed
- Add `StreamWrapper` base class which passes through everything
  to the given `TextIOWrapper` object. Don't subclass from
  `TextIOWrapper` or `TextIO`.
- Wrap and replace `sys.stdout` / `sys.stderr`
- Restore on `ConsoleOutput.close()`
- Fix typing in `ConsoleOutput`
Move terminal-related functions to `streamlink_cli.console.terminal`
@bastimeyer bastimeyer force-pushed the cli/console/subpkg-and-wrapper branch from 901e69d to 63ae8b8 Compare April 11, 2025 16:36
@bastimeyer bastimeyer marked this pull request as ready for review April 12, 2025 11:14
@bastimeyer bastimeyer merged commit b6c65ed into streamlink:master Apr 12, 2025
16 checks passed
@bastimeyer bastimeyer deleted the cli/console/subpkg-and-wrapper branch April 12, 2025 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant