cli.console: refactor and split into two streams #6461

bastimeyer · 2025-03-06T19:53:11Z

Replaces #6457
In preparation for #6449

This PR fixes a couple of console output issues and logger setup issues, and also splits the console output into two streams. The separation of console output streams is necessary in order for the download progress to be fixed later on. Progress output should never be written to a file, but since we want some console output to be written to log files as well, especially error messages, we need to use two streams.

Currently on master, the ConsoleOutput always inherits the stream of Streamlink's root logger. This is problematic because --logfile=... sets the logger stream to a file, which means all ConsoleOutput including user prompts are written to the log file instead of being written to a TTY where the user has to write their input.

Also, --quiet currently does only affect the log level, as it's treated as an alias of --loglevel=none, which means the inherited output stream is still stdout, so all console messages and error messages are printed regardless. --quiet should mean no output at all.

Another problem is that if there's no TTY output for user prompts, those prompts currently don't fail. If the user doesn't have a chance to know what's prompted, then the prompt must fail. And failure is currently also not handled correctly and None is returned instead, which is wrong.

I've decided to use one single PR branch for these changes, because they are all a bit intertwined. Splitting this up would make this unnecessarily complicated for me.

So, a quick overview of the individual commits:

Fix the method name of the ConsoleOutput password prompt according to Streamlink's public interface of the UserInputRequester
Fix the prompts, raise OSError on failure, and most importantly, check for valid input and output TTY streams (must exist and must not be file streams).
Update the error handling of the one function in Streamlink's CLI that doesn't use the Streamlink session's UserInputRequester and that instead calls the ask() method of the ConsoleOutput directly.
Refactor ConsoleOutput and use two separate streams, one for console output and one for file output. Both are optional, as stdout/stderr can be missing, or there's no log file stream. Also ensure that the file output is not a TTY, so that --logfile=/dev/tty doesn't duplicate console messages (as msg() and msg_json() write to both streams).
Refactor the setup of ConsoleOutput and Streamlink's root logger and actually use two separate streams. Make the root logger inherit the primary stream of the ConsoleOutput (not the other way around). If there's a log file, attach the file stream to the ConsoleOutput instance.
Check if the output stream for Streamlink's root logger actually exists. Previously, the logging module of Python's stdlib switched to stderr if the passed stream was None, and we also had to check if stderr existed and use an additional fallback to devnull as a FileHandler in that case. Simply don't attach any log handlers to the root logger if there's the passed stream is None.
Make ConsoleOutput fall back from stdout to stderr if stdout doesn't exist, to align with the previous fallback mechanic of the root logger that was removed in the previous commit. Since the primary ConsoleOutput stream is inherited by the logger now, the old behavior is restored. But now, both console and log output have a fallback.
Make --quiet suppress all text output, not just log output.

Considering that there are a lot of changes, this needs a bit of additional testing before I'm going to merge this. As always, I've added tests with full coverage for the changed code parts, but this doesn't cover all potential combinations of CLI arguments and environments.

gravyboat

Thanks for breaking this up in to several commits, made it much easier to review each chunk. Made a couple non-blocking notes. Feel free to address or not address whatever you prefer.

src/streamlink_cli/argparser.py

src/streamlink_cli/main.py

tests/cli/main/test_check_file_output.py

tests/cli/test_argparser.py

tests/cli/test_compat.py

tests/cli/test_console.py

tests/test_plugin_userinput.py

bastimeyer · 2025-03-07T10:10:20Z

Some quick checks... Will extend this later with the remaining checks... I have to go AFK now.

I/O streams

master

# OK: missing stdout results in stderr being used
$ streamlink doesnotexist 1>&-
error: No plugin can handle URL: doesnotexist

# FAIL: --logfile affects console output
$ streamlink doesnotexist --logfile=/dev/null

# OK: BrokenPipeError is ignored when writing
$ streamlink -l debug | head -n4
[cli][debug] OS:         Linux-6.13.4-1-git-x86_64-with-glibc2.41
[cli][debug] Python:     3.13.2
[cli][debug] OpenSSL:    OpenSSL 3.4.1 11 Feb 2025
[cli][debug] Streamlink: 7.1.3+9.gd74fef05

PR

# OK: missing stdout results in stderr being used
$ streamlink doesnotexist 1>&-
error: No plugin can handle URL: doesnotexist

# OK: --logfile does not affect console output
$ streamlink doesnotexist --logfile=/dev/null
error: No plugin can handle URL: doesnotexist

# OK: --logfile still has console output
$ streamlink doesnotexist --logfile=/tmp/logfile; cat /tmp/logfile; rm -f /tmp/logfile
error: No plugin can handle URL: doesnotexist
error: No plugin can handle URL: doesnotexist

# OK: BrokenPipeError is ignored when writing (was broken in previous PR state)
$ streamlink -l debug | head -n4
[cli][debug] OS:         Linux-6.13.4-1-git-x86_64-with-glibc2.41
[cli][debug] Python:     3.13.2
[cli][debug] OpenSSL:    OpenSSL 3.4.1 11 Feb 2025
[cli][debug] Streamlink: 7.1.3+18.gc5a92263

Prompts

Using a plugin which has required arguments and thus triggers a prompt without those args being set

master

# OK: but error message is ambiguous
$ streamlink clubbingtv.com/ 0>&-
error: no TTY available

# OK: but error message is ambiguous
$ streamlink clubbingtv.com/ </dev/null
error: no TTY available

# FAIL: prints both prompts at the same time, but only waits for one input and then continues
$ streamlink clubbingtv.com/ 1>&-
Enter clubbingtv username: Enter clubbingtv password: 
[cli][info] Found matching plugin clubbingtv for URL clubbingtv.com/

# FAIL: waits for user input without any prompts
$ streamlink clubbingtv.com/ 1>&- 2>&-

# FAIL: waits for input despite prompt being written to non-TTY stream
$ streamlink clubbingtv.com/ | cat
Enter clubbingtv username:

# FAIL: waits fpr user input without any prompts
$ streamlink clubbingtv.com/ --logfile=/dev/null

PR

# OK
$ streamlink clubbingtv.com/ 0>&-
error: No input TTY available

# OK
$ streamlink clubbingtv.com/ </dev/null
error: No input TTY available

# OK, returns the error message from Python's input() builtin, then stops immediately
$ streamlink clubbingtv.com/ 1>&-
Enter clubbingtv username: error: input(): lost sys.stdout

# OK: stops immediately
$ streamlink clubbingtv.com/ 1>&- 2>&-

# OK: stops immediately
$ streamlink clubbingtv.com/ | cat
error: No output TTY available

# OK: logfile does not affect console output
$ streamlink clubbingtv.com/ --logfile=/dev/null
Enter clubbingtv username:

--quiet

master

# FAIL: --quiet is not quiet and just an alias of --loglevel=none
$ streamlink doesnotexist --quiet -l debug
error: No plugin can handle URL: doesnotexist

PR

# OK: --quiet suppresses console and log output
$ streamlink doesnotexist --quiet -l debug

bastimeyer · 2025-03-09T17:23:12Z

f2b7764 is probably unnecessary if we re-arrange the commits. --quiet should set None as the console_output stream, not devnull_txt. This was necessary because of the previous logger setup that defaulted to stderr on None, as the console output was inherited here.

I'll have a look at this again some time next week.

Use the same method name as the public interface of UserInputRequester

- Raise `OSError` instead of returning `None` on error - Check for valid input and output streams when prompting the user - Pass-through `KeyboardInterrupt` - Fix typing - Update tests

Allow the primary output stream to be `None`: Users should be able to disable console output as a whole via `--quiet` (upcoming changes). Use a secondary stream for writing messages into a file: Inherit the `--logfile` stream from the filehandler of Streamlink's root logger (upcoming changes), but skip it if it's a TTY stream, so `--logfile=/dev/stdout` doesn't result in log message duplication.

When `--logfile=file` is set, write console output messages both to the default output stream and the log file stream instead of overriding the console output stream with the log file stream. Log messages will still follow the log file stream, if `--logfile=file` is set, and won't appear on the default output stream, just like before. If `--logfile=/dev/tty` (and similar) is set, ignore the log file stream on the `ConsoleOutput`, so messages won't be duplicated. Refactor and split up `setup_logger_and_logger()` into `setup_console()` and `setup_logger()`. Add `silent_log` attribute on the `args` object (`argparse.Namespace`).

Before creating the `StreamHandler`, check if `stream` exists, rather than letting the stdlib check for its existence when initializing the handler object and falling back to `stderr`. If the passed stream does not exist, then don't attach any handlers to the root logger and prevent logging this way instead of using a devnull `FileHandler` if `stderr` also does not exist.

Let the `ConsoleOutput` use `stderr` if `stdout` does not exist. Since Streamlink's root logger inherits the `ConsoleOutput` stream, the fallback mechanism is also inherited (see previous commit).

bastimeyer added bug cleanup logging CLI labels Mar 6, 2025

bastimeyer force-pushed the cli/console/separate-streams branch from def86ea to 4ef3669 Compare March 6, 2025 19:59

gravyboat approved these changes Mar 7, 2025

View reviewed changes

bastimeyer force-pushed the cli/console/separate-streams branch from 4ef3669 to e89fb5c Compare March 7, 2025 09:55

bastimeyer force-pushed the cli/console/separate-streams branch 3 times, most recently from c5a9226 to 9b01dca Compare March 7, 2025 21:05

bastimeyer force-pushed the cli/console/separate-streams branch from 9b01dca to 3a33046 Compare March 15, 2025 15:14

bastimeyer added 8 commits March 15, 2025 16:23

cli.console: rename askpass() to ask_password()

8c210d6

Use the same method name as the public interface of UserInputRequester

cli.console: refactor prompts

f84c36f

- Raise `OSError` instead of returning `None` on error - Check for valid input and output streams when prompting the user - Pass-through `KeyboardInterrupt` - Fix typing - Update tests

cli.main: refactor check_file_output()

0a2d67d

cli: fall back to stderr for console output

68baeeb

Let the `ConsoleOutput` use `stderr` if `stdout` does not exist. Since Streamlink's root logger inherits the `ConsoleOutput` stream, the fallback mechanism is also inherited (see previous commit).

cli: make --quiet also suppress ConsoleOutput

863094d

bastimeyer force-pushed the cli/console/separate-streams branch from 3a33046 to 863094d Compare March 15, 2025 15:25

bastimeyer merged commit be8e8ab into streamlink:master Mar 15, 2025
23 checks passed

bastimeyer deleted the cli/console/separate-streams branch March 15, 2025 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

cli.console: refactor and split into two streams #6461

cli.console: refactor and split into two streams #6461

Uh oh!

bastimeyer commented Mar 6, 2025 •

edited

Loading

Uh oh!

gravyboat left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bastimeyer commented Mar 7, 2025 •

edited

Loading

Uh oh!

bastimeyer commented Mar 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cli.console: refactor and split into two streams #6461

cli.console: refactor and split into two streams #6461

Uh oh!

Conversation

bastimeyer commented Mar 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gravyboat left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bastimeyer commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I/O streams

Prompts

--quiet

Uh oh!

bastimeyer commented Mar 9, 2025

Uh oh!

Uh oh!

Uh oh!

bastimeyer commented Mar 6, 2025 •

edited

Loading

bastimeyer commented Mar 7, 2025 •

edited

Loading