tests: increase TIMEOUT_AWAIT_WRITE and update requests #3930

back-to · 2021-08-22T08:55:48Z

tests.mixins.stream_hls: increase TIMEOUT_AWAIT_WRITE timeout, use --durations 10 for pytest

self.rebuild_proxies(request, self.proxies)) seems to slow down requests

psf/requests@60ea7f0#diff-c80d84057edadb04b5615f7169c55f88ccda358d7c20af966afb20c0d0d1056a

which results in a pytest timeout, because 5 seconds is not enough on windows

with kwargs.setdefault('proxies', self.proxies)

============================================================================================ slowest 10 durations =============================================================================================
0.86s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128
0.53s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_key_uri_override
0.43s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_with_map
0.36s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content
0.30s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_length
0.30s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_block_length

with kwargs.setdefault('proxies', self.rebuild_proxies(request, self.proxies))

============================================================================================ slowest 10 durations =============================================================================================
10.01s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128
9.80s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_key_uri_override
5.63s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_length
5.22s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_with_map
2.97s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_block_length
2.95s call     tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content

also show the 10 slowest tests on pytest

setup: update requests version >=2.26.0 and makeinstaller.sh

revert #3880

…durations 10 for pytest `self.rebuild_proxies(request, self.proxies))` seems to slow down `requests` psf/requests@60ea7f0#diff-c80d84057edadb04b5615f7169c55f88ccda358d7c20af966afb20c0d0d1056a which results in a pytest timeout, because 5 seconds is not enough on windows ``` with kwargs.setdefault('proxies', self.proxies) ============================================================================================ slowest 10 durations ============================================================================================= 0.86s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128 0.53s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_key_uri_override 0.43s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_with_map 0.36s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content 0.30s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_length 0.30s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_block_length ``` ``` with kwargs.setdefault('proxies', self.rebuild_proxies(request, self.proxies)) ============================================================================================ slowest 10 durations ============================================================================================= 10.01s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128 9.80s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_key_uri_override 5.63s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_length 5.22s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_with_map 2.97s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_block_length 2.95s call tests/streams/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content ``` --- also show the 10 slowest tests on pytest

revert streamlink#3880

bastimeyer · 2021-08-22T10:14:39Z

self.rebuild_proxies(request, self.proxies)) seems to slow down requests

psf/requests#5891
The issue is apparently caused by reading too many environment variables for applying proxy settings on each HTTP request when no proxy settings are set on the session itself.

There's a PR on the requests repo which is supposed to fix the performance regression introduced in 2.26, but there's another issue with side effects of proxy authentication headers, so it's possible that the performance fix won't get merged any time soon from what it looks like.

Even if increasing the value of TIMEOUT_AWAIT_WRITE will fix the tests failing, this technically doesn't fix the underlying issue and we'll have to keep an eye on the issue and bump the requests version again once this gets fixed in a new release.

bastimeyer · 2021-08-22T10:31:37Z

I've re-run the tests a couple of times now and from what it looks like some of the tests on Windows run for 5-6 seconds while most of them run only for 2-3 seconds, which means the tests were barely failing before increasing the timeout, which is unfortunate. Should be good now though. Thanks for the fix.

bastimeyer · 2021-10-07T01:22:10Z

The tests are failing again on Windows (even after increasing the time threshold) because the VMs are running extremely slowly today. I've restarted the CI runners of the latest commit on the master branch a couple of minutes ago and this time it went through, but this is way too annoying. Some of the other commits and today's scheduled build failed and can't be restarted without overriding the docs on success (merging multiple PRs too quickly is btw also an issue with docs deployment).

I'd say we should disable the HLS integration tests on Windows and be done with it.

bastimeyer · 2022-01-23T01:46:33Z

requests 2.27 has been released on the 3rd of January, and it should've fixed the HLS test timeout issues on Windows.

https://github.com/psf/requests/releases/tag/v2.27.0

Fixed performance regression in 2.26.0 for hosts with a large number of
proxies available in the environment.

requests 2.26.0 - Windows - Python 3.10

https://github.com/streamlink/streamlink/runs/4686658266?check_suite_focus=true#step:9:220

============================ slowest 10 durations =============================
3.54s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_key_uri_override
3.49s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_low_latency_has_prefetch_disable_ads_has_preroll
3.33s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_low_latency_no_prefetch_disable_ads_has_preroll
3.29s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_low_latency_has_prefetch_has_preroll
3.24s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128
3.20s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content
3.19s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_with_map
3.07s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_length
2.99s call     tests/test_cli_main.py::TestCLIMainLoggingInfos::test_log_current_versions
2.91s call     tests/stream/test_hls.py::TestHLSStream::test_map

requests 2.27.0 - Windows - Python 3.10

https://github.com/streamlink/streamlink/runs/4696697396?check_suite_focus=true#step:9:220

============================ slowest 10 durations =============================
3.01s call     tests/test_cli_main.py::TestCLIMainLoggingInfos::test_log_current_versions
0.92s call     tests/stream/test_hls_filtered.py::TestFilteredHLSStream::test_filtered_timeout
0.88s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_disable_ads_has_preroll
0.86s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_no_disable_ads_has_preroll
0.70s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_disable_ads_daterange_by_attr
0.70s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_low_latency_has_prefetch_disable_ads_has_preroll
0.67s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128
0.64s call     tests/stream/test_hls.py::TestHLSStreamEncrypted::test_hls_encrypted_aes128_incorrect_padding_content
0.64s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_no_low_latency_has_prefetch
0.62s call     tests/plugins/test_twitch.py::TestTwitchHLSStream::test_hls_low_latency_no_prefetch_disable_ads_has_preroll

back-to added 2 commits August 22, 2021 10:45

setup: update requests version >=2.26.0 and makeinstaller.sh

3ebdc6c

revert streamlink#3880

back-to added Windows installer tests labels Aug 22, 2021

bastimeyer added the dependencies label Aug 22, 2021

bastimeyer merged commit ac47d04 into streamlink:master Aug 22, 2021

back-to deleted the hls_timeout branch September 5, 2021 08:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

tests: increase TIMEOUT_AWAIT_WRITE and update requests #3930

tests: increase TIMEOUT_AWAIT_WRITE and update requests #3930

Uh oh!

back-to commented Aug 22, 2021

Uh oh!

bastimeyer commented Aug 22, 2021

Uh oh!

bastimeyer commented Aug 22, 2021

Uh oh!

bastimeyer commented Oct 7, 2021

Uh oh!

bastimeyer commented Jan 23, 2022

Uh oh!

Uh oh!

Uh oh!

tests: increase TIMEOUT_AWAIT_WRITE and update requests #3930

tests: increase TIMEOUT_AWAIT_WRITE and update requests #3930

Uh oh!

Conversation

back-to commented Aug 22, 2021

tests.mixins.stream_hls: increase TIMEOUT_AWAIT_WRITE timeout, use --durations 10 for pytest

setup: update requests version >=2.26.0 and makeinstaller.sh

Uh oh!

bastimeyer commented Aug 22, 2021

Uh oh!

bastimeyer commented Aug 22, 2021

Uh oh!

bastimeyer commented Oct 7, 2021

Uh oh!

bastimeyer commented Jan 23, 2022

requests 2.26.0 - Windows - Python 3.10

requests 2.27.0 - Windows - Python 3.10

Uh oh!

Uh oh!