Skip to content

Make UploadFile check for future rollover #2962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Make UploadFile check for future rollover #2962

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0aff77a
Make UploadFile check for future rollover
HonakerM Jul 10, 2025
4fcf16e
Fix Linting/MyPy Errors
HonakerM Jul 10, 2025
6fc62ab
Add test for large multi-part file
HonakerM Jul 10, 2025
e3cffca
Fix Coverage Lines
HonakerM Jul 10, 2025
f14ee57
Slightly improve efficency of Size Check
HonakerM Jul 10, 2025
753e790
Update test to actually test the bug
HonakerM Jul 11, 2025
297c09e
Fix test mypy issues
HonakerM Jul 11, 2025
f1fee10
Merge branch 'master' into fix_upload_file_hang
HonakerM Jul 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add test for large multi-part file 
Signed-off-by: Michael Honaker <mchonaker@gmail.com>
  • Loading branch information
@HonakerM
HonakerM committed Jul 10, 2025
commit 6fc62ab696938e35925f99c68bb68671d6f84e9f
13 changes: 12 additions & 1 deletion tests/test_formparsers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@

import os
from contextlib import AbstractContextManager, nullcontext as does_not_raise
from io import BytesIO
from pathlib import Path
from typing import Any

import pytest

from starlette.applications import Starlette
from starlette.datastructures import UploadFile
from starlette.formparsers import MultiPartException, _user_safe_decode
from starlette.formparsers import MultiPartException, MultiPartParser, _user_safe_decode
from starlette.requests import Request
from starlette.responses import JSONResponse
from starlette.routing import Mount
Expand Down Expand Up @@ -303,6 +304,16 @@ def test_multipart_request_mixed_files_and_data(tmpdir: Path, test_client_factor
}


def test_multipart_request_large_file(tmpdir: Path, test_client_factory: TestClientFactory) -> None:
data = BytesIO(b" " * MultiPartParser.spool_max_size * 2)
client = test_client_factory(app)
response = client.post(
"/",
files=[("test_large", data)],
)
assert response.status_code == 200
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test passes on master. We want some condition that proves this PR actually solves the issue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type of issue is hard to verify without profiling or adding breakpoints and checking if a write operation that caused a rollover happened on the main thread. I do not know if it is feasable or even possible to provide a unit test that fails if the blocking rollover happens on the wrong thread, that would require assert statements and additional logic in the actual code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we can remove the test, can't we?

Copy link
Author

@HonakerM HonakerM Jul 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kludex yeah in it's current state it's not doing a whole lot. Let met try editing the test to use a monkey patched SpooledTemporaryFile to ensure the IO thread is separate from the event thread

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not ensure that there is no blocking call on the event loop, but it does cover the new code paths that would otherwise not be covered.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kludex Just pushed the new test! I ended up creating a new class/app to track where rollover was happening. I'm still new to starlette development so please let me know if you have any code-quality suggestions.



def test_multipart_request_with_charset_for_filename(tmpdir: Path, test_client_factory: TestClientFactory) -> None:
client = test_client_factory(app)
response = client.post(
Expand Down
Loading