Skip to content

feat(markers): support 'extras' and 'dependency_groups' markers #888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 17, 2025

Conversation

frostming
Copy link
Contributor

Signed-off-by: Frost Ming me@frostming.com

Resolve #885

Signed-off-by: Frost Ming <me@frostming.com>
Signed-off-by: Frost Ming <me@frostming.com>
Signed-off-by: Frost Ming <me@frostming.com>
@brettcannon brettcannon self-requested a review April 3, 2025 20:24
@brettcannon brettcannon requested a review from pradyunsg April 3, 2025 20:26
@brettcannon
Copy link
Member

The PR LGTM (and thanks, @frostming !), but since @pradyunsg wrote most of the code being changed I want to give him a chance to do a review if he has the time. If not I will merge this in about a week.

@@ -308,7 +318,7 @@ def evaluate(self, environment: dict[str, str] | None = None) -> bool:
The environment is determined from the current Python process.
"""
current_environment = cast("dict[str, str]", default_environment())
current_environment["extra"] = ""
current_environment.update(extra="", extras=set(), dependency_groups=set())
Copy link
Member

@pradyunsg pradyunsg Apr 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not be setting these in the evaluation environments unconditionally, since that would allow packages to use the extras and dependency_groups marker variables in regular dependency declarations (outside of the context of a lockfile).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would make it ~impossible for downstream tools to implement the following behaviour from PEP 751's specification update:

Outside of the context of lock files, these two variables should result in an error like all other unknown variables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to add an explicit test for the unknown variable error case that this pertains to.

Copy link
Contributor Author

@frostming frostming Apr 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside of the context of lock files, these two variables should result in an error like all other unknown variables.

But other unknown markers are thrown as InvalidMarker errors during the parse phase. Should we, and can we keep consistent for extras? That may require a different version of parsing function, possibly with a flag

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it a "should" statement for a reason. We could add a flag to evaluate() to specify the context and also clean things up for extra while we are at it, otherwise there's precedent of this code being sloppy/permissive in what it accepts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brettcannon thanks for the clarification, not sure if I interpret your feedback correctly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I was thinking was instead of a for_lock argument, we have a context argument. That would take an enum that specified under what context the marker was being evaluated. We could have metadata which matches what it does now where extra is always set (and would be the default). We could have lock_file which doesn't have extra but has extras and dependency_groups always set. And then we could have requirement which has none of extra, extras, or dependency_groups automatically set.

FYI I'm not attached to any of these names.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pradyunsg does the approach Frost took work for you (it does for me)?

Signed-off-by: Frost Ming <me@frostming.com>
Signed-off-by: Frost Ming <me@frostming.com>
@frostming frostming force-pushed the feat/pep751-markers branch 2 times, most recently from 8fbf371 to 28a9c05 Compare April 9, 2025 01:31
Signed-off-by: Frost Ming <me@frostming.com>
@frostming frostming force-pushed the feat/pep751-markers branch from 28a9c05 to 6f37ebf Compare April 9, 2025 02:00
Copy link
Member

@brettcannon brettcannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a docstring change and docs will need updating.

frostming and others added 2 commits April 15, 2025 08:40
Co-authored-by: Brett Cannon <brett@python.org>
Signed-off-by: Frost Ming <me@frostming.com>
Copy link
Member

@brettcannon brettcannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think

.. method:: evaluate(environment=None)
Evaluate the marker given the context of the current Python process.
:param dict environment: A dictionary containing keys and values to
override the detected environment.
:raises: UndefinedComparison: If the marker uses a comparison on strings
which are not valid versions per the
:ref:`specification of version specifiers
<pypug:version-specifiers>`.
:raises: UndefinedEnvironmentName: If the marker accesses a value that
isn't present inside of the environment
dictionary.
:rtype: bool
needs an update and then if @pradyunsg wants to say if he's good with the method signature change (I am). Otherwise LGTM!

@brettcannon
Copy link
Member

FYI linting failed.

frostming and others added 2 commits April 16, 2025 08:26
Signed-off-by: Frost Ming <me@frostming.com>
@brettcannon brettcannon self-requested a review April 16, 2025 17:48
@brettcannon
Copy link
Member

LGTM! We will give @pradyunsg until next week to weigh in, else I will merge this.

Copy link
Member

@pradyunsg pradyunsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do this! Thanks @frostming!

@brettcannon brettcannon merged commit 3910129 into pypa:main Apr 17, 2025
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support PEP 751 in markers
3 participants