1

Hi all.

I am excited to share my proposal for replacing (over usual deprecation periods) the Windows installers (plural) with a single tool for downloading, installing, updating, and managing installs. This preserves our current preference for encouraging side-by-side installs of multiple versions, while also solving/avoiding a significant number of limitations due to the underlying installer technology.

I hope the PEP speaks for itself, so I’ll say no more here. I do have a prototype implementation, though it’s a little bit behind the PEP text right now so I want to bring it up to date before sharing another build (it’s been shared before in the Ideas thread[1]). I’ll update this post when I have one ready.

Current formatted text: https://peps.python.org/pep-0773/
Current PEP PR: https://github.com/python/peps/pull/4249
Current demo build: https://github.com/zooba/pymanager/releases/tag/0.1a7 (~20MB)
Demo video (for 0.1a4):

Looking forward to hearing what you all think!

Edit: Updated the number, seems I just missed 772. Hope it’s a good one though! This is now 773.
Edit (23Jan2025): Updated demo build link and video

  1. If you were part of that, all the controversial stuff has been changed, so read the text again. ↩︎

19 Likes
2

I think the new installer’s a fantastic addition, and there’s loads of great work gone into the PEP.
But I strongly oppose replacing and deprecation of the traditional .exe installers (e.g).

I didn’t notice any mention of taking over the whole install process on Windows in the initial discussions. The scope was limited to users that have chosen to install Python from the Windows Store.

6 Likes
3

Can you explain why? What do you need from the old installers that you don’t get from the new ones?

2 Likes
4

Getting rid of the existing standard way to install on Windows, is a significant breaking change.

With the traditional installers, different versions play well with each other, I can see exactly what version I’m getting, avoid installing optional extras, but most importantly, I don’t have to navigate a screen full of flashing ads from the Windows Store. Nor do I have to have it track what software I’ve installed. No app store should own the standard installation route for Python on an OS (even one run by the producer of that OS).

But that’s a reasonable point. Is there an example of such a new installer available to download elsewhere yet to test? If so, I’m more than happy to test it. If I find it works just as well, I won’t be opposed. But until the wider community has experience of that, deprecation and replacement of existing tried and tested installers should not be on the table.

2 Likes
5

I am a bit surprised about this part of the PEP:

Will this mean that installation of Python via e.g. chocolatey or one of the other OSS devops tools for Windows will stop working ? They are using the Nuget packages for installation.

FWIW, I don’t really care much about the Windows Store, since I generally try to avoid using it whenever possible. While I do see advantages of using it to improve the user experience for novice users, I’d very much dislike having us force all users down that path on Windows.

8 Likes
6

It’s a change, yes. But for manual use, people can learn the new approach. For automated use, this is covered (to an extent) in the PEP. I don’t know how much people automate the old installers, so I’ll let Steve comment on that. And every time the installers have changed in the past (which they have) it’s happened without a PEP, and no-one has had much of a problem. This is a bigger change - hence the PEP - but updating the installers isn’t completely unknown territory.

The new installer installs the manager. You then have full control over what versions you install from the manager.

The manager will be available as a download from python.org as well. It will be a .msix file that you double click, just like you do with the existing installers. By not using the store, you avoid all of the issues you mention, although you lose the benefit of automatic updates, so you’ll need to update the manager manually. You won’t need to update the installer every time a new Python version is released, though - your existing manager will pick up new Python releases automatically, and you can install them with py install 3.16 or whatever.

See Steve’s original post. It’s being updated based on feedback in the python-ideas thread, and should be available soon. I tested an older version, and in my opinion, it was really easy to use.

I don’t have Steve’s experience supporting the old installers, but from what the PEP says, they are far from “tried and tested” - they work well for most users, but there are plenty of people who have had issues with them (and because Microsoft aren’t developing the MSI technology any more, we aren’t going to get such issues fixed :slightly_frowning_face:)

I think the PEP should make it much clearer that we will still publish an installer on python.org. It will be a .msix file, and will be functionally identical to the one provided in the Windows store, but you won’t need to access the store to use it.

I had a similar discussion offline with Steve, so I know this is the intention. But it’s obviously not made clear enough in the PEP.

Personally, I’d prefer the install off the python.org website. But the attraction of having to install the manager once, and then get it updated automatically, while still having full control over what Python versions I have installed via py install/uninstall is sufficient that I will probably put up with using the store for that one-time install…

5 Likes
7

Wouldn’t the Python installation created by this new MSIX installer then have all the same problems the PEP mentions as affecting the Windows Store package, e.g. slower startup and all sorts of sandboxing issues?

8

Thanks for the in depth reply and further clarification. I appreciate the new transparency, from this going through the PEP process (unlike before).

Is this the latest test build, Steve?

It’s not a biggie for me personally (I’m adopting uv). So this is a hill that I will trust yours and Steve’s judgement on and vacate, rather than die on.

9

No. What gets installed is the Python manager. That is what ends up in the “protected” area where the OS installs things from .msix packages. The manager is a dedicated application which doesn’t suffer from the problems the Store Python had. When the manager installs a Python interpreter, it puts that in a normal directory owned by the user, just the same as the current installer does for a per-user install[1]. So the interpreter works exactly like a non-Store installation does currently.

As a side issue, there’s some fancy footwork so that when the user installs the manager and invokes the python command for the first time, that causes the manager to install the latest version of Python and run it. So it looks like you just installed Python. That keeps the user experience the same - I install Python from the store or from python.org, and then run the python command and the interpreter starts - while still keeping the manager and the interpreter separated, and ensuring that only the manager is in the weird “Apps directory” that the OS manages.

This is all a bit tricky to understand - it took me ages to work out what was going on. And Steve is probably too familiar with the technology to realise how confusing it is to the average developer/user (although I’ve done my best to make it obvious, by endlessly asking dumb questions!). But if anything is unclear, please ask - either Steve or I should be able to explain. And asking questions now means Steve can improve the explanations in the PEP.

  1. it uses a different directory name, but that isn’t particularly relevant here ↩︎

3 Likes
10

It is, but it’s well behind the current PEP, so you’ll probably get angry about stuff that’s already been changed but hasn’t been released as a test build yet :wink:

You can expect to find up to date test builds here in the original post, where there is currently a placeholder saying to wait for an up to date test build.

As Paul says, no, but the logic to get there isn’t necessarily direct and it links through a few design decisions (such as not bundling a ready-to-run runtime, but only a ready-to-extract one). Solving these issues is one of the reasons I’m proposal a replacement.

If they’re relying on NuGet, I’m not totally opposed to keeping those for longer, as they’re pretty easy to maintain (package signing issues aside). But the ones that are currently downloading and running our old installer will have to be updated, yes.

None of them “stop working”, because the old versions that they currently reference will still exist. And in general with installer changes, nothing ever “stops working”, because nobody is using an installer that hasn’t been released yet, and nobody gets an automatic or forced update to an installer. By definition, users are entering into incompatibility by doing the thing that led to them getting the new installer - moving from 3.13 to 3.14 does not “stop working”, it forces you to update how you do some things to adopt the new version. That’s all that happens here. Suggesting that anything will “stop working” is FUD.

1 Like
11

I honestly don’t know how to be clearer than I currently am:

PyManager is the internal name of our proposed replacement installer tool. It will be distributed both in the Windows Store and on python.org as an MSIX package.

I’m thinking that anyone who believes this won’t be installable from python.org has just failed the reading comprehension test and needs to go back to the PEP before they start discussing it :upside_down_face:

1 Like
12

By the way, @steve.dower - I was going to say that people really should try the new manager once you have the updated version out. For me, it was a very smooth experience - it worked just fine, and I could install it, use it and then uninstall it without any issues.

But I use a very default installation of the python.org installers. I don’t add python to my path, I don’t use the store Python, etc. My understanding is that if people do add python to their path, they will probably need to remove it, otherwise they won’t see the app redirector (and won’t be using the new install). The same is true of the py launcher, although in my testing I was using a version of the new manager before it added the py command.

I suggest that if you want people to try out the new installer, it might be worth documenting some details on how to do so, while not affecting their current install too much. Those docs will be needed when the PEP gets accepted anyway, as part of the “how to use the new manager” documentation, so it won’t be wasted effort.

5 Likes
13

Great work @steve.dower . I’ll be sure to test on my fleet (several business computers, hah) when the new build is available.

This brings harmony to the Windows experience. No longer will I fret, “store Python or Python.org installer?” One and the same :slight_smile:

1 Like
14

It is possible that choco install python will “stop working” for the task of “installing the latest Python version”, if the new distribution method was incompatible with Chocolatey. Last time I used Chocolatey (admittedly, long ago), it was fine with all sorts of random things, so I doubt there would be issues, but still — I think it would be good to assess the changes to user experience for users of Chocolatey, WinGet, et al. as part of the PEP.

I would be very surprised if it worked in Docker-on-Windows (and that is used in some CI systems, ask me how I know.)

(PS. the PEP needs a s/Nuget/NuGet/g.)

Additionally, the PATH environment variable cannot be intelligently modified - at best, we can prepend or append the install path. This usually results in the most recent install of Python being the highest priority. For example, if the user has Python 3.14 installed and then installs (or updates) 3.13, the python command will switch from the later version to the earlier version.

In theory, something intelligent could be achieved with a custom action (and a script or program would handle smartness). But that’s quite hard to get right with Windows Installer’s various install types.

15

That’s partly why I’m having the discussion, but it’s also not a good use of my time to go off and learn every possible install runner out there just to find out whether they can install tools or not.

Chocolatey’s recipe for Python hard codes the installer name, which means they need a manual step to update it which means they have an opportunity to change how it works before users get broken. That’s the main reason I can say it won’t stop working. New releases after the old installer is removed may just be deferred a little bit (though with a bit of luck I can get deprecation messages printed from the old installer so they/their users notice sooner).

The same thing probably applies to Winget, etc. and anyone else who has written their own install script. Given the choice between producing a testable prototype and writing documentation, or figuring out the changes needed to other people’s wrappers, I’m spending my time on the prototype (and then the docs).

Yeah, this is one that I probably do ought to spend the time figuring out. I don’t actually know how Docker handles any kind of Windows-wide level changes. It seems obvious that py install --target <mounted location> <version> is a better way to install into the container anyway, but I bet people will manage to get stuck trying to put the manager inside rather than just the runtime.

Yes indeed! Thank you for the support/agreement on this one. It’s not entirely impossible that we could end up back at an MSI with big custom steps rather than an MSIX, but I do want to avoid it, given how so many of the install failures relate to this kind of thing.

1 Like
16

As long as there is some form of standard and reasonably documented way to install without involving PyManager (e.g. classic MSIs, or ZIPs with a predictable URL that can be extracted anywhere and have their bin folder added to PATH), I think most tooling will be fine.

Typical Win32 management actions (like installing a .msi) generally work fine within a container. Some atypical Win32 management actions (like installing IIS/other Windows features) work too. But at the same time, the appx/msix infrastructure might be missing, since it’s a UWP thing (I can’t check at the moment).

Not all CI platforms give users enough control to manage mount points or install stuff on the container host. Or the container host might be incompatible with appx/msix as well (as it’s some form of container/slimmed down VM).

1 Like
17

The index described in the PEP allows 3rd party tools to locate and download unzip-and-run archives of any Python version you want. That should be sufficient for anyone who doesn’t want to run the manager. Honestly, I think this is a big benefit of the proposal which is quite undersold by the PEP - in my experience a lot of people want “unzip and run” versions of Python (they often misuse the embedded distribution for that purpose), and this proposal gives them that.

5 Likes
18

This is also what I came here to ask. I think MSIX does not work in containers, either at all or just nano server, I can’t remember.

19

At one point, MSIX was the only thing supported by NanoServer, because they didn’t want to port MSI over to it (but did want Python running there, which is how I was involved - this is well before Windows Containers were a thing).

Do you or @Kwpolska have any workflows I can take a look at? Right now it all sounds like “hypothetically things may not be supported”, but rather than reduce the average user’s experience to the lowest common denominator preemptively, I’d like to see what kind of tooling may be required to fit into the container workflows where people (presumably?) are downloading and running the traditional installer.

1 Like
20

Also, before we downgrade the normal user experience, could we consider the possibility that all that’s needed in a CI context might be a simple utility to read the official index and download and unpack the appropriate distribution?

To put it another way, do CI environments need the manager if they have easy access to a portable install of Python?