
Windows 10 has “python” command that redirects to the Windows Store - _bxg1
https://twitter.com/CasualEffects/status/1256806906015907841
======
deanCommie
I'm a developer who's never had to ship serious production systems with
Python, but I have had to modify the occasional library or open source
package. i.e. I know just enough to get myself into trouble...

I develop on a Mac where in theory everything should be easier:

* Comes with Python out of the box

* Platform has ubiquitous and wonderful package manager (Homewbrew)

And yet, 90% of my interactions with Python via the CLI involve it crashing
because the version that the current thing I'm trying to compile or execute
wants isn't the one the OS happens to be vending for it.

Some OSS projects suggest using things like pyenv to help. I don't know, it
just seems to make things even worse for me because now even 'which python'
doesn't tell me anything because it directs to some shell script shims that I
can't parse or understand.

Python the language is wonderful, as are the libraries that exist for it.

Python the runtime/execution environment is actively hostile to users, and
it's honestly keeping me from actually building anything real with it from the
ground up.

I'm not surprised that Microsoft went with this approach to TRY to help it's
users.

I'm also not surprised that Python devs are pissed because I've also been in
those shoes where FINALLY everything is working, and you think you have that
one version of python 2 and one version of python 3 both running and
coexisting and everything you need to run runs, and then something small
changes and WHAM the whole house of cards collapses.

But I wouldn't be angry at Microsoft, I'd be angry at the Python community for
letting things get so bad.

~~~
_bxg1
> And yet, 90% of my interactions with Python via the CLI involve it crashing
> because the version that the current thing I'm trying to compile or execute
> wants isn't the one the OS happens to be vending for it.

Do python releases introduce backwards-incompatible changes (ignoring the
2.x/3.x divide)? Would it not be as simple has making sure you have the latest
runtime?

~~~
jcrawfordor
This issue is largely OS X related rather than Python-related per se. OS X
ships with vendored copies of many common Python libraries which you cannot
update, and so if you need to use newer libraries you have to install them in
some other location and fiddle with the path to make sure your versions are
loaded before the OS X versions. Because the versions included in OS X
(including the version of the Python runtime) are pretty old almost everyone
ends up doing this.

In theory homebrew takes care of the whole thing for you but, in practice...
as an only part-time OS X user I have run into a lot of problems with the path
getting set in various ways that end up in various copies of the Python
interpreter or library paths getting loaded, resulting in runtime errors due
to library versions being inconsistent with the interpreter version or
expected libraries not being loaded...

A further complication is that, the specific way that Apple vendored a set of
python packages, it is possible but not particularly easy to add new packages
to the OS-provided Python runtime, so most users, if they want some package
that Apple didn't include, seem to end up installing a whole new environment
just to get that package, even if the OS-provided environment were otherwise
acceptable to them.

It's a very real problem and very frustrating, but I'm not quite willing to
blame it on the Python ecosystem because it seems to be pretty much a result
of Apple's decision to ship an interpreter and set of libraries that most
users will end up being unhappy with. Anecdotally at least I run into these
issues a lot less on Linux because even with fairly stable distributions (e.g.
RHEL7.5/8) the included interpreter and library are more recent and the
package repositories include most of the Python packages you would want,
maintained to match the OS interpreter version. So while some people do end up
using virtual environments it's less necessary than it is on OS X, where it
frankly seems near impossible to do anything without installing a whole second
Python environment and munging paths.

An extra complication is that, as a casual OS X user, it seems to me like OS X
terminal emulators and tools are more likely to munge with the path than Linux
terminal emulators and tools, so even if you get everything set up "just
right" you end up having it not work properly in certain contexts like plugins
because something has modified the path for some reason.

Edited to expand that, I guess what I mean to say but failed to articulate, is
that the fundamental problem here (in my opinion) is that OS X ships with a
Python distribution but _not_ with a package manager. That's the fundamental
difference from Linux where these problems are much less common. Third-party
package managers like Homebrew and Macports are used, and in order to have
full control of the Python environment they have to install a whole second
copy because they are not allowed to tamper with the OS X Python environment.
This pretty inevitably leads to frustration. The best solution would seem to
be for Apple to provide some kind of "first-party-ish" package management
functionality, perhaps something like Chocalatey which is not a Microsoft
product but is built on Windows-provided primitives, so that the package
managers that users are actually using can play nicely with the OS X Python
environment.

~~~
pvg
_An extra complication is that, as a casual OS X user, it seems to me like OS
X terminal emulators and tools are more likely to munge with the path_

This might be a function of the casualness rather than the tools. Here's a
thing to check out on OS X:

    
    
        man path_helper 
    

I didn't know about this for ages and it makes your path life much saner - at
least you'll have some idea where seemingly magical paths come from.
Personally, I use it for everything and don't let other profile scripts eff
with the global path at all.

~~~
tptacek
You set up your personal shell paths by adding files to /etc/paths.d/?

~~~
pvg
With neither shame nor ruth. Is there some non-aesthetic reason not to?

~~~
tptacek
No! Well other than not having your whole shell environment in your dotfiles.
It just wouldn’t have occurred to me.

------
Lendal
It takes a lot of IT knowledge to get a Python script to work properly on
Windows. And yet somehow you figured out a way to blast Microsoft for trying
in good faith to help out the Python community. And it was done in concert
with the Python community, so it's definitely not "fake".

This makes me sad.

~~~
zelly
Agreed. On the beloved GNU/Linux distros, when you "$ python" and it's not
installed, Package Kit will recommend you to install it through the native
package manager (like apt or dnf). This is literally the same thing. Windows
Store is the package manager for Windows.

~~~
gizmo686
The difference is implementation. GNU/Linux systems do not hijack the 'python'
executable. If such an executable is on the path, they will use it. If Python
is not installed a program can check the path and see that Python is not
installed.

It is only when the user tries to run python on the terminal that the system
(I presume the shell) detects that the program does not exist, and it then
executes another program to inform the user.

If you put python as the #! interperator for a script, then bash will complain
that interperator cannot be found.

If it is used as a command as part of a script, then bash does not call the
normal command-not-found program (which advises you to install it), but just
prints to stderr that the command was not found.

~~~
WorldMaker
Windows doesn't "hijack" the python executable either. This stub _should_ be a
low priority in your PATH and the official Python installers _should_ take
higher priority when they install. It's basic PATH management, with much the
same environment variable management issues on Linux as it is on Windows.

It's unfortunate that so many machines that this poster is working with are
getting malformed PATH variables with the priorities out of order, but it
should be a simple fix to the PATH to correct.

It should also probably be noted that the Microsoft/Windows Store installed
version of Python _is_ the official Python.org release. (It's CIed, and CDed
directly from Python.org just as with any other release binary.)

~~~
gizmo686
Even with correct PATH management, Windows still hijacks the command when
python is not installed; that it the mechanism it uses to advise you to
install it. Linux distributions use a different mechanism that maintain the
expected behaviour of there not being any program/comnand called "python" when
python is not installed.

Windows solution in this case is a lazy hack that makes the system more
brittle.

As an aside, Linux distributions tend not to have this sort of PATH management
issue because (for better or worse) they tend not to modify PATH when
installing software.

------
oefrha
There's an actual term for this "fake" command: it's a stub command. Stub
commands are by no means a new concept; for instance, on macOS many CLT
commands are stubs (that prompt CLT installation), and IIRC the java family of
commands in /usr/bin are stubs out of the box, too.

The real problem, or the claimed problem, is

> Windows 10 puts a fake python/3 executable in the path, _ahead of the actual
> python_.

(Emphasis mine.)

Somehow I'm on the slow ring and I'm not affected by this; the stub is
definitely not ahead of the actual python. Why? Because C:\Python38\Scripts
and C:\Python38 are way ahead of %LocalAppData%\Microsoft\WindowsApps (where
the python.exe stub lives) on my PATH. When you install things naturally they
prepend themselves to the path, CPython (or any python) installers included.
Begs the question: what kind of geniuses put third-party paths after system
paths, then bitch about breakage?

Edit: Upon further inspection of some complaints, I think what Microsoft
should do is fixing then teaching people proper PATH management on Windows
(which is terrible), since apparently all the top google-surfaced tutorials
ask you to _append_ to PATH, which opens you up to this kind of self-inflicted
problems. (Clicking on "New" when editing PATH creates a new entry at the end;
obviously doesn't help.)

~~~
JadeNB
CLT = Command-Line Tools, for those unfamiliar. (Unfortunately, I can't find
an official link that's not behind a developer.apple.com sign-in.)

~~~
oefrha
[https://developer.apple.com/library/archive/technotes/tn2339...](https://developer.apple.com/library/archive/technotes/tn2339/_index.html)

------
ninjacatex
This really irked me when I found out because I spent longer than I care to
admit diagnosing why a script I wrote seemingly did nothing.

I'm not sure if it's different now but when I had just installed anaconda
python, and ran a script the usual way (`python foo.py`) it just did nothing
(not even open the store page).

My memory might be a little hazy but I think not even changing PATH order of
precedence made a difference. I had to find a sub-sub page in the modern
settings app to disable the magic store hook.

I agree with some others here: I'm fine if I'm suggested a solution to a
missing command but when that hook overrides a valid installation I have a bit
of a problem :|

------
greggman3
This actually points out what I think is a problem.

The Windows Store is no longer only sandboxed apps.

I wanted to install the Line App (I live in Japan and everyone uses Line and I
hate typing on my phone). I have both a Mac and a PC. On Mac I go to the Mac
App Store and install Line. I know (believe?) that the app is sandboxed and
short of OS bugs can't spy on me.

I thought Windows Store was the same but then I saw Python on there and was
like "wait a minute, that can't possibly be sandboxed on Windows". So I
investigate and ... yea, many apps are not sandboxed. If you look at the Line
App (or WhatsApp or Python or FB Messenger) or many others they list

> Access all your files, peripheral devices, apps, programs, and registry

Which if you dig further Microsoft says

> Access all your files, peripheral devices, apps, programs, and registry: The
> app has the ability to read or write to all your files (including documents,
> pictures, and music) and registry settings, which allows the app to make
> changes to your computer and settings. It can use any peripheral devices
> that are either attached or part of your device (such as cameras,
> microphones, or printers) without notifying you. It also has access to your
> location, and can use platform features, such as location history, app
> diagnostics, and more, which are denied to most Store apps. You can't
> control most of the permissions for this app in Settings > Privacy.

------
withinrafael
This is a half-baked idea by Microsoft, one many of us in the Windows
community tried to warn them about last year [1][2] with no apparent success.
(One big problem is that build systems/various scripts tend to look for a
callable python/python3 to determine if python is installed. Calling out to
this stub eats input yet outputs nothing, leading to lots of confusion.)

You can turn all Python aliases off by navigating to Settings > Apps > Apps &
Features > App execution aliases. Alternatively you can Start search for
"aliases".

[1]
[https://twitter.com/WithinRafael/status/1171339229583962112](https://twitter.com/WithinRafael/status/1171339229583962112)

[2]
[https://twitter.com/BruceDawson0xB/status/117089849987067494...](https://twitter.com/BruceDawson0xB/status/1170898499870674944)

~~~
pjmlp
Done in collaboration with the Python community.

~~~
withinrafael
Is that true? Is there some sort of call for comments from the Python
community on this? (Genuine question!)

~~~
pjmlp
Yes, it was even presented at PyCon.

[https://devblogs.microsoft.com/python/come-meet-microsoft-
at...](https://devblogs.microsoft.com/python/come-meet-microsoft-at-
pycon-2019/)

"Steve Dower - Python on Windows is Okay, Actually - PyCon 2019"

[https://www.youtube.com/watch?v=uoI57uMdDD4](https://www.youtube.com/watch?v=uoI57uMdDD4)

And is part of official Python documentation, as mentioned by others on this
thread.

~~~
WorldMaker
It should also probably be noted that the Microsoft/Windows Store installed
version of Python _is_ the official Python.org release. (It's CIed, and CDed
directly from Python.org just as with any other release binary.)

------
nojito
the Windows Store is the official way to install Python on windows.

[https://docs.python.org/3.7/using/windows.html#windows-
store](https://docs.python.org/3.7/using/windows.html#windows-store)

Original Blog post from last year

[https://devblogs.microsoft.com/python/python-in-the-
windows-...](https://devblogs.microsoft.com/python/python-in-the-
windows-10-may-2019-update/)

~~~
nathanaldensr
Really? That seems to be the "official" _Microsoft_ way of installing Python3.
For those who'd rather go straight to the source (should be everyone), try
this, instead:

[https://www.python.org/ftp/python/3.8.2/python-3.8.2.exe](https://www.python.org/ftp/python/3.8.2/python-3.8.2.exe)

~~~
nojito
No?

[https://docs.python.org/3.7/using/windows.html#windows-
store](https://docs.python.org/3.7/using/windows.html#windows-store)

>The Microsoft Store package is a simple installation of Python that is
suitable for running scripts and packages, and using IDLE or other development
environments. It requires Windows 10, but can be safely installed without
corrupting other programs. It also provides many convenient commands for
launching Python and its tools.

~~~
enitihas
It is just "one" method of installation, and not even mentioned at the top of
the page. The page also says

> Note The Microsoft Store package is currently considered unstable while its
> interactions with other tools and other copies of Python are evaluated.
> While Python itself is stable, this installation method may change its
> behavior and capabilities during Python 3.7 releases.

It is "an" official method, not the preferred official method, and most
certainly not "the" official method.

~~~
qilo
That note was for Python 3.7, as of Python 3.8 the note is gone.

[https://docs.python.org/3.8/using/windows.html#windows-
store](https://docs.python.org/3.8/using/windows.html#windows-store)

------
2OEH8eoCRo0
So what? Some Linux distros will suggest installing a missing program if you
run it via terminal and it's not installed.

~~~
detaro
> _and it 's not installed._

is the key difference here, as the submitted tweets clearly point out.

------
D13Fd
Hard to imagine that this is some nefarious plot when Microsoft also provides
one of the most popular Python editors (Visual Studio Code) and an excellent
Python extension for it, and makes appearances at Python conferences etc. It
seems more like a garden variety screwup.

~~~
scarface74
And Python is supported as a first class citizen in Visual Studio.

------
xg15
This is part of a strange pattern of Windows aliasing popular Linux commands
to things that do something completely different.

There is also "bash", which launches the WSL and "curl", which aliases to a
powershell command with completely different arguments.

------
hutattedonmyarm
I ranted about this to my group of friends a while ago. I usually explicitely
state `python3` when running scripts from the command line in linux and macOS.
On my Windows machine I recently installed python for a mini-project and
forgot to create a symlink so I could use python3 as a command. I then
launched a script (which was working perfectly fine when run via VSCode) from
CMD with `python3` and nothing happened. It took me quite a while to figure
out, that python3 invoked the stub comman, which did nothing at all when
called with a script as parameter. No error, no warning, the store page didn't
open. Just empty output

------
nathanaldensr
Yep, it's real. Nice one, MS. :eyeroll:

    
    
      Directory of C:\Users\user\AppData\Local\Microsoft\WindowsApps
    
      01/15/2020  04:23 PM                 0 python.exe
                     1 File(s)              0 bytes
    
      Directory of C:\Users\user\AppData\Local\Microsoft\WindowsApps\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe
    
      01/15/2020  04:23 PM                 0 python.exe
                     1 File(s)              0 bytes

------
rdtsc
This made think of one of my favorite Pycon talks by David Beazley called
"Discovering Python":

[https://www.youtube.com/watch?v=RZ4Sn-Y7AP8](https://www.youtube.com/watch?v=RZ4Sn-Y7AP8)

The premise is David got locked in a vault with terabytes of tangled source
code on CD-ROMs, no network connection, no ability to bring software in or
out, and just a bare Windows machine, and and the story goes from there...

------
amanzi
Short of including a full Python installation in the base OS, what Microsoft
is doing is a great idea. However, it seems like in some cases your PATH might
get in the wrong order which can cause scripts to fail. I haven't encountered
this personally and I've used Python on several different Windows 10 machines,
each with the full version of Python installed from the website. Bugs happen,
and path issues are easily fixed.

------
pixelmonkey
This would be a violation of the advice in PEP 394, except that PEP 394 only
applies “to Unix-Like systems”.

[https://www.python.org/dev/peps/pep-0394/](https://www.python.org/dev/peps/pep-0394/)

Basically, the Python core team suggests that the “python” executable/command
be set to one of the following:

\- python2,

\- python3,

\- not provide python command,

\- allow python to be configurable by an end user or a system administrator

It would have been much better, IMO, for Windows to do nothing here. A
Microsoft-blessed Python installer from the Windows Store could actually serve
a positive purpose distinct from the official installer at python.org. That
would be to support python2 and python3 side-by-side cleanly.

It might nicely install Python 2.7.18 at the python2 executable, and install
Python 3.8.2, as well, and have python/python3 refer to 3.8.2 and beyond.

This would match an Ubuntu 20.04 system with python2 installed, and with
python-is-python3 installed, which is, IMO, how a system should be configured
if it needs to have python2 around for backwards compatibility, but wants the
“system/default python” to be python3. And it’d make sense for Microsoft to do
that since Windows, above all, is a system with a deep respect for backward
compatibility with old tools. But, that isn't what they did.

Instead, this stub forwarding to the Windows store seems like the worst of all
worlds. I am not really sure what problem they are trying to solve and for
whom. If you are the kind of Windows user who opens cmd.exe and manually types
in “python” into the black terminal window, are you the kind of Windows user
who won’t be able to figure out how to open a browser to the official
installer on python.org? Doubt it. Meanwhile, the stub likely breaks existing
code, even code that may just be running “python -V” to see what version is
installed at that path.

And apparently the Microsoft-blessed Python installer is just a python3
install (perhaps old), with no thought to python2 and backwards compatibility
for old python scripts for Windows? Hmph. This doesn’t seem like an
improvement over “do nothing”.

------
netsharc
Didn't OS X do this with some CLIs, you'd type a command and got prompted to
download Xcode..

~~~
lann
Sounds familiar, but Xcode does actually provide a bunch of CLI tools...

~~~
_fullpint
I think it’s when you have to download the Xcode CLI tools that many packages
rely on. But you do it once and you never do it again.

Definitely not equivalent situations.

~~~
anaisbetts
The only difference is that it's more finely grained? I don't see how it's
completely different

------
dstaley
I'm fairly sure that if you install Python using the official installer and
select the "Add to PATH" option that the actual executable will be used. I'll
check later in a clean VM to confirm and update this comment.

~~~
anaisbetts
The problem is that MS's python can end up taking path precedence even if you
install via the installer, I've hit this before. It's Just A Bug, one that I'm
sure that Microsoft would be very happy to fix if folx complained to the right
people. The stub python.exe could very easily search PATH to see if it's
getting put first and delegate, or the official Python installer could
recognize and fix this

------
jug
Why not just include python with Windows? They could take the opportunity to
update it with their biannual Windows updates. Otherwise not offer any support
for it if they wouldn’t like to.

~~~
whywhywhywhy
Which version of Python would you bundle? The community can't even decide on a
runtime.

This sort of mess caused Apple to ship Python 2.7 with OSX for 10 years and
will stop bundling it at all in future [1]

[1]
[https://developer.apple.com/documentation/macos_release_note...](https://developer.apple.com/documentation/macos_release_notes/macos_catalina_10_15_release_notes)
(search for python on this page)

------
ineedasername
Once you install python, is this redirection removed? If so is it only removed
if you install from the Microsoft store, or also from a standard install?

~~~
WorldMaker
The redirection is done by a low-priority PATH. Both the Store install and the
"standard" install (both of which are built/maintained/managed by Python.org,
fwiw) should install at a higher priority PATH. PATH bugs are just hard to
entirely avoid, but PATH management is relatively easy.

(One big for instance is that System PATH and User PATH are concatenated and
that the combined PATH when it exceeds a certain character limit sometimes
causes subtle bugs like this one. People unfamiliar with Windows-specific PATH
issues might not realize that issue.)

~~~
tssva
This is not 100% accurate. The "standard" install will place the installation
directory higher in the PATH priority.

The store version doesn't modify the path. Instead it creates app execution
aliases for the executables it installs. As of version 3.8.2 it creates the
following app execution aliases: idle.exe, idle3.8.exe, idle3.exe, pip.exe,
pip3.8.exe, pip.exe, python.exe, python3.8.exe, python3.exe, pythonw.exe,
pythonw3.8.exe, and pythonw3.exe. These can be found in the
%LOCALAPPDATA%\Microsoft\WindowsApps directory. Prior to installation the
python.exe and python3.exe app execution aliases in this directory will launch
the Microsoft Store. After installation of the "store" python they will
actually launch python.

------
carlsborg
Python needed a major technology vendor with political will backing it.

------
agpl
Well, Microsoft wants to train open source developers to use the Windows
store.

The whole install gets more and more complex. I've never had a problem when
Martin von Loewis was still in charge of the installer.

~~~
WorldMaker
The Python team themselves were just as interested in being in the Microsoft
Store. Python prides itself on trying to be an easy to install language
accessible to a wide variety of users.

It should also probably be noted that the Microsoft/Windows Store installed
version of Python _is_ the official Python.org release. (It's CIed, and CDed
directly from Python.org just as with any other release binary.)

------
Havoc
MS is doing a good job of making me move the *nix lately. I can't even _open_
Windows store anymore, let alone check out their python stunt.

------
DoofusOfDeath
I genuinely don't understand some decisions Microsoft has made with Windows
10. The unavoidable telemetry / updates / ads, this Python thing, etc. are
just _so different_ from what I want an OS to do.

I can only assume that I'm somehow outside of Microsoft's target market and
just don't realize it. But is there really a sizable fraction of computer
users that _want_ an OS to unavoidable behave like this?

~~~
Animats
Yes. My last Windows machine just died, and I'm migrating the last few Windows
things to Linux. Have to use Wine once in a while. I don't plan to replace the
Windows machine. It's so unnecessary today.

~~~
simonebrunozzi
I've tried to do this multiple times, and yet I have always kept either a
Windows machine or (more often) a Mac for a bunch of things. (e.g. Keynote;
Outlook in a corporate environment; etc)

I loved it in the past, but now I simply don't have time or willingness to
spend hours to configure drivers, etc, or do my own system maintenance.

My dream is a laptop with a solid linux distro, maintained by a company that
doesn't die too easily, with a 3-5 year maintenance guarantee (kind of Ubuntu
LTS, as a concept, but for laptop use, not server), where everything works,
including a (possibly amazing) trackpad.

The other thing is: Mac's trackpads are absolutely the best in the market.
That single item is responsible for a big chunk of my productivity boost. When
I use a non-Apple trackpad on another laptop, I feel my productivity goes down
significantly. I guess twenty years ago it was being able to use vim and bash
properly :) - but not today, not anymore, for me at least.

Anyway, all of this to ask: what laptop are you using to run Linux?

Edit: forgot to add that I've been looking at System76 and Pop OS, but somehow
they failed to convince me (together, or separately).

~~~
Animats
Laptop? I use a desktop machine. Better ergonomics and a much larger screen.

~~~
simonebrunozzi
Got it.

