Finally! I’m a big fan of Python, but I hope they remove ALL Python from the OS itself. I’ve spent many hours over the years messing with setups to resolve bugs caused by python scripts picking up the Mac OS python install rather than the user-installed version. They really shouldn’t be bundling any Python version with the OS - the user should choose the versions installed. It’s much cleaner that way.
Or at least, they shouldn't be putting it in the $PATH.
Whether you're using Linux, macOS, or BSD, OS-packaged language runtimes exist for the OS itself to use, in OS-internal scripts and underpinning OS services. They shouldn't be considered "part of" the userland; as this invites userland applications to rely on them. These runtimes are actually something more like an "implementation detail" of the OS — the kind of thing whose binaries should live in $PREFIX/libexec rather than $PREFIX/bin.
(Mind you, if you're a system software developer, whose software gets packaged into Operating Systems, then you do have to rely on this "implementation detail." Which often means targeting very old versions of runtimes and libraries, compared to the versions that application developers get to target.)
Also, of course, there are "standard" language runtimes that should be part of the userland — e.g. /bin/sh. But these are literally standard — i.e. part of some standard like POSIX, that an OS can be tested for conformance against. Which is important, because it means that developers can write to a stable target of "POSIX compatible", assuming the existence of (and particular stable semantics of) particular standard runtimes.
But none of the "modern" language runtimes are part of any such standard. The version OSes ship should be assumed to not exist. (And major third-party applications do already do the right thing, in not making any assumptions of existence, but rather installing their own vendored versions of the runtimes that they can upgrade as needed.)
I’ve always thought it strange that most Linux distributions and other *nixes don’t put up barriers between not only system and userland, but also between “essential” userland (i.e. DEs) and other user-installed packages. The lack of separation seems like a setup for failure and complicates package manager design considerably.
This is where the excitements of using linux for day to day tasks come from, right? Install Steam and suddenly your DE got uninstalled [1], for example.
But it really isn't. I never have problems in linux with python apps that are prepackaged by the distro. It's up to you to keep that separate if you install your own apps that require python. You really can't blame it on distro maintainers as they are really doing just fine with it. What happens is people won't take the time to understand what they're running instead of randomly typing a CLI or clicking on the app icon. Linux doesn't hold your hand (and really it can't! there are simply too many possible combinations) unless you use what comes with the distro.
Well I mean, there’s no reason why the user shouldn’t be able to control what’s installed as essential userland. It just needs to be managed entirely separately from the system and “normal” userland. It could be as simple as a flag on apt, dnf, etc.
I'd define it as "never have zero packages that provide a DE". This could be a global setting in the package manager which you would be able to set to `false` if so desired.
You can always build escape hatches for that kind of use case. Some more stringent separation between essential and installed components could have prevented Linus removing his DE.
By the same token, they should remove bash and libc from $PATH as well, as these are also runtimes that the OS itself uses. I think for a long time Python was just as much part of Ubuntu's intended user facing API as those two or GNU ls. Until the Python 3 came and split the Python community in two, there were few reasons to complain. If you want to package your own Python with your app, nothing stops you from doing so and calling it explicitly.
> I think for a long time Python was just as much part of Ubuntu's intended user facing API as those two or GNU ls.
To be clear, it doesn't really matter what Ubuntu's user-facing API is, because nobody writes a software package for Ubuntu specifically. They write it for, at least, Linux; or more likely, the de-facto subset of POSIX that includes at least Linux, macOS, and Windows MinGW. And that "Linux" target, almost always implies intended portability to "weird" distributions like CoreOS or Alpine Linux.
When you create a portable upstream software package that doesn't make any assumptions about where it's going to be installed, you can still rely on both the existence and the (POSIX subset) semantics of ls! But Python? No way.
In fact, you can't actually rely on libc being any particular way, either — but that's what autoconf is for. (Which in turn relies on the existence of, and standard POSIX semantics for, particular tools like m4.) Given that you have those POSIX-standard tools, you can generate a configure script. But you don't even need those tools to run the configure script. It just relies on, IIRC, POSIX-standard compatibility-mode cc(1). That's everything the configure script needs to probe the behavior of your system's libc (and kernel library headers, and a bunch of other things.) As such, a configure script, once generated by autoconf on some system somewhere, is portable to literally any POSIX with a compiler (which is an incredible feat requiring some horribly ugly hacks.)
As for bash: it's not something you can assume. When writing portable code, you can assume (re: POSIX) that whatever shell is at /bin/sh is a compatible implementation of the Bourne Shell; but you cannot assume that that shell is specifically the Bourne Again Shell, or that it can parse Bourne Again Shell syntax/semantics. Under Busybox, for example, /bin/sh is a compatibility mode of https://en.wikipedia.org/wiki/Almquist_shell, and bash-isms don't work. (Ever written a script to run on a Synology NAS? Ash is all you get.)
The moment you start talking about C compilers you've stopped talking about most software running today on Linux systems. The vast majority of packages are delivered as apt/rpm/apk etc installers, which are overwhelmingly binary or interpreted code. If you're delivering an apt package, you will want to assume (or ensure through declared dependencies) some things about the target system, such as bash or Python.
And you'll find numerous bash-only scripts online, either as examples, helpers, stack overflow answers, and many others. Comparatively few bother to support POSIX sh - though that trend is somewhat reversing with the rise of containers and Alpine Linux and BusyBox with them, plus Apple's moves against bash.
Just like you will find no tutorials telling you to edit a file with ed, even though that is the standard text editor.
Finally, if you think it's in any way common to write software to support any POSIX-compatible system, or even any Linux system, you live in a much nicer corner of the world than I do.
I already acknowledged the misguided Python 3 split made this into a problem. But that is not the reasoning offered by the GP - they were claiming that Python was only an implementation detail of the OS and shouldn't have been exposed in the first place, which makes no sense to me.
"They shouldn't be considered "part of" the userland; as this invites userland applications to rely on them."
Or should they? Windows applications rely on .net runtime that is part of the OS. For years now. It's one of the reasons why application development on Windows can be so laid back and stable.
I really, really don't understand why people in Unix-land feel the need to move fast and break things.
My understanding is that once they remove python2, there won't be any Python bundled. There will be a python3 handler that just prints a message asking you to use `xcode-select` to install the XCode command line tools, and that'll include the actual python3... but that's a fairly up-to-date version these days. (Right now it's 3.8.9.)
I fully agree with removing Python from OSes, but I don't know that it's reasonable for the average user to manage their own versions (I wish I didn't have to manage my own versions and I've been developing in Python for the last 15 years). The alternative is to ship the runtime, stdlib, and dependencies with each application, but that seems pretty sucky for an interpreted language and especially one that leans so hard on random native extensions (I've seen relatively simple applications pull in 250MB of dependencies in a compressed tarball, never mind runtime and stdlib).
Really I don't think Python (and probably other interpreted languages) is a very well suited for developing end-user applications--although some people have fewer scruples than me about wasting user resources.
Please rest assured that this is not a flamey / troll comment, it's just that your first paragraph to me sounds like "Python is just not worth the trouble" which you kind of allude to in your second paragraph after.
I mean, at what point should we start asking ourselves fundamental questions like "what is so heavenly about Python that we have to carry it everywhere and ensure its installation on every possible platform"?
Example #1: I know a guy who regularly writes scripts in OCaml and laughs at everybody else, saying he gets static types and mega-fast binaries (faster than Golang and slower than Rust) and a very terse syntax (he says it's often as short or shorter than bash even).
Example #2: I recently finally had enough of bash/zsh scripting weirdness for a few tasks I wanted done on my home server so I just learned enough Lua for an hour to make a fairly decent script that did exactly what I expected immediately after I understood a few of Lua's quirks (which are at least consistent and easy to remember, unlike the endless escaping and quoting rules of the scripting shells).
Example #3: I know a few people using a Rust library/tool that makes scripting with it much easier and almost painless. They swear by it.
--
I am not even mad at Python for potentially putting hundreds of megabytes or gigabytes of baggage on people's machines -- if we start having each app carry its own Python runtime. Meh. Many people would never care (me included). I do get ticked off by it being everywhere however; especially having in mind what an awful language for long-term projects it is (said to me by at least 5x senior Python devs at 40+ of age). And how often it messes up with other work you do on your computer, too -- which is the cherry on the cake and has been guilty of tons of broken Debian system-wide upgrades (among many others).
I wish people stopped getting tempted by "easy to start with" languages and just start seeing the forest and not only the trees... :(
I generally agree. I don’t think Python is even “easy to start with” these days. I think it got that reputation in the early 00s, but since then the competition has surpassed it.
Languages like Go are arguably easier to use and getting a basic project up and running and sharing it with others is far easier than with Python. If you really care about a scripting workflow, it’s as easy to do ‘go run main.go’ as it is to do ‘python main.py’ for projects without dependencies and for projects with dependencies it’s a lot easier to just do ‘go run main.go’ than configuring a venv and installing your dependencies and then running ‘python main.py’.
Nit: Ocaml is about on par with Go. Some programs will be faster and others will be slower, but OCaml is very much in the same performance tier as Go, Java, and C# (benchmark game prohibits idiomatic Go code for certain benchmarks so it appears artificially slow).
Per the sibling thread, I think you’re mistaken, but that’s okay. It’s a widely written about topic and anyone can read up on it. We don’t need to hash it out here.
It’s not allowed to preallocate memory in certain benchmarks, so it has to do allocations in a tight loop. Go’s allocations are more expensive by design because idiomatic Go doesn’t create as much garbage as other GC languages.
If you’re here for a language slap fight, you’ll have to find someone else to talk to.
EDIT: good grief, the overwhelming majority of your recent comments are “debates” about the benchmark game. Do you have an alert for references to this phrase or something?
> They really shouldn’t be bundling any Python version with the OS
I fully agree, Operating Systems bundling Python and Ruby has caused me many headaches in the past and I always perceived it as a downside, never an advantage.
> They really shouldn’t be bundling any Python version with the OS - the user should choose the versions installed.
Why? Having a bundled Python version - any version - means that you can ship Python scripts to others. Not having a Python version bundled means you can't ship Python scripts at all.
Sure, you can still technically ship them, but now you need to tell users "just run this simple scipt!... after you install Python X from place Y, and now you'll have multiple Python versions installed, so here is how to keep them separate, and [...]". Or, you can bundle your script with a whole Python installation!
It’s only really feasible to ship APIs in an end-user operating system that can make forward binary compatibility guarantees.
Python doesn’t make a sufficient guarantee of forward compatibility for either scripts or its C-linkable libraries. This means that, unless an OS vendor does a bunch of extra maintenance work on the Python that they ship, an application built in or making use of the Python shipped in one version of an OS may not run on a subsequent version of that OS.
I use pipx for this. It isolates each tool in its own virtualenv. You might still have to install a specific Python version (although the latest 3.X works fine for me), but it takes some of the pain out of the process and makes sure you don't pollute your global/system install.
On a Mac, it could be as simple as (assuming you already use Homebrew):
Pipx is really one of the best things that has happened to Python recently. Taking away all the headaches of manually managing venvs. They should integrate it into mainline imo.
I've only run into issues like this when upgrading from Python 3.X to 3.Y, which is to be expected, and Homebrew lets you install a specific 3.X (e.g. brew install python@3.9) if you don't need/want be on the latest version.
As a non python native, I sigh whenever I have to install anything in python. There's always a dependency I can't find and never know why. I never understand why the dependencies aren't just resolved by an application and local to that application. I mostly end up with something working. Sometimes I just create a VM for that single application.
I once really desperately needed a Python tool. Wasted an hour and getting more and more enraged trying to install it, overwriting hell-knows-what and ruining god-knows-how-many other programs in the process. Eventually it did occur to me to just use Docker and that did it. <facepalm>
Every time I am forced to do something with Python I end up hating it. It seems those who write it are just not interested in anyone who doesn't have a complete dev environment setup for it. :(
> It seems those who write it are just not interested in anyone who doesn't have a complete dev env...
And that's probably what makes Python so popular, and so great. I get it. It's attractive to people who aren't developing software for other developers.
But what you described is surely the result of that atmosphere.
> And that's probably what makes Python so popular, and so great. I get it.
I don't follow because that's actually a problem. Imagine having to install Haskell compiler -- exactly one particular version of it with the exact right combination of flags (and there are hundreds of them) -- for each project... while also having the whole thing kind of compile from source.
And that for people who just want to do something like `brew install a_valuable_tool` and have no patience (and certainly not the time) to tinker with Python's dev environment just so they can install a tool they need.
So maybe I am misunderstanding but no, that's not what makes Python popular or great, at all. It makes it a legendary pain for anyone who doesn't code it every day.
If I can't do `brew install your_tool` but have to fiddle with the intricacies of your language's ecosystem then you have failed as a tool author.
Contrast this with Golang tools (disclaimer: I dislike Golang) which in the worst case scenario involve 1-2 commands to download and compile+install the tool but usually don't involve even that because Golang's process is self-contained and that makes it super easy for all package managers to distribute it pre-compiled for their platform.
The distro should be packaging software for end users, relying on the python version they maintain. The problem is that Apple isn’t doing that at all, and brew seems to have issues getting all formulas to converge on known-good dependencies.
I'm not sure if it resolves your ownership issues, but I am glad they went with `/opt/homebrew` in ARM Macs to at least keep homebrew in its own directory.
The way it handles ownership (or at least used to) is completely broken with screwed up stuff happening inside /usr land.
Macports puts everything under /opt/local and you add /opt/local/bin to your PATH.
I've never understood the preference for homebrew, when I moved to MacOS (from Linux) 5 years ago as my working development environment, I read about both and homebrew seemed like a complete kludge, whereas Macports worked the way it should.
SIP just made it even more relevant. Macports deals with specific MacOS related integration (eg Java etc) using the Apple blessed mechanisms while still providing all of the flexibility of the many many packages.
There is a lot of good things to say about Python which is a fairly nice language but I feel it is messy on every platform. Tooling makes the situation bearable nowadays but that's definitely not one of Python strong point.
But it's so easy to set the path up to python? I hear people complain about this all the time but I never encounter it because I'm careful and I know which interpreter I'm running ALWAYS. It really isn't a hard task to do. Having python in the OS is fine. I wish windows had a default version and I love that it's available on linux in most distros.
A lot (all?) has been cleared out in Monterey. I can’t find smtpd or xattr’s scripts that required the previous version. I think whatever remained before 12.3 were Automator’s py scripts. But I don’t know what scripts those were.
Mine does seem to contain `/usr/bin/smtpd.py` with a modified date of January 22nd. It's an installation that's been updated through a number of versions of macOS, though.
Having some python always installed can be nice as an advanced calculator or short ad-hoc scripts.
My non tech mom needed help to wrestle a large set of semi structured data into a csv to use with Excel. On my dev machine I would have solved the problem in a minute with 5 lines of Python. On her machine, without any coding environment, I was completely stuck. Eventually figured out you can use the developer console in chrome to run some interactive Javascript and problem solved.
MacOS (Certified Unix OS) is bundled with multiple OpenSource projects since it's initial OS X release, just like many other *nix OSes. It's so easy to install your own version and override it that this is not an issue. Simple response... use brew.
PSA: don't use brew to manage your development python, unless you like having the carpet pulled out from under you by brew deciding to change 3.8 to 3.9 on you.
Use brew to install pyenv (or bootstrap it from github). Use pyenv to manage python versions, and then use a virtualenv.
Yes, this is what I do now and since moving to pyenv I haven't had any real issues. The only glitch was that I have to upgrade pyenv from 2.2.3 to 2.2.4 and it just tells me 2.2.4 is already installed when I'm running 2.2.3. I'm guessing that's a homebrew thing?
There's a bunch of places that can still get tripped up:
- scripts with `#!/usr/bin/python` instead of `#!/usr/bin/env python`
- shell scripts which call /usr/bin/python directly instead of the one in your path
- shell scripts which intentionally set a prefix in $PATH to some vendor directory with a custom python, but the vendoring is incomplete and leaky and it implicitly loads packages from your system python anyway
plus a hojillion other things I'm not thinking of...
you should absolutely avoid overriding the default python installation if you're on a distribution that ships python2 by default in particular, because that is a good way to kill a good chunk of your system.
Everyone should depend on “python3” or “python2”, never just “python”. They aren’t really the same language. python3 can’t run python2 scripts, nor vice versa, so no matter what you expected there will be systems on which you won’t get it.
I know it shouldn’t, but the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language.
Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? If 2.X -> 3.X was such a drama fest what’s gonna happen next time they need to upgrade? Etc
While I happily use Python 3 now, the transition was very difficult, and I was grumpy about it at the time.
I work with code that uses a lot of files in a lot of different text encodings. Some are XML, some plain text, and some binary. Coming from other languages, Python 2's Unicode support was very difficult to work with, and my team was excited to move to Python 3, until we actually had to do the work.
Long story short, Python 3's str/bytes separation was a nightmare, especially when having to deal with third-party libraries that were expecting one or the other. This was especially true for libraries that came from Python 2, and were still expecting strs as function parameters when they should have switched to bytes. There was so much encode() and decode()ing going on that we occasionally caught ourselves getting them backwards in code reviews. We took a step back to see if we were just architecting things poorly, but no, we were doing things the best we could for our problem domain.
In the end, we traded one set of Python 2 text encoding footguns for a mostly different set of Python 3 text encoding footguns. It's not like Python 2 was great. Having a single str type instead of the str/bytes separation is worse in theory. But because Python 3 didn't design the separation of those types well enough, and still has a ton of text encoding footguns. It wasn't better enough to really justify all the work that went into the conversion.
Transitioning was hard, especially with all the libraries in various state of support between 2 and 3, I had a similar experience.
But I disagree with you about the separation of bytes and string and the current state of the language. I write a lot of python that deals with bytes and text encoding, and now that all the libraries have caught up with 3, the situation is way better than it ever was. encoding, decoding, bytes manipulations are way less prone to errors.
I'm glad it's improved. I moved jobs a couple years ago, and since then, have only used Python for a few small personal projects. When I left, there were still some very rough edges around some very popular (and some not so popular) libraries.
And, as I said, I do prefer the str/bytes split over the unicode/str split, but I also wasn't doing a lot of raw byte manipulation. I agree that it was harder to do with the old str than bytes. I was mostly doing string operations on the grapheme cluster level, and then writing everything out as UTF-8, so I didn't see as much of the benefit.
That was probably an inartful way of saying it on my part. Echoing what I said in another comment, I think the API for converting between bytes and str types is bad. .encode() and .decode() are bad names for what these functions do, and let to occasional incorrect code in both Python 2 and 3. I would have preferred something akin to:
b = b"some bytes"
s = u"some str"
foo = b.to_string(encoding)
bar = s.to_bytes(encoding)
or
foo = str(b, encoding)
bar = bytes(s, encoding)
I also mentioned that I think there shouldn't be any mechanism to read or write files to/from a string without specifying an encoding. locale.getpreferredencoding() is a mistake.
But one thing I didn't mention that I think would improve the design would be to make it harder to treat str as the bag-of-bytes type it was before. Swift took some steps down this path, and honestly, it made it much less pleasant to work with strings, but a bit safer. I would hope that a better design could be found.
This may be controversial, but I don't think you should be able to just subscript a str type. instead of s[1], you should be writing s.codepoints[1] or s.graphimeclusters[1], etc. depending on what you actually want. If str is truly a string type and not a bag of bytes, it should deal with the extra complexities that being a string brings to the table.
Oh, I see what you mean. Yeah, I have gotten tripped up wondering whether I should be calling encode() or decode() many times. Using the str() or bytes() constructor would be more intuitive for sure.
Interesting thought on subscripting strings. It would be hard to make it fly after all these years of habits, but I see your reasoning.
What do you think the current text encoding footguns are?
In a different direction, I don't know what your problem domain is/was, but in general when I'm dealing with UTF8, I don't need to convert back to bytes very often. Was the need for conversion mostly due to the libraries that still expected strings instead of bytes?
It's been quite a few years since we went through the conversion, and at this point, working in Python 3 is natural to me, so I may not be able to recall all the footguns. I can say a lot of the difficulty was due to libraries, both third-party and standard, and that hasn't improved very much. I don't want to single anyone out here, because it's pervasive. In Pytuon 2, str was the bag of bytes type. I think a lot of libraries didn't want to change to accepting bytes types instead, because it broke API compatibility, but it caused a lot of issues.
I should also say that we were working with files in tons of encodings, not just UTF-8. We had UTF-16 and UTF-32, both little and big endian, with and without BOMs, but we also had S-JIS and a bunch of legacy 8-bit encodings. Often we wouldn't know what encoding a file was in, so we'd have to use the chardet library, along with some home-grown heuristics to guess.
Off the top of my head, the two biggest footguns are:
- There should be no way to read or write the contents of a file into a str without specifying an encoding. locale.getpreferredencoding() is a mistake. File operations should be on bytes only, or require an explicit encoding.
- .encode() and .decode() are very poorly named for what they do, and it wasn't that uncommon that someone would get them backwards. Sometimes, exceptions aren't even thrown for getting them wrong, you just get incorrect data.
Both of which were still issues with Python 2. There's a valid architectural argument to be had between the Python 2 way, where str was a bag of bytes, and the unicode type was for decoded bytes, and the Python 3 way, where the bytes type is your bag of bytes and str holds your decoded string. I favor Python 3's way of doing it, but it's almost six of one, half a dozen of the other. The advantages of one over the other are slight, and given how many library functions relied on the old behavior, it was probably a mistake to change it like that, rather than continuing the Python 2 way, and fixing issues like those above that caused problems.
I haven't done a ton of work with Python recently, but the problems I remember encountering came from the fact that python doesn't try to have encodings in any other part of the basic type system. So like, if you have an int or a float, you can pass those to any interface that takes a 'number-y' value and it will mostly work like you expect. That's also how strings worked in P2 - you could pass them around and things would accept the values (though you might get gibberish out the other side). Now, in P3, things will blow up (which is helpful for finding where you went wrong ofc - I understand the utility), but it means that your code handling things-that-might-be-strings-or-bytes often needs to have a different structure than the rest of your code.
I think the P3 string/byte ecosystem was made substantially weaker by P3 deciding not to lean more into types (something I have complained about on here before!). Like...they are the only values where the stdlib is extremely specific about you passing a value that has the exact right type, but the standard tools for tracking that are pretty poor.
> but it means that your code handling things-that-might-be-strings-or-bytes often needs to have a different structure than the rest of your code.
Isn't that the point? String and bytes are different beasties. You can often encode strings to bytes and just about anything accepting bytes will accept it, but the converse is not true. Bytes are more permissive in that any sequence of any 0x00-0xff is acceptable, but str implies utf8 (not always guaranteed, I've seen some shit), meaning e.g. you can dump it to json without any escaping/base encoding.
I actually like Python a lot. Although I'm no longer using it professionally, it's my first choice for personal projects.
I'm also an advocate for using the right tool for the job, and in this case, Python may not have been the right tool for this job, but this was only one component in a much larger system. Sometimes you have to be suboptimal locally to be optimal globally.
And it's not like Python couldn't handle it, it was just that it had some design decisions that made things a bit harder than it would have been in some other languages. We got it working pretty well in Python 2, then the Python 3 transition happened, and it was a lot of work to get everything working as it had been, for only a small benefit to our team, but we got it working in 3 as well, and to my knowledge, it's still humming along.
> the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language.
There weren't that many people who were outright against Python 3, other than Zed Shaw for a few years. It just took until Python 3.4 for the language to really be usable in production, and after that it took a couple more years for every library to be updated. But the community has been pretty unified for 5+ years at this point.
If by unified you mean no longer arguing, sure I switched from Python 2 to Go lang. (I will concede if I am doing exploratory stuff, I will start new repos in Python 3, e.g. for boto quick exploration.) I am still bitter about the fine Python 2 code I supported and extended that is now all go or Java.
This is my experience as well. There were plenty of people online vocally against Python3, but once we hit 3.4 I never worked with anyone IRL that was against it. The mindset was almost always "this is the way forward", so that's how we moved with our projects.
I used it production just fine before that. What didn't work for you? The unicode support was shoddy, but lots of other stuff worked fine.
The problem as I remember it was that there were some critical tools that just were not making the switch. The last one I could remember having to have a separate runtime for was graph-tool, but there were just a ton of them in the scientific computing community. Also, I don't think making print a function was worth it. So many people had muscle memory on that that personally I think it was worth just single casing it.
There definitely were, at least in the early versions. It would have been fine (I guess) if those early versions/iterations had been short-lived, so to speak, and for a stable and definitely faster version (compared to 2.7) to be made available shortly after that, but afaik that wasn't the case.
If it matters I've been writing Python professionally for more than 15 years and I started my career by seeing the (less famous) Zope2 to Zope3 botched migration. When Python3 was first announced I had hoped that the devs behind the project had learned from that related experience, guess I was wrong.
> But the community has been pretty unified for 5+ years at this point.
Sure, but they accomplished this mostly by displacing anyone who disagreed. That's fine, and Python 3 in a vacuum is a better language than 2. But I'd never make the mistake of choosing Python for a serious project going forward.
The "active community" is unified, I know a bunch of people with some personal piles of Python 2 who I don't expect to upgrade until they have no other choice. I also don't see why they should -- their code is running fine.
They forced incompatible changes on their users, all in the name of a Unicode fetish. They burned their credibility. The users acted rationally, it's the developers of Python who were abusive.
I think it’s less of a fetish and more of a “we made a huge mistake in the initial design of the language and confused text and the encoding of that text making a giant footgun that was hard to avoid.
The migration was difficult because you had to actually think about this stuff instead of it working by accident and living as a landmine if you encountered non UTF-8 text.
The fact that you actually have to decode your bytes and specify an encoding makes it really obvious when you’ve just assumed the world is UTF-8 without really backing it up.
What I don't get is it they break the language anyway, why they didn't fix classes. Most of python is pretty beautiful, but classes are so ugly, even basics like what is the Syntax for instance vs static cars, self/this, setting up constructors, calling inherited constructors. Like come on, how is it that they are uglier than Java in this regard.
I really don't know what you're talking about. I've written a lot of object-oriented code in Python and Java and there isn't a single thing I like better in Java (that doesn't come from the typing system). I don't know about 3.0, but it's been great for years. Take a look at some Django code.
The print() change drives me nuts: many scripts use the old form, and Python forces you to change it despite recognizing the old functionality. They could've simply allowed print expr to work as is and accepted it as a wart.
The async stuff feels a bit like a bridge too far for me WRT the language. Simplicity would be standardizing on a function color (as it were).
I don't envy their position of preserving backwards compat while evolving the language.
I think the person you're replying to was saying that there was no reason to remove the statement form. The only thing that removing the statement form of print did was break existing code. You could easily allow print to act as a function by removing it as a reserved word in expression contexts.
They made print a function out of a misplaced sense of language "purity", where they made a random ideal and broke code to support it.
Let's compare to JS: 20 years of new features and old code still works. Keywords were added, and old code still works. Even code that uses those keywords still works. It took more effort from the language implementers, but that's as it should be: it took a bit more effort for a few teams (who make the engines), instead of a large amount of effort for tens of thousands of groups.
If the reactions seem bizarre, maybe you're missing something.
For me it was more than "just the tech". Python 3.x was not the Python 3000 I was promised for a long time: many of the additions did not warrant breaking changes (as demonstrated by the backports) and many of Python's deeper warts still remain unaddressed, even today.
I think the deeper issue is that at the rate that 2.x got popular, the original motivation, principles and process behind the language's development got diluted. The orchestration of the push for 3.x adoption was not technically motivated in a convincing manner and signaled a change in the process. I stopped trusting the process.
My company started life in about 2014 and for python is only on 3. And python 3 is great. Having written python 2 before and also perl, I really appreciate UTF-8 just works.
And I hope that python learned from 2 -> 3 and that in the future it will be better
The fact that you think this way shows why that community is so broken about things like this. You think the user community behaved badly? It was the developers.
For the vast majority of companies, python 3.x is not better enough for the cost. That is what happened. You may think it's awesome. It may be technically better it, etc. But the cost of moving was much more for most companies than the productivity gains. We've even measured it before. That is entirely on the python folks for not providing something people want enough to move to. All CSAT surveys of python i've seen at my company, and that my counterparts had ever seen, said literally the same thing.
So it seems like something the community could have discovered pretty easily in a rigorous way if they wanted. I used to spend a lot of time on python-dev. It's a group of good people and engineers for sure. There was always good rigor around technical reasons to do something, and around performance benchmarking. But i don't remember seeing a lot of productivity research or CSAT or .... It's fair to point out that most OSS communities don't do that, but that's a bad thing, not a good one, and one of the things that often leads to community trouble
One of the other major factors, besides the productivity issue, that makes it not better enough is the migration cost. Software of this kind needs to be built to be migrated from and to. If not, it's broken and you deserve all the animosity you will surely get from your users. It doesn't matter if you screwed up the language, etc. Tough crap.
In python, for lots of folk with substantial stacks, the amount that could be auto-migrated by the tooling provided was low - 40-50%. Yes, simple stuff could be auto-migrated, but not most real stuff.
We actually ended up building our own tooling to get closer to 70% because how expensive the overall thing. It processed many millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation.
The total engineer cost of moving was enormous. We are talking well over a thousand SWE years.
For what, exactly? As I said, we already measured (because plenty of new projects were python-3 only) and determined that all the improvements and hullaboo did not meaningfully improve productivity for people. Not enough. It was small, at best. It also didn't reduce the rate of production bugs, etc.
So all this work, for no meaningful value.
Because not enough were willing to do it, the result was to force it. Damn the users, full steam ahead. The users are just lazy.
I talked with lots of counterparts at other companies - most felt the same way.
The result for us is lots of teams gave up python forever.
The only reason it's growing at all is because of ML.
IMHO, python2->3 is an object lesson in doing it wrong and then victim blaming.
"We actually ended up building our own tooling to get closer to 70%. It processed maany millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation. The engineer cost of moving was enormous."
This was definately an opportunity for commercial software, i am not sure why companies worth hundreds of billions can't sort this out and expect a small nonprofit to provide this.
FWIW - i don't disagree, and I don't expect anything in the end, it's open source. People can do what thy want.
I was more pointing out the opinion given of how the user community sucked is very broken.
What the "larger user" (however you want to frame it) community would have preferred (IMHO) is much better migration tooling, even if it came at a cost of improvements.
The dev community chose to spend time on improvements instead. That's their choice, but it doesn't make the community wrong to be upset about it.
That's the problem - they literally do not. Technically better is not a good enough reason to change something. It has to provide some actual greater user value along axes that matter to users, and make the cost worth it. What you are seeing is that python3, well, doesn't. The options in that case are reduce cost or improve along axes that matter enough that cost is worth it.
> Inertia seems to best describe the hesitancy to change.
I'd have gone for 'rigor mortis', but hey, we can't agree on everything ... like, say, a very minor change to string encoding with a 12-year managed rollout.
I did a bit of contract work for a place with over a million lines of custom python 2.x scripts that ran inside of a python interpreter embedded in a proprietary product which made it difficult to do things like write and execute unit tests for the code. I think they were still writing new python 2.x code in 2018.
A lot of the scripts were supporting processes that had a finite lifespan during the initial stages when the company infrastructure was being built out, rather than ongoing operational processes that needed to be performed indefinitely, so hopefully they'll be able to set a lot of that codebase on fire instead of maintaining or porting it to python 3.
> Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech?
What you say was bad about it? And who were the bad people specifically? The people who were using python 2 or python 3?
For what it's worth, python3 >= 3.0 && python <= 3.2 were hideously broken in their unicode support. Arguably had worse/unusable uncode relative to python 2.6 or 2.7.
So there was a huge failure to launch type of problem, especially given how long python3 had been development.
It most definitely left a very sour taste in many people's mouth that didn't start dissipating till 3.5 or 3.6 when enough "killer" features had accumulated.
Even then, for a lot of usages, python 2.7 'just works'.
>Choosing to keep an unsupported piece of software in the system is a security liability.
It is the OS design that doesn't make it possible to support the principle of least privilege that is the security liability. Blaming users for wanting their working software to just keep working the same way is abuse!
If they want to run a piece of code written in HyperCard for the original Macintosh, they should be able to do so. This push towards ever more complexity while failing to fix the inbuilt security fault inherited from Unix is insane.
Why not? If you have a fault in a lamp switch, the circuit breaker keeps it from burning down your house.
Why can't an OS keep a rogue program from taking down your machine? You could specify what to run, and with what resources, and then even if Satan himself wrote the code, it wouldn't ever corrupt anything other than the resources specified.
This has to be one of the dumbest arguments i've ever seen on this hellsite.
Firstly, a circuit breaker does nothing but protect the wiring in your house. A faulty lightswitch can still throw sparks and start a fire as long as it never draws more than 10A or whatever your circuit breaker is rated for.
Similarly, operating systems can prevent a rogue process from taking too much CPU or RAM, but can't prevent a process from ransoming your files, at least not without a substantially different sandboxing regime to what exists in mainstream operating systems today. Why we don't/can't have this is an interesting disucssion that is much larger than the python2/3 question.
If that was the argument they'd have replaced it way sooner, because 3.x came out 14 years ago, 3.4 (when the 3.x branch could be called "actually ready for use") came out 8 years ago, 2.7 was announced to go EOL "soon" 5 years ago, and was properly killed off almost 2 years ago now.
We've seen an entire major version release of MacOS come out since Python themselves went "look, we're killing this thing off, deal with it", at which point Apple should have dealt with it, but didn't.
This is a replacement for the convenience of their own staff working on the OS, nothing more. It's good that it's finally happened, but it's a good five years overdue, and there's no motivation beyond "we no longer use it" at play here.
We all knew this was coming, but doing it in a point release seems like exceedingly bad form, particularly as Apple does major releases annually. I never expect point releases to break stuff, at least not on purpose.
The risk of unpatched vulnerabilities between now and the next "major release" far exceeds the risk of inconveniencing anyone foolish enough to rely on the system Python 2.7 installation.
If they wanted to announce at the June 2022 WWDC that macOS 13 wasn't going to have it, that's fine. But we're several months into macOS 12.x and such a large change is bad form.
Let's face it: at this point macOS version numbers are a marketing tool, and not much more. It's only 2 major releases ago that a "x.y" release *would* have been considered "major".
I don't think that's particularly relevant. It was pretty clear that between 10.0 and 10.15, the second number was the "major" version for all intents and purposes, and now it's the first number.
Bingo.
This falls under the scope of support for an enterprise linux company-- orgs pay them to keep stuff backwards compatible long after the upstream deprecates it. Apple is decidedly not an enterprise support organization.
RHEL also doesn't want you using system python as your dev python. RHEL 8 uses 'platform-python' for the bits the system needs and if you install 3.6, they share the common bits
If your using a mac as a dev box, and really need python 2.7, you still have options.
Apple supported Python 2 for two years after it was officially sunset by the Python Foundation. They've been advising that they will drop support for longer.
How long is long enough to avoid breaking something the users depend on? One more year? Five years? Fifty? I think we can all agree that it's reasonable to drop support at some point, making it just a question of when and at what cost.
"How long is long enough to avoid breaking something the users depend on?"
10 years - a piece of software compiled with 'supported' libraries, SDKs and what have you should function for 10 years. An appliance or a service should function for 10 years and have spare parts avaliable for that long.
That is the guideline I ise as a consumer, if I can see that something won't last a decade, I won't be spending money or time on it if i can avoid it. At that point it's no longer a durable good, it's a consumable.
If there are users that still depend on python 2.7, they can install it manually?
I mean I already think it's... interesting that an operating system comes with (multiple) programming language interpreters, available to the end-user.
That’s true if they’re using it from the command-line, but not if they’re using apps that linked against the system SDK’s libpython2.7.dylib (and assumed PyObjC would be present)
Is there any significant software still using that?
Apple is pretty aggressive about breaking backwards compatibility, as they did with x86, so it's no real surprise for unmaintained software to become completely unusable.
Based on what they’ve done, I guess the answer is no.
My company was shipping (somewhat niche) software linking with the built-in python2 up until last year though, and it’s been a scramble since Monterey to rebuild things to avoid the warning message. Definitely thought we would have python2 until 13.0 though (and part of me wonders whether this is just a “brown-out” for the beta and 12.3 release will be back to normal)
I think an end user that wants to automate things with a script can be expected to install their interpreters themselves or use an interpreter that builds on the OS's shell.
I am not to familiar with MacOS, but I imagine it comes with a bash/sh terminal? If so, then that is something that should be first choice for automation.
Everything else, be it python, perl, ruby or whatever should be installed by a user as needed.
And I think that's totally reasonable—as long as Apple makes the change in a major update, when users can expect (some) things to break and should be encouraged to make time to prepare.
Part of it is that it's just a mental burden to keep this kind of compatibility information in my brain. I have VMs set up of past macOS releases in case I need to run something old. I don't have VMs set up for earlier versions of a release, and I don't even know where I'd get an installer. Admittedly, in this case it would just be a matter of installing Python, but I don't necessarily know that as a user, or the software could need to be linked to Cocoa or some such.
> Apple the corporation doesn’t have a package manager.
> MacOS, the ecosystem does.
> No different from Linux. There is no Linux package manager. There are numerous open source package managers for Linux.
macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
There is arguably no single "Linux ecosystem". There are numerous Linux-based distributions, which are largely comparable to macOS as an OS. One of the very key differences between each family of distro's is the package manager the OS ships with by default and uses to update itself.
So no. macOS does not "have" a package manager that's comparable to those included by default with linux distributions, but there are several third party package managers available for it.
> Another false statement. Homebrew is just as capable of searching as any Linux package manager.
I wasn't talking about homebrew.
It helps if you understand the comment before you reply to it and make accusations.
Edit to add, because fuck the HN timeouts. Heaven forbid someone make 6 comments in two fucking hours:
Try reading the whole fucking comment again, with the points being made really hammered home for you:
> macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
This is about the built in macOS package manager, that the OS itself uses to install packages. It's invoked via `installer` on the command line, or Installer.app in the GUI. It does not expose the ability to find/fetch packages from remote URLs.
> In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
This is referring to homebrew, macports, etc.
We certainly do see things differently, because only one of us is repeatedly calling the other a liar due to poor reading comprehension.
If you don’t include homebrew, as a package manager for MacOS, then you aren’t having a serious conversation.
This whole line of conversation is about whether it’s necessary to manually install python.
If you claim it is you are lying, because it can easily be installed via homebrew, and you are clearly aware of that.
If you have some need to claim that homebrew isn’t a MacOS package manager or that MacOS doesn’t “have” homebrew, because Apple doesn’t ship it. Go right ahead, but it’s a red herring.
No, it means that macOS and its third party software aren't one in the same thing.
Is faker.js a macOS vulnerability because there were people using Macs who installed npm and then installed faker.js? This isn't a theoretical question—whenever you install something, you need to consider where it's coming from and whether they can be trusted.
It has a third party package manager, yes. I'm pretty sure your parent meant that there isn't a first party one.
On Debian, packages in the official repositories are considered to be literally part of the Debian project. The maintainers are considered Debian maintainers, and bugs go into the Debian bug tracker.
This matters due to the trust issues I alluded to above. It also means that on Debian, there is one definitive version of Python (or at least one per Python version). The Homebrew, MacPorts, and Python.org Python binaries are all slightly different, such that it's possible for software to work with one but not the other.
No two Linux package managers are identical either, and not all Linux distributions work like Debian. Debian is a collection of volunteers. The choice to trust it is no different to the choice to trust Homebrew. Considering the package manager to be part of Debian doesn’t change this.
However none of this has any relevance to the claim in question, which is that python must be installed manually.
MacOS certainly has a package manager that can install and maintain python.
Except that most other open source package managers strive to be good at what they do.
Brew has a long history of treating any criticism of their truly woeful approach to security, dependency management etc, with "well fuck you we dont want to hear your opinion".
There were other better macOS package managers before it; macports was sort of even Apple sponsored.
Everyone switched to brew even though it was the most poorly designed because it came from the Rails hype era where everyone wanted all their tools to be written in Ruby and didn’t care about anything else.
I used and still use macports, but there were valid reasons to switch to homebrew: macports did not have precompiled packages, did not use system libraries, and machines were not as powerful as they are now.
This meant if you wanted to install, say, ImageMagick, you would spend one day compiling stuff.
Also contributing a brew recipe was (and probably still is) easier than contributing to macports, and brew casks are pretty convenient.
I deeply dislike some of the choices of homebrew, but I can understand why it was popular, and it wasn't just because of the ruby hype.
Just because someone has the resources to do something it doesn't mean that doing so is the right {business, technical} decision. You're confounding the two.
Customer service is still a business reason. It serves to keep your customers happy, and the calculus here is probably that there aren't enough customers to keep happy by maintaining python2.7 that would justify the costs and headache.
As a counter-point, my app was able to rely on the system Python 2.7 for FIFTEEN YEARS because it was an extremely stable component of the OS (and it allowed the app binary to be literally 10 megabytes).
Now? I honestly can’t think of a way to release without adding something like 100-200 MB to the app bundle, consisting of an entire Python installation. With a user-facing app it is not ideal to try to say something like “just download Python and tell me where it is”.
The risk of unpatched vulnerabilities between now and the next "major release" far exceeds the risk of inconveniencing anyone foolish enough to rely on the system Python 2.7 installation.
You make an assumption that the way your system and method of working is the same as everyone else on the planet. It is not.
I have two tools that I use monthly which require python 2.7. The people who made the tools never updated them to work with 3. For this reason, I have a work box that I cannot upgrade.
I know that Macs are aimed at end users, and not developers, but it really is getting harder and harder for me to use my Macs for the kind of work I do. Each major software update borks my virtual hosts. PHP has been removed, and because of code signing requirements, adding it back is so complicated it takes an entire day and filtering through a dozen half-baked online tutorials. No, the ones that say "Just use brew!" are not the answer unless you have a very simple job. I don't use Docker, but from what I read on HN, that's six miles of bad road, too. But as I understand the situation, that's Docker's fault, not Apple's.
I've previously worked at a vendor using Python2. Someone brought up this concern. I looked up the number of security releases we had cut in the past due to Python2 vulnerabilities -- it was zero.
Doesn't need to be an RCE to be considered an exploit of the user's intentions. I can't fathom how people are still trying to use an EOL language or why they would even want to. Learn some new syntax, you'll live and the writing has been on the wall for like 20 years.
I don't think a whole lot of people are writing new software in Python 2, but I think a lot of people are still running old software via Python 2. Software they didn't write themselves and wouldn't know how to modify.
And that's not to say those users won't have to find something new eventually (or migrate the software to a contained environment). It just shouldn't happen in an automatically installed update.
>The risk of unpatched vulnerabilities between now and the next "major release"
His point doesn't necessitate waiting. You could just classify it as a major release instead. That is not to say it should be a major release necessarily.
Right, Apple could have removed Python 2 last fall with the release of Monterrey, and I would have been neither surprised nor unhappy. I was actually kind of surprised when it was still in Big Sur.
Then they could have done it in 2021 with the release of Monterrey. Heck, Monterrey was released before Omicron, so if we're concerned about COVID that should have been less disruptive than doing it now.
This wouldn't have worked either. Lots of countries, where vaccines weren't widely available yet, were going through a huge Delta wave before and after WWDC.
Now that most places have vaccines, it probably doesn't make sense to wait for the next major release because this has already been long delayed.
Other factors could have also been at play - perhaps Apple Inc had a lot of internal tools on 2.X and has only recently gotten around to migrating all of those.
I think this mindset is dangerous. There's a really long tail of niche things people do on their computers, such that I suspect a majority of users have at least one niche need. Especially today, when Chromebooks and iPads are available for anyone who only needs basic web browsing.
If you stop paying attention to the niches, they add up to a lot of people.
You are right, but I would say that this is quite extreme case. This is a programming language which support ended 2 years ago. Only reason to use it is to avoid turning old code base to new one. There are some serious questions if that is really a good approach. And if it is, why you need to use pre-installed Python 2 instead of installing it by yourself?
I expect that there are users, who are not themselves developers, who are using either apps or scripts which rely on the system-installed Python 2. They likely aren't aware that they're using Python, and they may not even know what Python is. It could be one line in one shell script block in one automator action that a coworker put together five years ago.
It's okay for that stuff to break. (I wish Apple would put more effort than they do into making it break less often, but that's neither here nor there.) But, it should break in a major release, when the user is prepared for things to break. It should not break in an update that was installed automatically overnight.
It's funny how people think old, robust software is suddenly going to spring a bunch of vulnerabilities while sitting there, static, unchanging. It's far more likely the bleeding edge python(s) are going to be getting new bugs.
* One of the libraries that old code depends on has a vulnerability discovered. They provide a patch, but it's only for the current version rather than the one used 10 years ago, which means that it's not as simple as just rebuilding Python 2.
* A new attack on a cryptographic component, network protocol, etc. is developed and suddenly defaults previously considered safe are no longer considered safe. As a great example, how many old things broke when major sites and services like PyPI or GitHub deprecated SSLv3, TLS 1.0, etc.?
* One of the Python libraries you use has had a vulnerability which was only recently discovered. The maintainer quite reasonably does not provide unpaid support for free software which they updated years ago.
What all of those have in common is that the problem is less the vulnerability than the fact that you are likely to need significant amounts of unrelated work getting other components updated to the point where you can install the patch. Given how often vulnerabilities are discovered which have been present for many years, this is likely to happen for any sufficiently large code base.
Of course it's not going to get new vulnerabilities, but that doesn't mean hackers won't find previously unknown vulnerabilities that have always been there. Looking through the CVE database, core python alone has had 15 CVE reports in 2020 and 2021, and for old pythons those will never be patched.
I'll never understand why people think they can write some software and it'll stay pristine forever. Even if it was had no known bugs at some point in the past, the world is a harsh place and your program, its dependencies, its programming language or even its OS can be found to be subtly broken at any moment. Even if it is not outright buggy, the world will change around you until the interfaces you depend on are no longer there. (looking at you, software program at $WORK that only works with PS/2 keyboards but not USB keyboards)
In the old days, software was like math - it had immutable, provable qualities. In the last several decades software has gradually become more and more like biology, where the core mechanisms are shrouded in mystery and we rely on observation and experiments to understand. The Star Trek game in Basic I typed into a computer is eternally true, a platonic form in a way software now no longer even aspires to be.
I'm sure it is a fine program, don't get me wrong. But citing a Basic I program as something that is eternally true is exactly what I mean when talking about the world moving on. Does it even run on 64-bit operating systems? Will it keep working correctly after the unix timestamp goes beyond its signed 32 bit maximum and overflows? I'll also eat my hat if there is anything you can prove about it in the modern meaning of formal verification.
There is plenty of software being written to very high standards of quality (formally verified real-time control software for rockets/power plants/etc comes to mind), but most of that is not available for free on the internet (or for $5/month as a SAAS) because you get what you pay for.
NASA software is very good, but not in general formally verified, I think (based mostly on reading postmortems on various NASA failures).
Even TLA+ analysis and proof of correctness doesn't stop things like "new zookeeper clients are pushing a lot of new writes => disk space needed for logs increases => across all the nodes in the zookeeper cluster => the entire formally correct cluster keeps falling over now but it didn't last week"
I meant more in the sense of the proofs in TAOCP - these are programs that don't interact with anything outside of the program itself, no non-specified input, so a lot simpler, no dates, no actual machine memory, just a little synthetic world defined by the language itself. Even TeX is imagined to be unchanging after Knuth's passing, and is basically unchanged since the 70s.
For sure the world has moved on, but we have lost the ability to have confidence in the software and have had to develop tools and ways of thinking inspired by biology.
If it is dynamically linked (probably the case, as that’s the norm on Mac OS), it might even get new vulnerabilities.
The combination (old Python, new shared library) could have vulnerabilities that neither (old python, old shared library) nor (new python, new shared library) has.
Many of those CVEs will not impact your application code, though. This is just a side effect of Python having a huge and somewhat crusty standard library.
I think, striving for software to not have a maintenance need if no new features need to be implemented is sound. I am very well aware, that the complexity of any but a trivial piece of software on a trivial piece of hardware is beyond human comprehension.
The general approach of most projects and vendors at the bottom of the stack to not accept full responsibility for their part is striking. The amount of lying, faking, leaky abstractions, adapting but not 100% etc. is so immense it is a wonder we get something done at the higher levels of the stack at all.
I cannot comprehend, how it is possible, to hammer the RAM in such a way, that you can affect neighboring cells charge until the information interpreted based on the amount of charge in the cells is changed. This should never ever be possible. I haven't seen any such disclaimer this was possible on the package or in the data sheet of any RAM module. The same is with changing the kind of magnetic recording in the same hard drive model without notification (or change in model name that would be more honest). And then there are "subtle" problems, like coil whine on mainboards or PSUs for no reason - we know how to fix this. These high pitched sounds can give you headaches without you hearing much or anything. There is no indication about it on the package and almost no review tests this. And the whole Meltdown and Spectre type of bugs is just the cherry on top with major performance regressions because of it. So how are you supposed to write reliable software on top of all this if just a tight loop might actually change the state of hardware in such a way that is becomes a problem? I mean, this is just crazy.
Vulnerabilities are going to come up, absolutely. Unless you believe that the older version is somehow 100% bug free and vulnerability free just by virtue of it being old. Being old doesn’t grant it any magic powers.
What’s old now is what was bleeding edge years ago - it hasn’t been updated (otherwise it wouldn’t be old), it has just gotten older.
And the problem with keeping old versions around is that when (not if) someone does discover a vulnerability, what are you going to do then?
If your answer is “we’ll patch it then”, then that’s not going to be an old version anymore, is it?
If your answer is “I guarantee it will 100% never have any vulnerabilities ever and so it will never need to be updated”, I don’t think we’re living in the same reality.
You leave out that even if software A itself doesn't change
1. The OS and hardware under it can change and lead to new vulnerabilities.
2. Dependencies of Software A can change, which can lead to new vulnerabilities in Software A.
3. It's easier to find more and more vulnerabilities on a target that does no longer move. And once they are found there will be fewer experts to detect/fix/report it as time goes on.
There was some recent news about a vulnerability in pkexec that's been around for 12 years. Who knows what Python 2 exploits are still undiscovered in a similar manner? Since it's no longer actively supported, attackers have all the time they want to find those problems.
Why is this so downvoted? I really missed the days of python 2.7, I spent far less time debugging breaking version changes and spent more time on my actual work.
For security critical work, like web services, it makes more sense to constantly upgrade to the newest version. But for scientific work, which is a huge audience of python users, who often use obscure poorly supported libraries there is a lot of value in a stable foundation.
> I really missed the days of python 2.7, I spent far less time debugging breaking version changes and spent more time on my actual work.
This has been the Python 3 experience for years. If you had lots of previously ignored bugs in your Python 2 around string handling, it took longer but on a clean codebase it was often only a matter of minutes.
And yet just the other day on the python 3.10 migration in arch stuff broke and I wasted a good day hunting the error down. The solution was installing python 3.6, which should hopefully work for a year or two until THAT is left unsupported.
Kind of like how various Python 2 releases broke stuff and required time to fix? It's easy to forget that if it wasn't memorable but, for example, I remember the specific 2.7.9 release for that reason.
Sibling post mentions the kext breakage happening as well as another example, but I feel like they've been slipping on this as they've had increasing difficulty matching major features to their unbending yearly release schedule. So more and more (and this is the case with macOS 12 also, see Universal Control) we've seen big headline features announced at WWDC keynotes not manage to make it into the .0, but rather trickle in over the .1 to .3 releases. Once that rubicon was crossed having more other major/breaking changes start sliding into point releases doesn't seem too surprising I guess, even if the cultural slippage is irritating.
Apple probably has to either give up strict semantic versioning for breaking changes, give up their strict release schedule, or be willing to defer announced features that don't make one major version all the way to the next. Something needs to give. Looks like they've decided to go with a "semantic-lite" where breaking changes and major features aren't the rule with point releases but can still happen in old enough/deprecated areas.
It does seem more needless though in cases like this granted, unlike other examples it's not really clear why they'd do this now rather than 12.0. Not as if Python 2.7 was a spring chicken last year either, support ended in 2020 and that was like 11 years after release already? Heck, why not 11.0?
It's not necessarily strict semantic versioning. However, Apple has always had a distinction between major and minor updates. Minor updates are installed on most Macs automatically, whereas major ones are not. Major updates have longer developer preview and public beta periods. Major releases continue to receive security patches for a couple of years.
If Apple isn't going to follow any types of rules, then what's even the point?
I kinda view them as using a modified version of semantic versioning where, if we take A.B.C, the normal roles of A and B are combined into B, and A is the big annual release with the media hype and advertising and shiniest features at the forefront.
Meaning A and B bumps could be breaking, but C is still just for bugfixes.
Sure it's not a great system from the technical side of things, but it shouldn't be shocking that Apple puts marketing first.
> It's implied in the fact that Apple has different types of updates in the first place.
As far as I understand, Semantic Versioning is not a synonym for "versioning scheme", but refers to a specific scheme (https://semver.org). So I don't understand how Apple having any versioning scheme implies that it must be semver.
I edited my post slightly. I never meant to imply that Apple follows strict semantic versioning—and in fact, they clearly don't, since they stayed on one major release for well over a decade.
It's okay if Apple's "minor releases" occasionally have breaking changes. But I don't think "wait until the next major release to remove an entire language runtime which has been built-in for 15+ years" is too much to ask.
Semver is a particular set of rules for defining what is a patch level, what is a minor update, and what is a major update. Before semver, these ideas still existed, but different groups had different ideas of what they meant.
That is perhaps still true in some large companies.
> it's not really clear why they'd do this now rather than 12.0. Not as if Python 2.7 was a spring chicken last year either, support ended in 2020 and that was like 11 years after release already? Heck, why not 11.0?
I don't think introducing breaking changes in 2020 (with the world turned upside down) for 11.0 would have been a good idea. The same reasoning applies to 12.0, many countries were going through a Delta wave without having much vaccine availability this past summer.
I think Apple has been very reasonable about this - they've been clear that 2.x is going away, they have been extremely accommodating of changing circumstances, and 2.x can still be installed manually after this change.
Yeah, and it’s been years or decades since you wanted to use the bundled software on MacOS for stuff that was important. Them removing Python 2 probably will fix more problems due to multiple Python installs than it will break anything. I tend to get a lot of support questions on “how to make this Python thing work on my Mac laptop” and I have seen some screwed up things.
It’s not semver, but I’m willing to bet (although I haven’t checked) that there were probably a few leftover system apps that didn’t get moved to Python 3 before 12.0 (many of the features were delayed, I’m guessing due to the pandemic their work schedule has been disrupted since Big Sur was feature frozen).
But see, if that is the case... Apple is a large company with lots of resources and knowledge of their internal roadmap. If it took Apple's own internal developers until now to remove their dependence on Python, then third-party developers and their users should get at least until next fall anyway.
(I'm not defending the use of Python 2. I'm arguing that, as a general rule, if something takes the vendor X time, users should be given X + Y time.)
It was deprecated back in 10.15 this didn't come completely unexpected. Using it since then always had the risk of it getting removed in a major or a point release.
Major release are already such a pain in terms of just how many dev ecosystem things break every time. Doing it in a minor release is really a blessing in disguise -- all you have to deal with is this.
Let's say I have a little utility shell script I wrote ten years ago. I basically haven't touched it ever since. Inside the script is one line of python for fetching the current time with more precision than `date`, or something, which I forgot about a long time ago.
When I install a major update, I test all my scripts to make sure they still work. I do not test after every point release, and I shouldn't have to!
You should be testing early and often and not relying on arbitrary OS version numbers (Apple disagrees with you regarding the utility) to provide some kind of api stability or assurance.
This is why docker and other hermetically sealed packaging and languages (golang) systems work by avoiding the OS implementations which must change and update as quickly as possible.
The fact anyone would rely exclusively macOS level implementations and somehow expect them to remain stable and backwards compatible for any stretch of time blows my mind.
macOS isn't quite semver unfortunately. The mid year releases are usually quite significant.
While I'm not a fan of removing Python 2 mid major cycle, it's not without precedent, and they've had popups since the first Monterey release anytime anything linked against the system Python 2
Apple has been warning people for several years[1]. It's continued to be included in macOS until they could resolve dependencies with shipped software. And Python 3 is still there.
Removing python2 is great, but as someone who writes an awful lot of scripts and makefiles that need to run on Mac & Linux, I'd love for them to update bash and coreutils.
It's not fun discovering `uuidgen` has different args because my coworkers have a 2007 version of the program, or they think `pushd` works in a cross-platform makefile. Repeat those problems for every GNU utility.
A tiny little bit of GPL3 software won't kill them, and surely shipping up to date utilities is easier than backporting security fixes over nearly two decades.
My personal opinion is we should be clear about why many companies are not contributing to or shipping GPLv3 software.
The schism over GPLv3 was widely known and discussed at the time the license was being drafted. Many companies told the FSF they would not be able to continue shipping or contributing under those terms. The FSF chose their own path and moved their projects to the new license, with predictable consequences.
I happen to think it's an unfortunate footgun that set back the free software movement but I can see why others might disagree. I also agree that the FSF was within their rights to make such changes. The FSF bears responsibility for the decision to prioritize perceived "anti-TIVO" concerns over unification of the free software movement.
This seems like a lot when you're one brew install bash/zsh coreutils away. I have a script that installs all the GNU utils, and does the thing so that the you can use the g versions by their normal name. I haven't even noticed any of the outdated userspace.
> A tiny little bit of GPL3 software won't kill them, and surely shipping up to date utilities is easier than backporting security fixes over nearly two decades.
I don’t disagree at all but I’m not a lawyer at Apple, so I can’t speak to their general risk or not.
The good news is that the zsh included is at least recent and it is (thankfully) trivial to install and alias a newer version of bash, but I have totally been bit by this myself.
Yay! Glad to see this rather than the Red Hat approach of continuing to support dead versions (yes, I know that enterprises value stability blah blah blah)
When I was at Apple, they discontinued bundling the command line version of emacs, and they sent a long internal email stating why. The stated reason was that any newer emacs would suffer from license incompatibility with macOS, and they didn't see the point of supporting a 10 year old version that the FSF doesn't even support when, if someone really wants emacs, it's just a `brew install emacs` away.
I find it kind of frustrating that Apple would use that as a rationale without taking efforts to support homebrew. If they didn’t like the way homebrew was engineered, they could have afforded to port dpkg or something similar. If they did like homebrew, they should have hired on the core developers.
It’s less true today than it used to be, but Mac was the best Unix developer box in the corporate world. A huge portion of their sales (especially high end sales where they make the most profit) relied on these community efforts.
I’ve moved over to using nix instead of homebrew and it’s so much better that it makes me wonder how it ever got this way. I think this is how.
The funny thing is that Apple uses Homebrew internally. We had custom taps and it was part of a lot of setup documentation in our Confluence.
I think the issue is that Apple isn't "one" thing. There are thousands of engineers, each with their own opinion, but also a thousand lawyers and corporate executives who are very concerned with how every action will make Apple "look" as a company. Sometimes their decisions made sense, sometimes they don't.
They hired the original lead developer of Homebrew, which seems to me like support. He chose not to continue working there and seems to now be selling NFTs of his old tweets and git commits, but it seems hard to pin that on Apple.
I'm assuming, of course, the idea was to let him spend at least some of his company time working on Homebrew.
What I don't get is why they bundled that version for so long. It would have made perfect sense to stop bundling it when emacs switched to GPL3. It just caused annoying issues where trying to use emacs in your terminal would bring up this ancient version that would potentially conflict with your .emacs.
What I see when I open a terminal is this:
The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
I believe I switched from the default zsh to the bash that also came with the operating system. I don't think I installed a new bash.
I was just trying to add on to/respond to the question "Will they completely remove bash, rsync, eventually?" . My answer was too short and I didn't actually make it clear what I was saying. I think the answer to the question "Will they completely remove bash, rsync, eventually?" is yes, at least for bash, I think they will remove it. My evidence for thinking they'll remove it is if you switch from the default zsh to the bash that comes with the operating system, they'll kindly ask you to switch back to zsh every time you open the terminal. Yes, you can change the default from nagging you with BASH_SILENCE_DEPRECATION_WARNING, which is great to know.
I don't really see the problem with this though; people who are familiar with what bash and rsync are will probably understand computers well enough to install them with Homebrew or Macports or something.
> people who are familiar with what bash and rsync are will probably understand computers well enough to install them with Homebrew or Macports or something.
This is probably somewhat optimistic: there are a lot of people who are not developers who still use those tools — I've supported scientists, librarians, analysts, etc. who had a few scripts using tools like that but weren't really comfortable making changes. That doesn't mean it's _hard_ to install Homebrew but it's something which they'll need a little pointer and possibly institutional approval to do.
That's fair, I guess it has a potential to affect non-geek users at some level if they're using geek-supplied scripts.
I guess I just feel like the net good would be to de-bundle these non-compliant tools and get them to install upstream versions from Homebrew or MacPorts or something.
It's annoying, and I'm not one of those people that tends to be the "OMG JUST RTFM IT'S NOT THAT HARD!!", since if people are struggling with it then it is "that hard", but I feel like there isn't a solution that can satisfy all parties with this.
It seems that we either break the workflow, which makes a few of the aforementioned parties for a bit, keep bundling the old versions of these tools and leave potential security problems in there, or somehow forcing Apple to change their OS to be GPL3 compliant. For better or worse (probably worse), that last one isn't going to happen, so it feels like the lesser of the two evils is to stop bundling software that won't be getting future support.
I agree that plenty of people who are not developers use those tools but I also don’t think the lift of installing homebrew or macports is going to be too much for people who are already smart enough to run those scripts in the first place. Like you said, it just requires a little pointer, which hopefully people will be happy to provide.
The approval thing is valid, but I feel like that is likely going to be an issue that becomes more and more common as Apple moves to move more and more tools and utilities to the Xcode tools download anyway. Support/approval teams are just going to need to adjust to that reality, even if it’s an annoyance because I don’t see things shifting back, at least with Apple.
But issues of approval aside, I honestly think back of myself as a teenager, installing fink and Macports and figuring it all out as I went along and I wouldn’t have called myself a developer then. I was just a girl who wanted to tinker with stuff.
I think we (and I’m not talking about you specifically but the more general “we” as in the types of people that frequent HN) often underestimate the abilities of our non-dev brethren. We overestimate how much users know too, of course, but I do think we often slot people who have shown that they have the capabilities and desire to solve a problem into a “lesser” box just because they don’t carry the title of developer. And that affects those users too, who then assume they can’t do something when their own track record proves otherwise.
Definitely. I work a lot with librarians — that's a field which isn't considered “technical” but then you look and there are people writing shell scripts to process collections, writing tools in languages like Python to manipulate metadata or work with APIs, etc. and that distinction starts to look pretty arbitrary. (This is actually a big retention problem in the field: someone who learns that in college / on the job is going to look at the pay multiplier they can get by applying for a developer job)
It means I can’t go to help someone with their laptop and use emacs to look at their cruddy code :) but covid ended that sort of help anyways, now I have to watch them type in screen share and see if they know about tab completion and shell history thru the up arrow.
Part of the magic of macOS was a pretty UNIX that just works.
Sure, I can install and deal with the burden of a package manager, but I don’t want to and didn’t use to need to for basic tools. I want Apple to do this.
If you’re installing your own shell making sure it doesn’t conflict with anything else, you might as well run Linux.
Personally, I'd like to know why Apple decided to ban GPLv3 software in macOS. It's not locked down enough to trip the TiVo clause[0]. My ongoing guess has been "just in case we need this in iOS", but these tools are all things that Apple would never ship in iOS anyway - the whole point of that OS is that you don't get access to any of those tools[1].
Alternatively, they might have just taken the Linus Torvalds tact of "this changes the deal", even if that change is minor. But that's not so much a license compatibility problem as much as it is a difference of opinion. Which is something that might never actually be resolved. Thanks to the v3 split, "GPL" now means two different licenses based on RMS's personal opinions on what restrictions best preserved software freedom at that time[2]. Even a hypothetical GPLv4 that dropped the TiVo clause for the sake of appeasing Linus and Apple would be changing the deal for people who wrote v3 programs expecting them to be a bulwark against locked-down devices.
In either case, it does not make sense for Apple to ship decades-old utilities that it refuses to update for legal or moral reasons. At best, they're dead weight - it is very much common practice for people developing on macOS to install the newer versions of those utilities themselves. At worst, they are an actual security hazard, as they aren't sandboxed in any way. Bugs in those utilities would be very much exploitable on macOS. So if we've decided "we're not going to update this ever", then we might as well get rid of it and replace it with something else.
[0] The TiVo clause in GPLv3 is actually not that strong, BTW. It requires that, if you ship the software as part of some hardware, that you provide a way for the owner of that hardware to modify the GPLv3 bits and have the hardware still work. The only thing that would remotely trip the clause on macOS would be SIP, and that's also owner-overridable.
You will lose iOS app support in the process on M1 Macs, but the TiVo clause doesn't cover that scenario - which is ironic because that's exactly how TiVo worked around similar language in GPLv2.
[1] Apps like iSH or a-shell do exist, but they run x86 or WASM binaries within the App Store sandbox. They also don't trip the GPLv3 TiVo clause.
[2] At one point, RMS was seriously open to the idea of just rolling the Affero clause directly into GPL, which probably spooked people way more
I wonder if it has something to do with the Busybox lawsuit. I think that it might be more to do with "we don't want to rock the boat with licensing...GPL2 has worked for us, and we don't want to be forced into a corner like Linksys was with Busybox".
This is entirely speculation on my end, IANAL and I didn't work in that part of Apple.
In perpetuity? Why? In forty years they should be using a 55 year old version of Emacs and a 45 year old version of Python? I respectfully but firmly disagree.
Very few are using it, it's not getting updates, and is a potential security hole because of the lack of updates, it seems to me to be smart to get rid of it. Most people wouldn't want to use that ancient version of Emacs, and would install a newer one via Homebrew or GNU Emacs for OS X.
As noted on the GNU emacs site[0], you can download prebuilt binaries here[1], if you really can’t be bothered with ‘make install’. Homebrew is hardly necessary (for most things.)
I think it was a "removing from future versions of macOS" situation, not removing from existing installs. If your mac is old enough to not have homebrew, it's probably too old to install Catalina due to the fact that they stopped supporting devices without Metal support.
Also, feel free to replace `brew install emacs` with the macports equivalent and the point still stands.
Well sure, there are alternatives. I was commenting on the casual assumption that everyone has brew. Brew has worse support for older Macs than a lot of other software.
RHEL and macOS are two very different products with two very different audiences and can hardly be compared like this.
That being said, Python 2 will last through the end of RHEL 7's lifespan through June 2024, and that includes the AppStream module in RHEL 8 which will similarly end support in June 2024. After that customers are recommended to upgrade to a supported version of Python or continue using Python 2.7 at their own self-supported risk, we will not support them in that endeavor.
RHEL 9 (due out in a few months) does not ship Python 2.7.
With Ubuntu, if you delete Python, it obliterates everything. Have fun re-installing your system. Learned that the hard way. You can bet I "use Arch btw" now.
I don't see how that's relevant here, unless you interpreted my statement to mean we will be removing Python 2.7 in RHEL 8. That isn't happening, the module will remain in the AppStream repositories but will not receive any support.
Removing the system Python is prevented on RHEL/Fedora (with an exception). The package manager, DNF and YUM, are marked as protected which prevent their removal. You'd need to remove the protection file from /etc/dnf/protected.d before you can do that. Removing the system python would remove the package manager which depends on it, therefore the transaction is stopped. Using the RPM utility directly also will not work unless you a) pass --nodeps or b) list out every single dependency at the same time manually. You can test this in the CentOS containers with the following:
It’s not just “value stability”, there’s a good chance that without them knowing some part of their billion dollar enterprise absolutely depends on an old python component that would break if 2.7 were removed.
They know that at some point, their RHEL version runs out of extended support. Two years before that they’ll experiment with the new RHEL version on their test environment and notice everything is totally broken. And then they’ll have two years to fix it which is just barely enough time, as “fixing it” requires buy in and planning and resources from dozens of divisions and hundreds of people.
To enterprises, having Red Hat support some ass backwards old version of whatever package is not just “valued”, it’s exactly why they pay millions to Red Hat instead of running some free Linux version.
The way I have always understood it - the Python that RedHat supplies is intended for it's scripts. If we install our own packages or remove any packages, then the risk destabilising the system. I have seen situations where people have removed python packages that resulted in yum no longer working, for e.g.
Hence, I have always (including from the RHEL 6.x days) provided my own local python version by compiling and packing as a custom rpm. I now use venv to do the same.
Actually RedHat no longer ships Python since RHEL 8, instead they have platform-python for internal use of the operating system and you need to explicitly install the Python version you want
This is a RHEL 8 only feature, RHEL 9 will have a global python3. Whether alternative version are offered as modules or as side-by-side installs like in Fedora, I do not know.
Sure, and perhaps in that context "deprecation" is the right description, but in general I've only heard that word applied to an API being in-process for removal, not as the descriptor of the final removal itself.
Deprecate, from Latin deprecari, "to pray (something) away".
It would seem the gods eventually answered!
OT: it's the same precari that is in precarious, which originally meant something more like "dependent on the will of another". Both definitions have loosened nicely, but it's still fun to see how we got here, and to think of marking something Deprecated as an act of piety!
I just hope that they don't make the same decision as Debian where they made `python` launch `python3`.
It's really useful if `python` is simply removed, like they did in Ubuntu. There you'll know immediately that your script which invoked `python` was about to execute a v2.7-script.
Else you might get errors which only crash the script when a rarely used path is executed.
Only 14 years after 3 was released, and over 2 years after 2.x was truly killed off for good , but better late than never.
Bundling a dead technology that users couldn't remove or update made using actual normal python borderline impossible without being forced into a third party software manager.
Shipping with it is not the same as "if you uninstall it, which you can't even do unless you boot into safe mode and turn off kernel lever application protection, your OS is now broken beyond repair and you'll need a full reinstall", though.
When I introduce young people to coding, it's always great to know their Mac has Python and Ruby on board, and we can quickly drop into a Python shell, or create a trivial .py file, and have some fun.
Just use repl.it? Honestly it's even easier, you don't have to think about dependencies, you just "import numpy" and woah it just installs it for you when you hit run!
M1 MacOS 12.2, never installed any dev tools / homebrew
python --version
Python 2.7.18
python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
It's possible there was a script lurking somewhere under /System/Library/CoreServices that might have needed Python 2. The script got rewritten/snipped off, and there you go.
Now, it's included in the CLI tools. You know how when you run 'clang' or any of those, it's a wrapper that offers to download the real tool for you? That's how the 'python3' binary works now.
Is there any chance that they’ve stopped confusing everyone by aliasing gcc to clang transparently? That’s the change I really want to see. The programs inherently accept different arguments. The amount of GitHub issues that boil down to someone thinking they’re calling GCC but calling Clang instead is staggering. Let my build fail.
Thanks, but I was trolling the shameless, senseless and topically irrelevant plug for homebrew. Get homebrew! Don't miss out! Everyone's doing it! All the hip youngsters know!
EDIT: According to one of the commenters xattr is now a binary instead of a Python script, so that's probably the real reason why they got rid of 2.7 now!
They've done some contentious things in point updates but this is really going to put the cat among the pigeons, especially with the "alternative language" jab. Way to rub salt in the wounds :-)
About bloody time - the sooner we move on from the Python 2 to Python 3 fiasco, the better. I get hives every time I see a nice project on Github, recently updated, with "Tested on Python 2.7".
Great! This is a security thing, it sunset two years ago, PSF does not even provide security updates any more. There is no good reason it should remain on user computers.
Python 3 released in 2006. 2.7 was sunset two years ago (after a 5 years stay from execution). No one can claim they did not have enough time to make the switch.
Sorry for my short comment. I'm glad you understood what I meant.
For anyone else who reads this, strings and str() went from being arrays of bytes in Python 2 to Unicode strings in Python 3. That change broke a lot of programs and is one of the most criticized Python 3 changes.
It's about time. Python shouldn't be in the OS at all, and definitely not for core system utilities that need a working (and small) interpreter with few dependencies that can be easily upgraded.
The way the Python community handled the deprecation and EOL of 2x was a total disaster. Python isn't (or wasn't) only a toy language. It was being used for real, production code, in many cases for a decade or more. That's not to say that upgrades should be impossible, but (except for security patches), a language is such a core piece of infrastructure that you can't change it in an incompatible way without breaking working systems everywhere, and for no reason.
It was truly bizarre that a new, backwards-incompatible version was created really just to fix a missing unicode datatype and add async. Both of these could have been added to the core language or as a library. In fact, I'd argue that even if adding that to the core language was quite difficult (and I don't believe it would have been), it doesn't matter, because you don't break production code.
Python was a nice language to read and write, but considerations of long-term community support, your ability to support deployed apps, a commitment to backwards-compatibility, packaging, and a focused and workable deployment story, are all eminently practical concerns that (in many cases) completely override any otherwise nice features of the language.
The strange behavior of the python community seemed to indicate that they're not at all interested in the billions or trillions of lines of production Python 2x code out there, and that if you are going to eventually upgrade the underlying OS, you now have to completely rewrite all of those lines, even if the original developer isn't even there anymore and the app was working just fine. It was the Y2K problem all over again, but this wasn't due to short-sightedness or byte thriftiness while coding from 20 years prior, but simply because the developer bet on the wrong horse! But what a pretty horse..
Some people still say, "oh, but there were ten years before it was EOL'ed", but the reality is that Python 3000 was never spoken of as being the next (completely incompatible) version of Python, but instead was an experiment, until one day it wasn't anymore.
And then it became Python 3. And, far, far worse: the PSF took an extremely aggressive stance about enforcing the trademark and not allowing a fork (even maintained by others) of Python 2x. This is extremely antithetical to open source principles.
Python's pain was largely self-inflicted, and Python proved that it really only wanted to be a hobbyist language. Some languages might have aspirations of being a 100 year language, but apparently not Python.
Many people (including me) have now spent millions of dollars ripping and replacing Python code. Sure, we should have seen the writing on the wall back in the Python 3000 days, but, surely, we might have thought, didn't the PSF just see what happened with the Perl 5/6 debacle (a similar backwards-incompatible split that split the community and basically killed Perl) a few years prior and weren't going to make the same mistake?
> It was truly bizarre that a new, backwards-incompatible version was created really just to fix a missing unicode datatype and add async.
That's a very reductive way of looking at what was in reality a far more complicated situation, that required a tremendous amount of engineering to even arrive at the compromises they came to. It's not very charitable to all the folks who worked very hard to make that transition even happen.
You think they would have actually gone through all this pain and trouble to add a unicode type and some async? You'd think if it were that easy, MS, Google, Dropbox, and friends would have just banded together, slapped on some unicode classes, and called py2.8 a day.
The problems necessitating python3 run far deeper than that.
