Hacker News new | past | comments | ask | show | jobs | submit login
Numpy: Plan for dropping Python 2.7 support (github.com)
662 points by AndrewDucker on Nov 15, 2017 | hide | past | web | favorite | 375 comments



Good. The glacial migration from Python 2 to 3 is one of the worst things about an otherwise fantastic ecosystem. The tide is turning though, with Django having already dropped support for 2, and now with Numpy too hopefully Python 2 can be properly consigned to the history books.

For people wondering why it's been like this for almost a decade(!) since Python 3.0 was released: Python 3.0 was actually terrible. It lacked many critical things, and was riddled with bugs and security flaws until 3.3, which was released 4 years after 3.0. This cemented in many people's minds the idea that Python 3 was a bad thing. Python 2 was and is a fantastic programming language, so it was a very high bar to move up from. Python 3 has come a long way since then, Python 3.6 is now a decent step up from 2.7.


Frankly, I still haven't seen a single reason to switch to Python3 beyond the fact that the original authors have gotten bored of providing security and bugfix updates and will stop in 2020. That's it.

The only thing in the last decade or so of Python3's existence that even got me slightly interested in using it was asyncio, and after looking into it a bit, it frankly seems like more trouble than its worth.

I know Python 2.7 extremely well. It works perfectly fine for just about everything I use it for. It's STABLE. For the tasks I use it for, it runs faster than Python3... (Not that performance is my greatest concern when using Python.) So, please tell me -- why on earth is it a good thing that people are trying to kill off Python2? What exactly makes Python 3.6 a "decent step up" from 2.7? I'm still at the point of thinking, as you said, that Python3 is a bad thing.


why on earth is it a good thing that people are trying to kill off Python2?

Because the community is fragmented and that weakens language adoption, productivity, and enjoyment.

The 2-3 schism in Python has been a pain to deal with for years. I use Python casually here and there, but I'm so sick of trying to do something quickly in Python and finding out that I'm on a machine that only has 2 but the module I need is only for 3 or having the bulk of my app written in 3 but the only version of a module I need is for 2.

Same goes for examples you find on the Internet to do a particular thing. Ooops, it's 2.x syntax/modules so you have to fix it up for 3. Ooops, it's 3 syntax/modules so you have to fix it up for 2.

For the good of any computer language, old versions need to eventually die off.


>For the good of any computer language, old versions need to eventually die off.

I would say instead: for any good computer language, new versions need to retain compatibility with old versions. Every single system I run has python 2.7 (including my brand new Macbook running latest OS X). Luckily I don't need numpy, and I can probably do without python at all if I have to.

Compare to perl: I can run old perl scripts on basically any system without having to worry about it breaking. I do all of my server scripting in perl for this reason.


Perl6 is incompatible with Perl5, so they're also going through a similarly painful transition.

Everyone waxes lyrical about how Python 2.x was "good enough and why would you change it", but there were several things in Python 2.x that were objectively awful (unicode was broken, iterator variables can leak to the outer scope, division of integers producing integers, xrange, raw_input, mixed indentation "working", etc). And while no single issue alone would justify Python 3.x, the combination of all of these issues as well as some other semantic problems justified the need for Python 3.

Of course, that being said, Python 3 also had some issues that it took several versions to iron out.


I am wondering. Why is division of integers returning an integer an awful thing?


https://www.python.org/dev/peps/pep-0238/

{Describing the old python-2 behavior:}

-------- Quote: -------

The classic division operator makes it hard to write numerical expressions that are supposed to give correct results from arbitrary numerical inputs. For all other operators, one can write down a formula such as xy*2 + z, and the calculated result will be close to the mathematical result (within the limits of numerical accuracy, of course) for any numerical input type (int, long, float, or complex). But division poses a problem: if the expressions for both arguments happen to have an integral type, it implements floor division rather than true division.

-----------------------

To guarantee the correct mathematical behavior in python2, one would probably have to write:

    def true_math_div(a,b) :
        if type(a) is int or type(a) is long :
            a = float(a)
        if type(b) is int or type(b) is long :
            b = float(b)
        return a/b
as a and b could be int, float, complex, or some object defining the method __div__.


What's wrong about just `float(a)/float(b)`?


not everything that can be divided can be reasonably cast to a float


Also, it's ugly as sin


I wouldn't call it awful, but it is slightly annoying. In a dynamically typed language it's hard to know apriori if a variable is an int or a float with a whole number value and you end up having to write x/float(y) all over the place just to make sure your code does what you want it to do.

The new case of / always being float division and // always being integer division just makes everything more explicit.


This particular language quirk I don't think has anything to do with dynamic typing: it's equally annoying in C-style languages where 3/2 and 3/2.0 mean different things.


sure, but in C if you have a line that looks like

   z = 3/y
you'll know that y is either always a float or always an int depending on its type and thus you'll 'know' what z is.


Because it's a duck that doesn't quack.


Perl 6 is a different language. Last time I checked there was no plan to discontinue Perl 5.


Wont they have a naming/versioning problem?


No. Perl 6 has its own version scheme, and the number at the end of the language name is just a number.

I'm not a particular fan of that - they should have changed the name entirely, IMO, eg. to Rby - but Perl 5 is not getting discontinued.


They have it already. Perl6 is about 10 years old, had at least 4 virtual machines. When it was announced people thought perl 5 was obsolete. And in the same time came RoR. A few years(like 7 maybe?) ago a some new developers took over perl5 and added some really interesting new features. That made some perl6 developers unhappy, wanting to focus community around perl6. Perl5 developers believe a completely new language should have a different name. Some suggested renaming perl5 perl7. Then there was a website perl11.org redirecting visitors to scala-lang.org. That was about 3 years ago. Since then - I don't care:) Perl6 will have to find itself a niche. The one once occupied by perl5 - a ducktape of the web is now taken by js. So, a long transition is better than a remote perspective of unknown transition.


Good point, I didn't realise Perl6 was in a similar situation. However it seems that adoption of Perl6 is very low, so I suspect Perl5 isn't going to get killed off any time soon.


Very little new stuff is being developed in Perl, period. I don't know anyone working in Perl who is not maintaining existing codebases. I haven't heard Perl proposed for anything new in over a decade. That further reduces the motivation to move to Perl 6.


It depends what you're using it for. Certainly it wouldn't make sense to use it for web stuff. However I use perl exclusively for server-side "shell" scripting, and it excels at that. More powerful than bash, less compatibility worries than python (I've had issues even between 2.* versions). If I have perl5.* on a server my scripts work everywhere. I regularly start new server-side scripting projects using perl, including a recent webpage-to-video conversion batch script. To be honest there is no other platform that I would even consider using for this kind of thing. Is there even anything that comes close to perl's cross-platform, rapid development, stability and general compatibility?


I dunno, on Arch I've seen several times as Perl 5.x updates roll in, Perl scripts will break on stuff. Especially Perl 5.22 and 5.26.


That's because of binary API changes, and only affects Perl modules with C parts in them. Source compatibility has been preserved at least across the entire 5.x line.


Python?


The plan for Perl6 is to eventually run Perl5 code with the `use v5` statement. You just don't throw 30 years worth of CPAN into the garbage bin. Meanwhile, Perl5 is being actively developed, new minor versions are released at least once a year, people are actively improving the language via extensions and even the core is evolving slowly but steadily. I don't thing it will be abandoned any time soon. Perl6 is being developed separately, mosty by other people.


Perl6 has nothing to do with perl5, it's a completely different language. I would call the transition fatal, not painful.


> I would call the transition fatal, not painful.

It's really sad. In retrospect they certainly should have named it something different. The Perl 5 community could have progressed, maybe even made a Perl 6, while the NGPerl skunkworks project continued independently for 15 years.


I never quite understood why the division of integers resulting in integers was a problem.

I get that for people with no knowledge of programming whatsoever it can be confusing, but it's standard behavior in nearly every typed language.

In C/C++, you divide an Int and you get an Int. If you want floating point division, you divide by a float. Problem Solved.

Almost EVERYTHING else could have been done with slow migration, or simply documenting the odd or quirky behavior.

Iterator variable leaking is just a result of poor programming.

Raw_input vs input could have been solved by slowly deprecating input. Or just left as is.

xrange is the same story.

It all seems like someone just decided to throw their hands up in a fit, throw their toys on the ground, and make a new Python. I enthusiastically jumped at Python 3 in the begining, then ran into problems, and crawled back to my 2.7 and decided I could let others fight the stupid cult war that was coming over 2 vs 3. I'd rather use a tool that works than change all my code over someone's idea that newer is better.


> I never quite understood why the division of integers resulting in integers was a problem [...] but it's standard behavior in nearly every typed language.

And hence why it's a problem in Python. If you have a function like

    def foo(a, b):
      return a / b
What are the types of a and b? You don't know. Sure, you could check with isinstance, but now you've broken the illusion of duck typing. The argument is that Python should just do "the right thing". Just because 3/4==0 in most languages doesn't justify repeating that mistake. Not to mention that float(a)/float(b) -- aside from being incorrect in the general case -- is ugly as hell.

> Iterator variable leaking is just a result of poor programming.

I think it's a language bug. Why? Imagine if you could do this in C:

    for (int a = 1; i <= 3; a++)
      foo(a);
    printf("%d\n", a);
People would think this is clearly a language bug because it breaks the scoping semantics (not to mention common sense). Now, you could argue that Python supports this sort of nonsense so it's "fine":

    # a is not defined
    if something:
       a = 1
    else:
       a = 2
    print(a)
But the reason that works is because otherwise Python would need to have JS-like "var" syntax to declare a variable. Personally I think that'd be better, but the for-loop situation is different.

> raw_input vs input could have been solved by slowly deprecating input. Or just left as is. xrange is the same story.

And this is how PHP ended up having three different string escaping functions for SQL statements. The first one (escape_string) was broken, so they added real_escape_string, and then they needed to handle connection-specific quirks so they added mysqli_real_escape_string.

Some people still use escape_string even though it's broken -- and the language devs cannot fix it because "that would break users".


Serious question, what was the complaint against xrange?

I don’t recall it ever surprising me or behaving poorly.


The issue is that "xrange" does the right thing, but the seemingly-more-correct (unless you happen to know about "xrange") "range" does the wrong thing (create a list rather than a generator, which is inefficient). Same thing with "raw_input" doing the right thing, but "input" being completely wrong (and an attack vector because it's just a glorified "exec").


And instead of behind-the-scenes lazily creating the range, so you have a generator that reifies when its modified and has a nice __str__ etc they go make incompatible changes; got it! ;)

I know rather too much about how CPython and other VMs work under the hood and am in no mood to try and save the day any more. I still use python, but the latest stuff around MyPy and async io just make me despair frankly. I think rust+go will probably pick up a lot of people who used to care deeply about python. So it is.


Perl is a weird language language for you to cite when we're talking about languages that do major transitions well. Perl 6 was created because Perl 5 was an evolutionary dead end, took 15 years, has no adoption, and Perl 5 usage shrunk tremendously in the meantime. Not saying this glibly as I love Perl and I hope Perl 6 sees some adoption, but come on now. Also remember that Perl 5 itself was nearly a complete rewrite from 4.


Exactly my thought regarding the need to evolve as a language. Perl is mostly dead because it didn't keep up with the times and I say that as a person who spent a tremendous amount of time learning and using Perl. The Schwartzarian transform was a beautiful thing to behold.


Perl is born to solve portability and limitation problems of shells (csh, sh, ksh*), awk and sed. People who do not live on the command line can not appreciate the power of perl.


> The Schwartzarian transform was a beautiful thing to behold.

Meh, more like a clutch for systems where you couldn't install List::UtilsBy. (One of my favorite Perl modules of all time.)


There's 17 years between the introduction of the Schwartzian Transform (1994) and List::UtilsBy (2011 according to CPAN).


Huh. I guess it's one of those cases where once you have a thing, you wonder how you could have gone 20 years without it. :) Although, to be fair, I only started working with Perl (beyond one-liners on the shell) in 2012.


There's also no push, whatsoever, to "get with the parrot program". Larry Wall is not holding a gun to Perl 5's head.


Unlike Python 3's relation to Python 2, Perl 6 is genuinely a new language. It would make little sense for the community to discontinue Perl 5 and push for Perl 6.

In retrospect it probably would have been better for the community if they'd given Perl 6 a new name.


Yeah, I really don't get this Python mindset at all. I have out in the wild both Scheme code that's approaching 30 years old, and Perl code approaching 20 years old, running on commercial systems. There's no real need for constant version churn.


Perl example is a bad one given Perl 6 having near zero adoption, and being incompatible with 5. And this is hardly "constant" version churn. This was a major event for Python to fix some long-running issues that needed fixing, but could only be done with an incompatible version.


Over the course of 2014, the "pip" project alone went from 1.4.1 to 6.0.6(!) through 2 minor releases, 1 major release, and 9 point releases, 6 of which were on the same day as another point release (it's called a "release candidate", people). They included regressions such as "segfaults on Windows at every invocation", "freezes if the specific server pypi.org is not reachable", and dropping support for a python version that was less than 2 years old at that point. They also introduced a third package archive format into the ecosystem.

The library ecosystem is the problem, and it's what's driving the language churn.


The python devs learned their lesson and aren’t going to do a major breakage like 2 to 3 again.

https://www.curiousefficiency.org/posts/2014/08/python-4000....


It’s a balancing act. Breaking compatibility creates work, unfixed bugs and design flaws create work, which is more work varies. Retaining compatibility can be, in sum, a harmful decision.


Only if you are pretty lucky with what you write.

Back when I cared about Perl, doing "man perldelta" was a nice way to entertain myself.


That's a really poor reason to kill an old version. To be clear, I FEEL YOUR PAIN. However, new isn't legit just because it's new.

As a non-Python pro, I cannot say why one version over another. However, arguing that the old one is bad just because it is inconvenient isn't valid.

As someone who struggles with versions of Python on my Mac and on production servers (and with code that runs on 2.7 and 3.6+), as far as I'm concerned (as a pragmatic solutions-first person goes), I cannot discern the difference between ancient Python and new Python.

There are really few languages with such impact as Python. So we cannot blame the Python community for this situation as they probably really labored over their decisions regarding compatibility and versions. But in retrospect, I would have preferred they killed off the old version long ago. It would definitely have made life better for the users in the long run.


I think the argument here is that Py3 is better for many people (Unicode-by-default was generally the reason, but these days it's also the numerous language improvements, e.g. async/await). For those who find themselves on the fence with no particular personal reason to go either way, though, going where the others are is a legitimate way to choose.


What we often forget are the large numbers of people who just use Python (or any other tech tool) as a means to an end.

In this case I'm included. Python is not my first, second, or third love. But it is the most available and the simplest tool available to glue things together. CSVs, json, web apis (private and commercial), etc., are all so easy to do with Python.

So my guess is that there are a lot of users who may not even realize the benefits that Python3.x gives vs 2. "We" don't know or care about features we don't need. But we do feel the pain of modules that only work for one version.

In hindsight, I would have voted for a hard break from 2->3 perhaps 2 years ago. I suspect the ultimate human time effort would have been less than we waste now straddling or stumbling with two versions.


2 years ago is still very far into the 2->3 split. The 9 year anniversary of Python 3 is just over a fortnight from today.


*I can feel the pain of devs that have to deal with Python 2 and 3. To me the beginning of the ugly Python 3.0 announcement was too off putting, it stopped my interest in Python and I went back to PHP, Java and later also NodeJS, Go.

In comparison PHP and Java always successfully transitioned to new versions, keeping it backward-compatible and do baby steps instead of a big incompatible cut. PHP canceled the ill fated PHP6 fork, and went from PHP 5.2 then up to 5.6 and then jumped to PHP7 (as several PHP6 books got published about an alpha version).

Language with a rocky transition (mostly due to incompatible syntax) were C#/dotNet 1->2, Perl 5 -> 6, Lua 5.1 -> 5.3, Ruby 1 -> 2, Swift 1 -> 2 -> 3, Rust 0.x -> 1, and more



And? So did PHP 5.x. So do all languages. That's what I mentioned with "baby steps".

You can run PHP3, PHP4 and PHP5 projects with little or no change at all, code dating back to 1990s with PHP7. If you cared a bit and adjusted your code over the years, most changes are announced many versions ago and got deprecated. E.g the original MySQL API had been deprecated for a decade or so years, and only got removed with v7, yet it's easy to update the code to the newer APIs, as it was possible since early 2000s, when the newer API got introduced and stayed unchanged since then. And you could use a shim too.

PHP and Java (and several other languages) have really kept an eye on backwards compatibility, you cannot deny that or paint it in another light.


I was responding to "keeping it backward-compatible", which it did not do. In any case, Python 3 is not exactly a brand new language, many codebases can be adjusted to run on both engines without much effort. I don't think the difference is as stark as you've painted it.


On the other hand, there’s still a lot of code that needs porting. See http://portingdb.xyz


> without much effort

It's an enormous amount of effort. It's been a decade and the end is nowhere in site. Python 3 is a tragedy.


Sure, but the community could continue to reject Python3 instead as it had been doing for years. It seems like it is catching on a bit now, but I hadn't really seen any good reason for it other than the upcoming EOL.


Look at the statement from the numpy group:

The NumPy project has supported both Python 2 and Python 3 in parallel since 2010, and has found that supporting Python 2 is an increasing burden on our limited resources;

That's a real team saying that they just can't support 2 major versions of the language any longer.


That is like the biggest fake argument ever. There is plenty of resources and Python 2 support is neither a burden nor this burden in any way increasing. The are plenty of people ready to step up to continue Py2 support (Even I would be glad to help).

This is a pure political decision based on ideology.


As the codebase grows, you need to maintain 2 growing codebases, how is that not increasing the burden?

Official support will be dropped by 2020, by then you will be relying on the community (who ?) to provide bug and security fixes. I'm not aware of anybody stepping up and declaring they will take over maintenance.

At this point, insisting on python 2 is the ideological "side". There's no practical nor realisitic reasoning behind it. Major parts of the community are moving to python 3 and dropping python 2.

You can stay with python 2 and maintain the language / libraries, but don't begrudge those that move on.


Can you be more precise with what you mean by "the community". It wasn't the community that said it was dropping support.

I've plan to migrate away from py2 by 2020 too, just not to py3.


The resources aren't "code" but people and time. You have a limited set of folks who consistently contribute and become reviewers/committers. This is all based on volunteer time - no one is paying these folks to do it. So, asking these folks to divide their limited volunteered time between multiple versions of python is unfair. I think this is the right decision to take.

If you feel you have "plenty of resources" you can fork the python2 version of numpy and maintain it.


> There is plenty of resources and Python 2 support is neither a burden nor this burden in any way increasing.

Aside from what others have said, NumFOCUS is woefully underfunded. If you're interested in seeing continued development of NumPy and other amazing scientific Python packages, you should think about contributing!

https://www.numfocus.org

(Not a NumFOCUS person although I occasionally volunteer with them and definitely donate on a recurring basis.)


You can fork numpy and maintain Python 2 support.


Python would have lost popularity and would have eventually died without Python 3. Being a scripting language with a relatively low barrier to entry has always been among its selling points. Text processing is a very common use-case for such languages. In a Unicode world, you can't really have a language that is supposed to be easy to use yet requires contortions and has major pitfalls in simply handling text.


Yep, I'm always surprised by the number of people of people here who dismiss the usefulness of unicode. Not "dismissing" the hard way, but simply saying it's not a problem. I understand that we may have a lot of western/english people here but, unicode for me is a life saver. For example, I work on stuff for french speaking people, and I need the euro sign. In that very simple case, unicode already solve many issues : I don't have to convert between ISO-8859-1, ISO-8859-15, CP-1252 anymore, whatever my database uses, I just enforce unicode. Moreover, I can have french comment in source code (that's very important because sometimes we're talking about laws and some of our technical terms in those laws are not translated in english).

(I understand that in this particular case, I could have enforced ISO-8859-15 as well, but the crux of the matter is that with unicode built in python3, I don't have to think about it anymore)

And now my customer is copy/pasting additional information from documents coming from other countries as well...


You do realize unicode has been "built into Python" pretty much since the beginning (Python 2.0, 17 years ago), right?

The main difference is that unicode has a more efficient internal storage since Python 3.3+ (a neat technical detail), and that mixing bytestrings and unicode will fail with an explicit error since 3.0+ (a good idea IMO).

But that Python 2.7 didn't support unicode is simply FUD.


I have moved away from Python 2.7 because unicode support was not good enough.

Not good enough means I had to prefix every single string with "u" to make sure it's unicode. It was especially painful with code like this :

   logger.debug("Name of the person I'm debugging {}".format(name))
forgetting the "u" simply lead to various crashes if the name had characters that could not be encoded in whatever the debug was piped to. Always thinking about the "u" was nothing I had the time to.

Just an example.


Well now you have to prefix b"" instead for bytestrings. The argument works both ways -- there's no magical improvement that happened in Python 3 (except that nice storage optimization in 3.3 that I mentioned above).

It's actually good practice to be explicit about the type, and write b"" and u"" always (easier on the person reading the code). The u'' literal prefix was re-introduced in 3.3 for this reason.


The argument works both ways

It doesn't because strings and text are a lot more common than bytes. Yours is a really weird line of argument - that the 3.x changes and what's in 2.7 are fundamentally equivalent and thus the changes are outright unnecessary and that the people who made them just got it wrong and did it for no apparent reason or benefit. I get that someone might not like what they did or how they did it but your take should give you pause just by its general improbability.


You're mixing up two things: separating string types in the language, and using prefix literals. Completely orthogonal concerns.

As I said, u'' literals were re-introduced by the Python developers themselves, in Python 3.3.


I don't think I am. In this thread you're repeatedly making the point that 2.7 supported Unicode and the difference is mostly technical details of things like internal representation and/or a matter of prefixes or whatnot. This just isn't true. The fundamental change is - in Python 2, strings are bags of bytes and in Python 3 strings are collections of Unicode codepoints and you need to go through an encoding to convert to and from bytes. This is a big (and necessary) deal. No amount of debating the finer points of implementation, feature retention or reintroduction, etc is going to make that difference not matter.


What I said:

1. BOTH Python 2 and Python 3 come with built-in support for BOTH bytestring and unicode (contrary to OP's claims I responded to)

2. That mixing bytestrings and unicode will fail with an explicit error since 3.0+ (a good idea IMO)

3. Unicode has a more efficient internal storage since Python 3.3+ (a neat technical detail)

4. It's good practice to be explicit about the type of literals, and write b"" and u"" always

5. That Python 2.7 doesn't support unicode is simply FUD.

Can you articulate which point you're actually contesting? I'll be happy to clarify, but I'm honestly unsure what you're responding to.


I think almost all of these are wrong.

The person you replied to didn't claim Python 2 doesn't support unicode. 'Bytestrings' has what is wrong with Python 2 neatly summarized in a single word (and this, incidentally, is a term the Python documentation avoids these days because it's bad). 3 is true but not really related to the topic at hand. 4 is, I think, outright wrong. As to 5, I'm not sure why you would even want to defend that. It's not what the poster said and even if they had said it, they'd be just wrong - it's not 'FUD'. That is just you being grumpy and rude.


u"" was reintroduced to avoid people of fixing all their strings (see PEP 414 rationale).

that was my point, by moving to python 3, I removed all my "u", the thing other developers not wanted (see PEP-414 again); I loved the "purity".

but removing u was tedious (at best).

In my use case, strings are "string" and binary are "bytes". Which I think is much safer.


I don't think anyone claimed it didn't support Unicode. Only that it allowed mixing bytes / strings and the default type most people used from the beginning was str. That's a trap that they'll regret the moment they actually need to handle something outside of Latin-1.

Lots of python 2 code out there fails because the default option was simple but bad. I know, because my name broke the CI pipelines in a few different projects.


Did you actually read the thread you're replying to, or are you on auto-pilot?


Let's be honest, A lot of peoples experience with Python is restricted to the North America.

For these folks encountering anything other than ASCII is pretty uncommon.

Personally, I've worked on a Python 2.x project deployed in heavy industry across the globe including Japan and the number of times we had Python 2 unicode nightmare issues was too many to mention.


For these folks encountering anything other than ASCII is pretty uncommon.

Hmmm... [THINKING FACE (U+1F914)]


> Frankly, I still haven't seen a single reason to switch to Python3 beyond the fact that the original authors have gotten bored of providing security and bugfix updates and will stop in 2020. That's it.

It seems pretty clear now that 3rd party library developers are going to stop releasing packages that support 2.x and target only 3.x. Isn't that a bigger problem for Python 2.7 hold outs?

Originally, I was not super excited about Python 3. I liked "print" as a keyword. I liked the space efficiency and speed of latin-1 strings by default. I did a lot of network protocol stuff and bytes() was a pain to use. I knew how to use the 'u' prefix to get unicode when I needed. However, after using Python 3 for a few years now, I find Python 2 clumsy. Print as a function is better. Unicode works better. The implementation is just as fast or faster than Python 2 and getting faster every release. If you tried Python 3 a few releases ago, you should give it another go. It has matured a lot.


> Isn't that a bigger problem for Python 2.7 hold outs?

Yes, but it seemed like libraries (such as NumPy here) were mainly switching because of the EOL of Python2 rather than for any actual benefit provided by Python3. I've considered that as more of "Python 3 is a bad thing" by splitting the ecosystem further, and creating additional churn and rework of existing projects. Thus my question of why people seem to think this is a good thing now -- i.e. what has changed in Python 3.6 that they're happy to do this now when they were pissed off about it a couple years ago?

I've gotten a couple interesting responses here, and I hope for more. I wasn't really aware of f-strings -- that does seems like a nice-to-have feature -- and I wasn't aware of improved JSON serialization performance either.

> If you tried Python 3 a few releases ago, you should give it another go. It has matured a lot.

I may do that.


> what has changed in Python 3.6 that they're happy to do this now when they were pissed off about it a couple years ago?

Most of mine have to do with developer productovity and I didn’t find until Python 3.5.

– @, the matmul operator

– fstrings (f”{foo}”)

– parameter typing (foo: int = 0) which has exciting work with Cython

– other async features. I forget what library it was (tensorflow?) but for Python 3 it had better async support


* Not only is print better as a function (I can now call print in a lambda!) doing anything beyond just printing variables (e.g. printing to a stream/file or suppressing the final newline) is both more straightforward and more readable. Also print(xxx, end="", flush=True) is my bae.

* API-wise keyword-only parameters are absolutely fantastic, even (especially!) for non-default params.

* Extended unpacking, and the ability to unpack in collection literals, make the language much more "expressive" (in the sense there are more cases where you can make do with just expressions) which is very convenient.


Thanks for the keyword-only parameters mention. I'm a Py3 fan so have been looking down this thread for inspiration of features I've missed. I've often wanted to use keyword arguments for readability but without supplying a default - this is just the ticket.


There are some great things about Python 3, and then there are some things that are more subjective. But we have to ask at what cost? Breaking compatibility has required developers to spend huge amounts of time porting and worrying about compatibility that they could have spent on other things. Many of the best features of Python 3 could be introduced in a backwards compatible way. Further, the language itself could have advanced much faster if it stuck with compatible changes. We could have had microthreads or a JIT by now.

Even if you like "print" as a function, is it really so important to wipe out the huge inventory of working Python 2 code and stall Python development for a decade? Python 3 is a tragedy.


> the language itself could have advanced much faster if it stuck with compatible changes. We could have had microthreads or a JIT by now.

That is asinine bullshit and you should be ashamed.

> stall Python development for a decade?

The development of Python never stalled, it barely even slowed.


> That is asinine bullshit and you should be ashamed. ... The development of Python never stalled, it barely even slowed.

Thanks for the language, but the only thing I'm ashamed of is Python's lost opportunities. How many thousands or perhaps millions of hours of developer time was spent on compatibility or maintaining two versions? What if that energy was directed on the improving the language instead. There's no question that it would be further along than it is now.


Er, maybe you never heard of Stackless Python (micro-threads) or PyPy (JIT)?


Of course I have, they were what I was referring to. If we didn't have to deal with this python 2/3 schism, there's a solid chance they'd be part of mainline python by now.


Indeed. Not enough people seem to be aware of the new dict merge feature introduce:

  merged_dict = {
      **source_dict1,
      **source_dict2,
      **{
          'some_key': 'w00t!'
      }
  }


I haven't jumped into Python3 yet (for work/legacy reasons), but I'm really looking forward to nested exceptions and pathlib. Also, even when just working in English, unicode (or special classes for strings) were really annoying when interfacing with other libraries like Qt or databases.


Don't forget how nice super().__init__() is.


The double underscore methods are some of the ugliest parts of Python. If they were going to break compatibility, I'd get rid of them too.


I think you're right, only this wouldn't require breaking compatibility. It doesn't even require a python upgrade, you could just make a base class that remaps nicer named functions to the standard ones, like baseclass.is_lower_than = baseclass.__lt__


You're probably right. Sadly, the vast majority of the python upgrades probably did not require breaking compatibility. Or if they did, they weren't worth it. I understand that iterators are arguably better than lists and print should arguably be a real function, but those advantages are far too small to justify breaking compatibility.


Why is the super().__init__() comment dead below me? In Python 2 you had to do something like super(ClassName, self).__init__(), which is more ridiculous.


This could be because someone flagged the comment. If you believe there’s been a mistake you can vouch for a comment from comment’s page. I did vouch for x14 already—being able to shorten parent’s method calls into super().methodname() minor syntax sugar though it may be is one of the things I’m really looking forward to when I switch to Python 3 at my agency.


pathlib is pretty awesome, too. Has some growing pains, to be sure, but overall it makes paths so much easier than without it.


> Yes, but it seemed like libraries (such as NumPy here) were mainly switching because of the EOL of Python2 rather than for any actual benefit provided by Python3.

Actually if you ever wrote code that you wanted to make backwards compatible with Python 2.7 is a nightmare. The fact that they did not even wait until 2020 before depreciating it is a testament to that.

> I've considered that as more of "Python 3 is a bad thing" by splitting the ecosystem further, and creating additional churn and rework of existing projects. Thus my question of why people seem to think this is a good thing now -- i.e. what has changed in Python 3.6 that they're happy to do this now when they were pissed off about it a couple years ago?

I don't know about others, I always been supporting Python 3 since 3.4. I feel like many people were complaining about Python 3, but never used it, then eventually started using and realized that it is not so bad.

> I may do that.

You should, is much more enjoyable experience. Perhaps because you're so used to Python 2, you don't notice, but Python 2 has a lot of warts that accumulated over the years.


There should have been more innovation, making Python 3 compelling enough for Numpy use to deprecate Python 2 in 2010 for obvious, noncontroversial technical reasons. Instead there has been a thoroughly unpleasant and glacially slow transition between "we have to support Python 3 for bleeding edge users" to "we have to support Python 2 for retro users".


This is an ancient argument because no one can convince you that you need 3 for your use case. All I can do is say why I like it, and list reasons that probably don't apply to you or you would have switched already.

For example, print and division made more sense to me in 3, judging from friends who taught 2 and said those were always sticky for some students in every class. Intuitive lowers the barrier to entry. (But 2 is probably more intuitive to you because it's second nature to you by now.)

Unicode--when I would test some tiny scripts with Chinese characters or weird ciphers--was pretty easy out of the box in 3.

On systems that only support 2 I find myself slipping in basically a 'from future import all the things.'

These are admittedly mostly cosmetics. But cosmetics matter for noobs like I was, or maybe still am.

I guess 3 also fixed ambiguity in corner cases for error handling? Never came up for me so I don't know much about that one.

You probably just don't have those use cases?

Two people could use distinct subsets of python and neither is using it wrong. Meaning... there could be reasons many of us want 3 that simply don't apply to you. Which sucks, because you get hit with switching costs to help the rest of us.

I think that's the recipe for an endless debate with two reasonable sides.

I will say py2 was already fragmented without 3. PaiMei only ran on... 2.4 maybe? You'd find weird projects that you liked that would then get abandoned. Suddenly you're shimming them all or running four versions. I think 3 woke people up to this as a problem--by making it much harder to patch and way more universal. That made the project more conscious about future and backwards compatibility. Those dividends will only be seen over time. I hope they vest but can't prove how or if they have.

I hope this is helpful... Just know that I'm not saying you're wrong to want to use something that works for you. Switching costs are a real thing, I know it sucks to feel dragged along. But py3 is a lot better for me and others, possibly because we're using the language for different things.


It's pretty hard to make a case for transitioning from something that works pretty well to a new thing in general. The nicer the old thing is, and the wider it's been adopted, the harder the sell. That's just the way it is.

Too look at it a different way... If I see the benefit of & would like to make a change, actually making the change competes with all of my priorities. I'm under the impression that Python is used by many people who use reliable things with APIs that don't change often (I mainly of thinking of Bash & some Posix OS). I can see why they wouldn't be fans of making changes.

Personally I like using newer things, all other things being equal. That's mostly because it's easier to chat about recent stuff with people learning the same lessons I'm learning. Almost every time I ask a C/Bash/*NIX question on a Stack Exchange site, the question gets marked as a duplicate, links to a question with answers I had already, but failed to understand. That happens much less when inquiring on newer topics.


The project failed because print was not a function, is not something one hears often.


Unless the project is teaching students python.

EDIT: More seriously py2's print handles parens in unpredictable ways if you have open questions about types, which has been a nightmare for me on multiple occasions.

I trust you that you never encountered them, but I did.

The whole point of my original post was begging for people to realize their personal experiences aren't universal. The py3 changes solve something for us, help us read code and avoid bugs, it's not just a novelty fetish, I promise. Unfortunately you only have our word for it...


The change to string handling justifies it for me. Writing `u’foo’` everywhere is a pain, and the rest of the 2.7 model is extraordinarily prone to runtime errors. With 3.6, I don’t have those problems.


Yeah, I was pretty much sold on Python 3 when I stopped having to use the `codecs` module to read my files and when it would strictly enforce the separation between `bytes` and `str`, rather than having to manually track what sort of sequence I was dealing with.


Yes, yes. My Python life is mostly about impelemnting odd protocols with Twisted. Will probably get serious about asyncio soon, but I have a lot of historical Twisted to deal with ATM.

I've noticed that the "Who cares about Python 3" types are not the people working on the kinds of problems I am. So my conclussion is that for many kinds of problems the benefits of Python 2 are kind of meh, so people stay in their comfort zone. For some kinds of problems, Python 3 is a huge, huge win over Python 2. Whether or not someone is a Python 3 advocate says more about the kinds of problems they work on than anything else.


How is the Twisted Py3 support coming along in practice? It looks like they are steadily ticking items off the todo list, but is it ready for production use yet?


Everything I need is production worthy. That's obviously an idiosyncratic assessment.


+1

Just because we learned to work around unicode issues doesn't mean they're not completely bonkers in 2.7


Most people didn't learn to work around unicode issues during the 2.7 time frame.

Many third party modules tended to explode spectacularly with encode/decode errors, when they were fed non-ascii strings, when used by people who write using all these funny characters.

Just the correct handling of strings in Python 3 alone is a hell of a reason to switch to it.


Let's not forget the joy of third party libraries that wouldn't fail, but rather silently convert, your data because they had hidden ascii asumptions...


I have a contrary anecdote: for the sorts of work I do, having to distinguish between strings and bytes makes using Python3 more difficult, and more prone to errors.

I mostly deal with network protocols, lots of numeric content, and scarcely any non-English text.

But I appreciate other people have different needs.


I feel like the answer to this is "then use bytes everywhere". Why doesn't that work?


That's akin to saying:

> For the sorts of work I do, having to distinguish between lists and dictionaries makes using Python 3 more difficult and more prone to errors. I mostly deal with lists, and scaresely any dictionaries.

Just use bytes where appropriate.


Yes, and while performance on some tasks has gotten worse, on others it has got much better - JSON serialisation in particular, which was huge for us. Python 3.7 will be faster again in many areas.


https://eev.ee/blog/2016/07/31/python-faq-why-should-i-use-p... is a fairly good summary. There are loads and loads of syntax improvements (I personally love the new unpacking), new features, and other enhancements.

Sure, you can continue using Python 2, and it's going to work for you. I guess you can also keep using Windows XP (no reason not to use it beyond the fact that Microsoft has gotten bored of providing security and bugfixes), and code using your PS/2 keyboard...


USB for keyboards is slower (noticable for some types of gaming) and does not support over 6-key rollover. So PS/2 is superior (for keyboard, not mouse) and not dead. Ever noticed around 2010 motherboards stopped shipping with PS/2, but today most for desktops come with a single PS/2 port again?


> and code using your PS/2 keyboard...

That's not even a hipster thing, it's quite mainstream with gaming keyboards...


> it runs faster than Python3

Generally speaking, Python 3 is actually faster now. I am a bit surprised that no one else has commented on this. There was a talk about performance recently (https://www.youtube.com/watch?v=d65dCD3VH9Q). To sum it up, some parts are slower and other parts are faster, and the reasons depend on two questions:

1: Is it using a lot of small ints? Python 3 changed int from being small int to long int, and for work which deals with massive amount of ints this will result in a slow down. If for some reason you want to use a pure Python implementation of AES rather than using hardware acceleration (generally builtin to the CPU) or a C implementation, or the one built in the linux kernel, then you will hit the performance test that get the biggest negative difference between python 2.7 and master. Then there is numpy which uses C modules that can happy do things as small ints.

2: Bytes -> Unicode. Libraries that are Unicode unaware will run faster than libraries that are Unicode aware. The feature to understand that "ö" is a Swedish letter and not several characters does cost some CPU time. For parsing where per character manipulation is relevant (like say HTML), such parsing will be a bit slower in python 3.

Practically everything else is faster now in python 3.


> What exactly makes Python 3.6 a "decent step up" from 2.7?

The language and the standard lib improved a lot. There are so many improvements, it would be impossible to list them all after so many years of progress...

Just to name a few of the bigger improvements, that I have discovered recently:

* The typing module lets you add type annotations to your code and write code like this:

    class User(NamedTuple):
        id: int
        name: str
        age: int

    def remove_user(u: User) -> None:
        # ...

    fred = User(123, 'fred', 42)
    # ...
    remove_user(fred)
* F-strigs are awesome:

   file_path = f'{base_dir}/{user_name}/{latest_dir}'
* I think asyncio and the async/await syntax are great.

* The refactoring of the subprocess module:

    subprocess.run(my_cmd)
* The documentation of the standard lib improved a lot, if you ask me.

This is just off the top of my head...

But really, it's the overall improvement of the language and the standard lib that make the difference, not just the big features.


Frankly, I still haven't seen a single reason to switch to Python3

This typically means one of two things:

1. You've never really looked at the features added in the Python 3.x release series, or

2. You develop abandonware which plans never to upgrade any part of its platform ever, for any reason, and so no conceivable new feature would be sufficient to convince you to do an upgrade.

At this point (1) is untenable because of how many people have written about the useful things available in 3.x, and (2) is disingenuous (but there are still plenty of people who use "don't see a reason to upgrade" as an excuse for "I always planned to abandon this").

Just in case you're one of the people in (1), here you go:

https://eev.ee/blog/2016/07/31/python-faq-why-should-i-use-p...


The irony of calling software that still runs decades after it was written "abandonware" is probably lost here, but I enjoyed that.


Just adding my random piece of anecdata:

docker-compose uses Python2 and has encoding issues on Windows that have no real workarounds other than upgrading to Python3: https://github.com/docker/compose/issues/2775


For anything non-trivial, 95% of the value is in the library ecosystem. So long as most prominent libraries kept releasing new features for Python 2 and 3, there's inevitably not a big pull factor to upgrade. That's changing as a number of major libraries start to make releases that require Python 3.

From a library maintainer POV, I do want to use Python 3. There's no one killer feature, but rather a bunch of small ones, like more specific exception classes (FileNotFoundError etc.).

But if you want to keep using Python 2.7, no-one will take it away from you.


> For anything non-trivial, 95% of the value is in the library ecosystem.

Toy programs and rapid scripting and prototyping...

FWIW I love my trivial use of python.


Until you've used them for a while, you won't believe how pleasant f-strings are. It's worth the upgrade.


Nobody is forcing you to stop using Python 2.7, but the rest of the world is getting over it and moving on.

I'm sorry if that seems mean, but in the grand scheme of things the differences between 2 and 3 are pretty trivial, and most people still complaining about this are just being stubborn. I refuse to believe somebody can know Python 2.7 "extremely well" but then also need more than ten years to learn the few areas where Python 3 is different.


The killer feature of Python3 is string representation. IO work in python2 (e.g. network programming) was made more complicated by the evolutionary mess of python2's string design.

If you want to get strong at network programming without spinning up on all the complex topics in async io (coroutines, futures, etc), you may like my networking system: github.com/solent/solent-eng

It is designed to allow the programmer to reason about exactly what the process is doing, rather than to hide things away.

The docs are not in great shape at the moment. Good starting points: the telnet client (in tools) and the snake game (demo package).

This was originally in python2. I moved because of subtle improvements in python3 packages over python2. But once I appreciated the string changes, I wished I had moved much earlier.


You should give Python 3.6 a try and you will see.

There are many minor improvements and new functionality that working with it is so much enjoyable and make code more readable and shorter.

I got to a point where it feels like a chore whenever I have to use Python 2.7.


Nick Coghlan has written on this extensively: http://ncoghlan-devs-python-notes.readthedocs.io/en/latest/p...


Surprised nobody posted this yet: https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-...

It's a presentation by core developer Brett Cannon which explains what's better about Python 3. The list is quite long for an HN comment, so I'm not going to repeat it here.

Now it's pretty dated as there had been more things added like async, so Python 3 is even better now.


Optional type annotations did it for me - I don't usually use them, but I find them very helpful on a few types of tasks.


Could you please elaborate on this?


You can write type annotations in 3. The benefits there are the same that come with most optional typing. https://docs.python.org/3/library/typing.html


There's all the pain of python 2's string/byte handling... is it a byte str? Or unicode code points? Python 2 likes to keep you guessing, and that's not a good thing. If you haven't felt the pain here, there's a good chance your stuff is subtly broken in ways you may not be aware of.


Moreover there were some packages even last year that were simply broken with Python 3.x. What am I supposed to do if I have to deliver something to a client and the Python 3 version of my stack is broken? I do not necessarily have the time to debug the issue and fix it and it is much easier to switch to 2.7 and just continue working on the business problem. Software guys (especially guys working on programming languages and environments, tools) have to understand that providing a broken alternative is not an real option at all.

Anyways, I haven't been running into issues with Python 3 recently and I use it as much as I can.


For me, it was Unicode - it was much easier to move to Python3 than trying to retrofit my Python2 code base to use Unicode.


> What exactly makes Python 3.6 a "decent step up" from 2.7?

Proper handling of Unicode, and more sensible distinction between bytes and characters.

Seriously, Python 3 has done that _so well_ that I wish other languages would take notice.


You must not have looked very hard then.

Python 2.7 has just about the worst unicode handling of ANY language I've ever used.

Plenty of otherwise crappy languages (Java, JavaScript, etc) managed to make one right choice when they decided that strings are always unicode and bytes are a different data type entirely.

Also, the asyncio implementation in Python 3, while a bit complicated, is on the whole very nice, providing a lot of flexibility.

Long story short, if you still think Python 3.6 is "bad" in this day and age then you need to move on to another language.


It's a bunch of smaller things, but I think the major one is the new compact dict implementation. Using less memory is a good thing and you basically get it for free using Python3.


Python2's unicode support is kind of an error-prone mess and that was what prompted the decision to make 3 a breaking change in the first place. It is much harder to shoot yourself in the foot with text handling in 3 IMO, though it's certainly still not perfect. Of course lots of people think their programs with broken unicode handling work fine until someone passes them data in any language other than English.


If you ever need to deal with encoding, you will very soon know why my friend. I've hated Python3 just as much but reality left me with no choice. It's time to say goodbye sooner rather than later.


As a 2.7 user, I drool over f strings.


My personal reason that may be relevant for quite a few people. Not quantifying that any further, just my 2c.

https://news.ycombinator.com/item?id=12930793



What was your reason for changing from 2.6 to 2.7?


There were no breaking changes in the language.


Because python3 is the new version.


Wild speculation (I'm relatively new to Python), it might also have to do with the Python community having several personalities.

Python 3 solves a lot of problems for me, as someone who does a lot of NLP work, and generally has to deal with strings from the outside world and multiple languages and all that on a more-or-less constant basis. I imagine it solves some problems for Web developers, too, though possibly to a lesser extent. But I don't see a whole lot of devops people being eager to jump off of Python 2, and I'm not sure I see it solving more annoyances in that domain than the process of migrating to Python 3 would create, either.


Agreed. My background is also in NLP and I made the switch to Python 3 early and enthusiastically because it resolved a lot of issues around working with multilingual text.


Same here!


I think there were also more compatibility issues prior to 3.3 (when they started allowing u"") for libraries that did want to support both. It seems like the core team predicted that the most popular migration path for libraries would be `2to3` or something like that, when in fact the single-codebase-supporting-both strategy has been much more popular? Probably the transition would've been easier if everyone had seen that coming.


That's correct. If something like "six" would have been provided as part of 3.0, things would have went more smoothly. There could have been some language changes in 3.0 to make single-codebase easier. It was only later that the core developers realised that would be the normal way for libraries to support Python 3 (e.g allowing u prefix on strings in 3.x). Quite a bit of time was wasted while this got sorted out. Things are a lot better now. Soon, I think most people will stop worrying about 2.x backwards compatibility.


> Soon, I think most people will stop worrying about 2.x backwards compatibility.

Numpy is certainly not the only project with a plan to sunset Python 2 support: http://www.python3statement.org/#sections30-projects


Yeah, for the longest time everyone kept using library support as an excuse for sticking with 2. But at this point I think it's the exception that a decent Python library doesn't support 3, rather than the rule.

The only one that comes to mind for me is fabric, to be honest.


> But at this point I think it's the exception that a decent Python library doesn't support 3, rather than the rule.

Right now 187 of the top 200 packages on Pypi support Python 3. https://python3wos.appspot.com/


Infrastructure changes to PyPI made it hard to work out what the top packages are. http://py3readiness.org/ is another effort, which currently lists 345/360 packages as Python 3 compatible.


And of the 13 not migrated, 9 are Mozilla packages, so they're probably just waiting to write them all in Rust...


Those are internal tools at best so it won't matter that much for migration.


That list is of the top packages. If internal Mozilla tools are downloaded enough to become top 200, then either Mozilla is larger than I thought or Python is smaller than I thought.


Download counts -- which usually power "most popular" charts -- are extremely misleading, because usually every run of an automated CI system triggers a download.


Arguably that works better as an indicator, as you can see which packages are being actively developed with. But that's beside the point.

Are you suggesting that Mozilla's CI system runs so often as to artificially inflate the download numbers of their internal only packages to the top 200 Python packages? Because if so, then my point still stands. Either Mozilla is a lot bigger than I thought or the Python community is a lot smaller than I thought.

I also am of the belief that if any team would think to cache their dependencies, it would be the team behind a modern browser.


Keeping in mind it's been a few years (my last day at Mozilla was in mid-2015), and I was in the web dev org, not the browser...

The sheer number of platforms, variants, etc. of the browser and the number of test runs that need to happen are mind-blowing. So it's entirely within the realm of believability for me that the mozbase packages (which provide a lot of the foundation for all of that automated infrastructure) could hit the most-downloaded list. It's not that Mozilla is necessarily that much bigger than other big-name tech companies, it's just that Mozilla open-sources everything by default and puts most stuff on PyPI.

As an aside, though, I think people do tend to underestimate the scaling stuff Mozilla deals with. Once I got to give a conference lightning talk pointing out we could probably claim the highest-traffic Django deployment in the world, for example (the version check done by Firefox on startup, if you're curious, is or at least was a Django-backed service). Though I'm pretty sure Instagram has taken that title now; back in 2013 when we talked about that we "only" served on the order of a billion requests/day :)

Even things like MDN -- which is what I worked on in my time there -- presented interesting challenges. Building a wiki with the kinds of features technical writers need, that's still responsive and fast to build and render the pages post-edit and capable of handling the traffic of a top-300-ish (Alexa currently puts MDN at #107 worldwide) site, isn't entirely simple to do.


Why are internal tools showing up in the top 200 PyPI package list?


I've wondered that too. Are they automatically downloaded by a Firefox build script, perhaps?


I have always been interested in that cycle. We use 2 because NumPy supports it still. NumPy supports it because we use it. If a major library had dropped support earlier on, I am curious what would have happened.


I bet the same dynamic happened with Linux distributions including Python 2 as the default as well. Double whammy.


> The only one that comes to mind for me is fabric

Speaking of Fabric, does anyone know of any Python 3 projects similar to it? I've been using Fabric3 since the switch but it's not a 1:1 port. I'd consider dropping it in favor of something better if it exists.


The Fabric dev(s) seem to be developing a v2 branch with Python 3 support: https://github.com/fabric/fabric/tree/v2

But I'm not sure how far along it is yet.



Twisted port is a work in progress.


Ah, forgot about twisted. The new asyncio stuff probably changes a lot for how something like that could be implemented.


It's not just libraries that I use when developing.

Searching for code snippets and copy/pasting stackoverflow answers can easily take more time out of my days than library documentation. But it's so much slower when I have to rewrite something I found for python 3.


Yes, but during the long migration, it was no mere excuse.

Almost every time I tried to use a library, it was 2.7 only.

That situation has only changed recently, and with it people have moved on to 3. (Which sort of underscores that the library support was the main issue).



Not having a story for interoperability between 3 and 2 was incredibly shortsighted. The migration path from Objective-C to Swift is an example of how to do this sort of transition right; Python got it wrong.

Widespread adoption of Python 3 would've happened years earlier if you could start a new project in Python 3 and not have to worry about libraries you want to use not supporting it yet.


The Swift 1->2->3 breakage has permanently poisoned the ever-important well of StackOverflow answers.

For some tasks, there are literally 5 mutually incompatible syntaxes to understand (Obj C, Swift 1234). Even though the UI can sometimes autoconvert for you, it doesn't help at at when you're looking at someone else's code.

Swift is a great example of what not to do.


Django has not dropped Python2 yet. Django 2.0 will drop Python2 but that has not yet been released, and Django 1.11 (with Python2) support will be supported in LTS for several years.


Right, though no new python2 compatible features have been added since January.

And Django 2.0 should come out December 1st, so it’s not far off.


In stark contrast to the brutal python 2-3 dichotomy, I feel compelled to express my gratitude for the degree to which — contra all the hating and churn and “fatigue” — es6+ (transpiled to es5 via babel) has become such a pragmatic and obvious center of gravity.


Yeah, why isn’t that part of the python ecosystem? Is the reason that unlike the browser, it is possible to upgrade the major version, so there wasn’t that driving force to support an ecosystem for transpiring?


Python2 is the new Perl.

There are gargantuan Python2 projects out in the wild which are expected to continue, and can't realistically be migrated to Python3. In 15 or 20 years we'll be looking at those jobs the same way we look at Perl jobs today.


If this is the case then wouldn't re-naming it Python 4 shake the negative view of it? Guess it's too late now for Python but I've seen it done in other open source projects and it worked pretty well.


"Python 4?? People have not even migrated to Python 3 yet!"


Or just be like the cool kids and skip a version. Python 5, or name it after the year. Python 2020 sounds exciting enough to make everyone want to upgrade.


Python 3 was codenamed "Python 3000" during design and development.

(It was always intended for the finished product to call itself Python 3.0, though.)


Python 3.4+ doesn't have negative view.


I wonder whether Python itself will improve faster when the development team no longer needs to maintain 2 versions or whether the burden on them will just reduce somewhat.


RHEL7 "runs on" Python 2 and is to be supported at least until 2024, so I don't think Python 2 is going away.


At the same time, Red Hat is aggressively switching everything to Python 3.

http://portingdb.xyz

(Red Hat has been one of the primary maintainers of Python 2.7 in the last few years. https://news.ycombinator.com/item?id=7582300 )

The BSDs on the other hand (including MacOS) "run on" Python 2 and haven't made as much progress switching to Python 3.


Extended EOL for RHEL6, which runs on Python 2.6 is in mid-2024, too. Python 2 isn't going to go away for a long time in the enterprise yet.


We use Appengine, which only supports python 2.7. As well as being a reason for us to stick with 2.7, I imagine Google prefers what is effectively a LTS release. If (hopefully "when") they provide 3.x, they will need significantly more support resources to keep up with continual releases.


App Engine already supports Python 3.


Thanks for correcting me - after excitedly checking the docs, the "flexible" service indeed supports 3.0. The managed service is still 2.7, however.

The flexible service is basically running your own instance of appengine on a VPS, which means you have to provision and support the infrastructure. To my mind, that negates most of the advantage of Appengine - if I was happy to do my own system admin, I'd build my own stack on VPS.


Python is the ecosystem :)


The following example is pretty much the chief reason why I will not be porting much software to Python 3:

  $ python3 
  Python 3.6.3 (default, Oct  3 2017, 21:45:48)
  [GCC 7.2.0] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> 4/3
  1.3333333333333333
Python3 is basically a different language than Python 2. If I wanted to port software to a different language, I would use any number of available languages that make other kinds of improvements over Python as well. The only remaining use case for Python for me will be as a quick scripting language, and data analysis and graphing tool.


It is a change, but you can use the // operator for floor division:

  $ python3
  Python 3.6.3 (default, Oct  3 2017, 21:16:13) 
  [GCC 7.2.0] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> 4//3
  1


I think you've just proved OP's point...you have to change the code.


Alternatively, we need to quite ignoring when projects warn about changing behavior and provide tools to mitigate the change.

  __future__.division was optional in 2.0.0, which was released in 2001

  __future__.print_function was optional in 2.6.0a2, which was released in 2008

There is nothing a language team can do if users ignore changes for close to two decades.

Simply adding the following line to new project over the past 8 years would have made the move to python 3 less difficult.

  from __future__ import (division, print_function)
While I am annoyed that they didn't just alias xrange to range etc...us users do need to take ownership for our bad decisions.


You have to change the code a lot more to port it to an entirely different language.


Of course, but the point is that you shouldn't really need to spend weeks updating your code just to upgrade to a newer version of the compiler.


I know right, you usually have to change a lot of code just to change versions of a framework.

One of my teammates just upgraded our django framework from 1.4 to 1.8 and it took him almost an entire week to do.


A week! It took me six months to port our main app from django 1.5 to 1.8. Though our app is huge, and most of that was migrating from DRF 2.3 to 3.3.


You can use to 2to3 command. So far I had no problems while using it.

https://docs.python.org/3/library/2to3.html


When I use python as a scripting language for small tasks I want something concise and convenient that just works. Python 3 is a downgrade in all those areas. I want to just some input into a string, do some operations on it, and print the result. Now I have to make extra special sure to convert the string between binary or utf8 as appropriate, catch encoding exceptions, not try to print anything that might have binary in it, and even the simple print statement was caught in this impractical mess that feels like it was designed by an out of touch academic committee comprised of people who'd never written real code in their lives deciding that string should actually mean only printable characters within a defined encoding and that statements are considered harmful. I can't wait for return to be a function too in python 4.


I suggest reading the eev.ee article others are linking in these comments. Or Joel Spolsky's "Back to Basics" article. Both do a pretty good job of highlighting some of the complexity in this area, and why things sometimes seem like they're being made harder.

Put most concisely, it's something like "making the most common use case as easy as possible can make slightly less common use cases wrong, dangerous and hard".

It's not an out of touch academic committee making these changes out of ignorance or spite. It's people who have to support more use cases than yours. Even if yours is the most common, that doesn't mean that the rest (internationalization, binary data transfer) are distant, rare outliers, or that the frequency of non-traditionally-"simple" ascii text handling applications isn't changing.


You don't need to do any of that. If you use `input()`, you get a string - all conversions are done for you. If you're reading from a file (or from stdin, treating it as a file object), again, in text mode you just get strings. Same thing for printing - if you print strings, they will get converted to the encoding appropriate for the target terminal.

So, what is it exactly that you're trying to do?


IRC logs with IRC control code characters in them don't work nicely with python 3 text strings, for example.


Why not? There's no restriction on what kinds of characters can appear inside a Python Unicode string.

Or do you mean that the text in the log is UTF-8, but the log itself as a whole is not, because those control characters are mixed into it in (effectively) a different encoding? Then the log isn't a single string, and shouldn't be treated as such.


> When I use python as a scripting language for small tasks I want something concise and convenient that just works. Python 3 is a downgrade in all those areas.

Funny, I found the exact opposite, and despite $dayjob being LCD(2, 3) and the default Python on my system being 2, I've been writing all my small scripts in 3.6.


Python 3 has plenty of objectively good stuff, and some that is definitely more subjective. I have my opinions about the changes, but I don't get outraged over them.

The real crime of Python 3, is not the changes, but the compatibility break. Hundreds of thousands of engineer-hours wasted on porting, worrying about compatibility, and supporting unsupported libraries or two environments at once. In the meantime, the language does not meaningfully advance, and thousands of projects are slowed down. It's such a huge effort for a gain that is not worth it.


To preempt the inevitable discussion of which version to use, please refer to the amazing FAQ created by Eevee: https://eev.ee/blog/2016/07/31/python-faq-why-should-i-use-p...

As a library writer, I cannot wait until we get rid of this self-inflicted handicap of writing in a subset of two incompatible languages. I'm glad the heavyweights are joining the cause.


This is great news. I've been working with Python 3 for a few years now, and it really is a nicer language than 2. Granted 2.7 is a hard act to follow.

Even though the transition from 2 to 3 has been slow, I take it as a positive sign about the Python ecosystem. The general Python community has traditionally erred toward stability, and conservative feature introduction. The gradual introduction of large changes to the core language reflects that characteristic.

Hopefully this will cajole Python 2.7 users to migrate. Assuming you have decent test coverage, the migration generally isn't that difficult.

If you do a lot of NLP or string processing, the sane unicode support is worth it alone.


That’s about the same time python 2 support drops in general. Sounds strange to support packages after the underlying python has moved on.

Kudos to the numpy team for writing the most awesome python module under the sun, and for providing great support!


It's not strange. Python developers might declare Python 2.X EOLed, but that won't stop people from using it. Like it or not, Python 2.X is not going away.


There are big companies with huge legacy 2.x codebases who will stall as long as possible. But popular libraries are dropping 2, and many new libraries aren’t supporting it to begin with. Python 2 is accelerating toward irrelevance.


You make it sound like they're holding out of some weird sense of spite. It's more reasonable to assume most large or small companies still using 2.x have not prioritized upgrading over, I don't know, staying in business?

As large libraries drop 2.x support over time they'll have to prioritize upgrading.


> have not prioritized upgrading over, I don't know, staying in business?

Banks use the very same excuse for a long time, which is why they can't even find Cobol devs to do the work. Bottom line: if having some new grad developer convert Python2 to 3 for some legacy shit would put you out of business, you're already well on your way out.


Sure, and nothing prevents those people from grabbing the last 2-compatible numpy version.


Huh? I'm starting projects in Python 2 because I know for once that the language won't move out from under me. I love it.


How do you plan on getting on when packages from PyPi stop supporting Python 2 at all? When Django, flask, numpy, requests, pandas, and more just don't even install? Will you just use ancient versions?


I don't understand: are you saying package authors are going to retroactively remove Python 2 support from existing versions? Why on earth would they do that?

Scheme that I wrote in the 1980s still runs (and is running today in commercial systems). I have so little patience at this point for this "higher version = better" nonsense.


And what happens when security issues are inevitably discovered in the libraries that are not updated anymore? Sure it's probably OK if you write things that never interface with a network.


Yes, no doubt there's going to be a crippling security vulnerability in this linear algebra library I use. Come on, you don't really believe that, do you?


I hope you don't use cryptography then, or networking, or image/file manipulation, or do anything with untrusted user input.

If you're only doing algebra on an air gapped computer then why do you even worry? Programming languages and tools will evolve but nobody is forcing you to. Keep a local copy of what you need and be happy. Just don't assume any new libraries you might need will support your stack forever.


Still nowhere near as worrying as the business risk inherent in writing on a moving platform.


Even if you are using Linear Algebra, older versions will not be optimized for newer architectures and instruction sets. This is a problem.


If they're not calling my own ATLAS then they aren't "optimized" in any real sense to begin with.


Will you backport security fixes and other significant bugs yourself?


I don't have do that for my 20 year old Perl code or 30 year old Scheme code. Is the bare-bones Python interpreter really that much more full of vulnerabilities?


Also, Python is the only ecosystem I know of where people actually get angry that you use an old and stable version of the software, as those downvotes indicate.


Python might be an especially loud case, but I've seen a fair amount of that in some other areas, for example:

- People getting angry at browser JS authors for coding in explicit support, or huge back-compat shims, for obsolete browsers and/or deprecated JS features.

- People getting angry at Perl authors for writing code that depends on a stable hash iteration order (changed in Perl 5.26 or something? I forget). There was an extremely vocal (tiny) minority of users raising a big stink about the change in that behavior and claiming that it would break the universe.

- C developers being criticized for coding to older (and often more verbose or tricky to use, but omnipresent) pure-POSIX macros and behavior rather than $latest_GNU_or_GCC_feature, which also had the pernicious effect of making things less portable, and damaging the culture of portability.

There are more examples, but you get the idea.


At the SciPy 2017 conference, for the first time I saw speakers who were presenting new libraries say that they were not going to be developing Python2 versions. Quite a change from a year or two ago.


All my libraries have been py3-only since around last year. I'm also not caring much about supporting 2 on the old ones I maintain. 2 is pretty old, it's time to move on.


> 2 is pretty old, it's time to move on.

Why? Just because "it's pretty old?" The 2-to-3 change seems mostly cosmetic. When Python users ignored 3, Python devs started flogging their dead horse. When the horse still refused to move, disappointed riders started shouting "shame!" at users who suggested that the horse looked dead. Eventually, the riders tied ropes to the horse and pulled it along the ground. The onlookers either switched to bicycles and cars, or figured that they might as well ride the man-hauled carcass awhile longer.


That's a very melodramatic comment for someone who clearly hasn't been keeping track of the 3.x changelogs – which pretty much all contain at least one compelling update to the language. Modern Python is v3. Legacy support drop-off is the same with any project.


I was going for "humor" more than "melodrama," but maybe I failed at both... The actual breaking changes between 2.x and early 3.x struck me as cosmetic and pointless.

To compare: Perl went one way by breaking everything to essentially make a new language, and failed to get people to switch; C++ bent over backwards to maintain compatibility, and failed to fundamentally evolve the language. Python basically failed in both ways, changing the language just enough to break most people's code, while changing it too little to effect significant change. It combines most of the disadvantages of both backward compatibility and novelty, with few of the advantages of either.


I'll happily debate the merits of breaking compatibility in general, but it's boring and completely academic for Python 3 which is getting on for a decade old and has very wide community support.

Same thing every Python thread. It doesn't matter if you liked the way 3.0 was handled (and there certainly was complaint at the time too), it's what Python is now, and that's that.


To also compare: Ruby made big breaking changes similar to Python 3 and called it "Ruby 1.9".

It wasn't really a big deal. There was some pain and it became essential to have multiple versions of Ruby installed. But I haven't heard of anyone insisting on new code written for Ruby 1.8.

So languages can break backwards compatibility once in a while, and it doesn't always have to be the shitstorm it was for Python.

Early 3.x was bad, and that probably gave critical mass to the backlash. But there's no reason to care anymore about what early 3.x was like. Modern 3.x is a way better language.


I was an outsider to the Ruby changes between 1.8 and 2.0, so thanks for the perspective. My sense was that, like modern JS programmers, they were used to constant churn and breakage, so having the language break as well was only a minor additional nuisance. Most of the stuff they used with 1.8 would be "obsolete" in a year or two even without 1.9.


Your view of Ruby programmers might be confused with Rails programmers. Those are more used to the "constant churn and breakage" which I also see as coming somewhat from the web side of programming. Although it's less of an issue in Rails as you can have properly stickied versions in a project.

AFAIK there was no breaking change in Ruby after 1.9, so anything that runs on 1.9 should run on any Ruby version up to the most current.

The biggest change in 1.9 was the support of different encoding and making UTF-8 the default encoding. Ruby did it differently than Python, so a string in 1.8 was a byte string, whereas a 1.9 string was a UTF-8 string. This off course broke many scripts.


Python 3 demonstrates exactly how much change a language project can get away with. Python is just barely managing the transition; any more change and it would have ended up like Perl 6. Perhaps future developers can use this data to plan large and painful migrations.


It's a nightmare to maintain compatibility with 2. Also you can't use any new features only because some lazy ass developers don't want to move.

You have a legacy app that you don't want to convert? Fine, use legacy modules, they won't suddenly disappear. You can even user python2.7 after 2020. Want to use something new? Why also not use more recent version of the language.


Similarly, I'd always watch at least a handful of Pycon talks every year. Last year I couldn't really find anything relevant to Python2. The closest thing was and update on removing the GIL...but that was more informational than practical for me.

(I'm still stuck using Python2--not by choice)


After such a long and painful road from 2 to 3, I feel like the Python developers aimed too low in fixing the legacy problems that existed in 2.

All that time for Unicode. Not concurrency or type safety or static guarantees or better lambda syntax or anything fun like that. Just Unicode.


> type safety

The text changes are type safety, but there are many more miscellaneous type safety improvements (like the default comparison between types (alphabetically by type name, except NoneType!) being removed, so now you can use set comparison operators with confidence) – plus support for type annotations.


> better lambda syntax

What do you mean by better lambda syntax? Is there something that you feel isn't adequate in the current syntax?


It's a bit verbose. A lambda with a single one letter param takes ten characters to write: `lambda x: `. In JS, it's 5: `x => `. Ruby blocks take sevenish: `{|x| }`. Six in haskell.


How did you get six for Haskell? I get five if you require a space at the end

    \x->


I was counting a space after the x.


Also, still no support for multi-line lambdas is there?


Multi-line lambdas will never happen. Lambdas are expressions. "Multi-line" means statements. There is no sane way you could embed statements into an expression with whitespace-based block syntax.


Fine, not technically lambdas, but still: function bodies declared inline in function calls. This provides great flexibility in higher-order functions.

> There is no sane way you could embed statements into an expression with whitespace-based block syntax.

Ruby does it. Of course, Ruby has a limitation that a function call may only have one "block", but still - Ruby blocks are statements embedded into an expression in a whitespace-based syntax.


Ruby has explicit syntax to end groups of statements. Python is equally powerful when it comes to higher-order functions; it just enforces a flatter, more explicit style.


But you also got the "print-tax"!


Applications are open for YC Winter 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: