Hacker News new | past | comments | ask | show | jobs | submit login
Where are we in the Python 3 transition? (snarky.ca)
110 points by joeyespo on Jan 1, 2016 | hide | past | favorite | 132 comments

The python 2.x to 3.x change will be studied for many years. How to completely rob an ecosystem of its momentum in one fell swoop. I still don't understand why 3 couldn't have had a mode switch defaulting to 2.x behavior. The whole debacle would have been avoided and all the ugly details would have been hidden behind the language implementation. It would have definitely been more work and you might even call it an ugly hack (and Ed DeCastro would likely have thrown a fit if one had asked his opinion) but it would have at least made the transition instant and seamless.

The present mess will - if not resolved in a much more drastic fashion - cause more and more teams to switch away from python. It's a real waste of the potential that python had (and still has).

I never even bothered porting my code, I'm 'stuck' on 2.7 and anything new gets written in another language. Even bloody PHP, the language everybody loves to hate on, gets deprecation and backwards compatibility better than python.

That's the same approach that IE took for many years - IE7 embedded the IE6 rendering engine, IE8 embedded both the IE7 & IE6 rendering engines, etc. You could switch between them with a meta tag, or with developer tools.

It didn't stop IE from being robbed of momentum.

I think this is more a case study of what happens when fundamental assumptions that a technology ecosystem is based upon change. Things like string & collection types, async mechanisms, interface/protocol definitions, package managers, programming paradigms, developer preferences, and hardware performance characteristics are foundational to a language's definition. After all, the network effect that binds a language ecosystem together is all based upon being able to share code between multiple unrelated developers.

In Python's case, unicode happened. A lot of old Python code was written on the assumption that everything was a sequence of ASCII bytes; it was just plain broken in the presence of unicode characters. There's little you can do here other than accept that much of your ecosystem's strength is gone anyway and build for the future.

I suspect we'll see similar problems crop up in other languages in the near future; in particular, promisification in ES7 is a big threat to the existing Node ecosystem, which is all based upon callbacks. A language that jumps on the promise bandwagon early (Rust? Go? Elixir?) could rob Node of much of its momentum. Similarly, if Android & iOS win out over the web, that could lead to a big resurgence in Java or rise of Swift over server-side technologies like Node or Go.

Have a look at how the Erlang ecosystem managed the unicode transition.

I would take the Python 3 ecosystem over the Erlang ecosystem any day.

Interesting, why?

I'm far more likely to find a library I need in a Python 3 version than in an Erlang version.

Take a look at the PyPI stats for Python3 packages vs. the Hex stats for Erlang packages:



The vast majority of common packages are compatible with Python3. Moreover, a common Python package gets about 100x the downloads of a common Erlang package. Even with only 20% of Python users on Python3, that's still about 20x the users that Erlang has.

Python3 adoption only sucks compared to the installed base of Python2. Compared to any niche language - Haskell, Erlang, Rust, Swift - it's massive, and you're much more likely to find a battle-tested library in Python3 than in your niche language.

(The bulk of my work is actually in Node and Swift these days. Node has a mammoth package library, but that's because its culture encourages many small modules and lots of code re-use. In my experience, I've found that the chance of finding a robust, well-supported Node module is roughly equal to the chance of finding a robust, well-supported Python2 library, and is maybe 20-30% greater than the chance of finding a robust, well-supported Python3 library. The Swift library situation is much, much worse than Python3.)

Agreed on the library part, that's my biggest hang-up in making more (and faster) progress with Erlang projects. Sometimes I even end up rolling my own and coming from languages with substantial 'batteries included' library systems this is a real drawback.

On the other hand, when it finally works it is absolutely bullet proof and that's a feeling I have with very few other languages and eco systems.

The degree of isolation between subsystems and the reliability are absolutely un-matched. Neither is the learning curve :(

Have you tried Elixir? If so, just wondering if you found the learning curve to be as difficult as Erlang.

No, but it is definitely on my 'todo' list.

> It didn't stop IE from being robbed of momentum

IE actually handled the transition very well, because backwards compatibility was the thing that kept it in the game. When it lost it was because this feature no longer mattered (that much).

so python is the IE of programming languages? yikes ;)

Python 2 is, yep. And Java, and C++, and PHP, and all those other languages that were super popular at the turn of the millenium but now feel really dated.

Actually, C++ has an even bigger claim to the title, because it now has "modern" versions that are much better, more standards-compliant, and have language features borrowed from newer languages...but a number of enterprises are stuck with no-exceptions, no-STL, no-move-semantics subsets because that's how all the existing code is written.

The IE metaphor is terrible for programming languages :)

I don't know about PHP, but C++ (and I suspect Java too) is backwards compatible and doesn't belong in the same class with Python. One can still compile older code and start using new features as they see fit, there is no hard break.

> How to completely rob an ecosystem of its momentum in one fell swoop.

I suspect your perception depends a lot on which part of the community you are observing. As far as I can tell, the Python community is as vibrant and fast-growing as ever, with continuing huge crowds at Pycon, brisk sales in Python books, active meetup groups around the world, hundreds of python bloggers, several very active mailing lists, etc.

Most of the change I’ve seen is the rise in other language communities (Clojure, node / server-side Javascript, Go, Swift, ...) which share certain cultural priorities or programming niches with python, and have caused some number of Pythonistas to switch to something else. I suspect this would have happened about as much with or without the Python 3 project though.

> I suspect this would have happened about as much with or without the Python 3 project though.

That could be, of course for me the lens I'm looking through is very much a personal one. Backwards compatibility is a very important concept to me and I really dislike breaking changes, especially when there is no graceful intermediate phase with deprecation and warnings and ample time to move. It's what completely turned me off drupal and python more or less did the exact same thing. I'm sure that for newcomers to python the result is a much improved experience but from my point of view this was an un-necessary hardship.

It seems like kind of a double standard to say "I don't like that they released breaking changes, so I'm going to switch language." Isn't switching language an even larger breaking change?

It is, but if you have to do a rewrite anyway?

The real gem in the rough with Python2 is PyPy4. You don't have to rewrite anything, while gaining CPU performance as a feature and dropping the GIL.

If you're not gaining that, I would probably switch to another language too. Elixir, Go, Swift, among others are definitely worth porting too.

Yeah, I'd argue this article is itself a quite amazing form of denial. The idea that 7 years into the transition, "20% of the scientific Python community and 20% of the Python web community have [switched to Python 3]" is simply abysmal.

Ruby and Python were always going to have problems with computing's concurrent future, but holy damn did Python just tie its shoelaces together and completely unsurprisingly fall on its face.

The fact no one is doing a "Ruby 2, 20% of rubyists finally reaching acceptance" blog post is a testament to how to move a language forward properly. And ffs is Ruby 2.x not more of an upgrade from 1.8 than Python 3 is from 2?

There was nothing wrong with Python 3, what they really messed up is to give so much time for the transition. 10 freaking years!!! That's the problem, people won't do anything unless you light fire under their asses.

Ruby f-cked up with 1.8 to 1.9 transition (notice that it was also a minor version change) yet people migrated without a problem because they knew that if they stay on 1.8 soon their programs will stop working everywhere.

Python did everything they could do to help migrate (it even allows you to install all versions (I know there are no problem with anything between 2.4 - 3.5) and have them run without conflicts side by side.

The only mistake they made is to give too large window for migration. People just now started working on migrating to 3.x because development on 2.7 stopped in 2015 (even though 3.x was available for 7 years now). And there are still people who will wait until 2020 (once security patches will stop being provided) until they will decide to move.

There's so much FUD with Python 3, and goes as far as people wondering which version to use to learn it. I mean learning all the difference is at most as couple hours (and I'm being conservative here). For a beginner the only major difference is that the print is a function and not a statement, the whole unicode is barely noticeable unless you are writing code that should work on 2.x which shows it was needed, because wringing correct unicode code in 2.x is hard.

Ruby 1.8 code does not just "stop working". Just as this 2020 Python2.7 political stunt won't stop anyone's code from working.

The migration window was extended because the core development team didn't want to lose control of Python(2). Someone else would've taken control for sure, and for myself, I've already ported to PyPy4 instead of Python3.

A much easier transition than going to Python3 and nets you CPU performance and GIL-free Python.

What I meant is that as time passes it's harder and harder to get working Ruby 1.8 installation. With RHEL for example you're guaranteed that there will be Python 2.7 until 2024, which delays adoption.

As for PyPy4, majority of people don't chose python for performance. Even PyPy will perform worse than statically compiled languages. If performance is the goal, you should probably look into golang or even C.

You can have the best of every world with Python on PyPy. Most people agree PyPy's performance levels reach the point of "good enough", while allowing you to write pure Python.

Personally, I prefer to write Python over Go or C. No reason to not have your cake and eat it too.

Sadly I have to agree with you. Some things that changed were not even necessary.

Why would the development team make replacement packages rather than fixing underlying implementation, for example imp and importlib.

There was also no sugar added to support this mangled jungle gym of package replacements. Backwards compatibility is a thing of dreams for the python developer.

This is only one of many of the problems with the change.

> gets ... backwards compatibility

That should be past tense, "got" as the Python community has certainly learned its lesson. Luckily it wasn't as bad a mistake as the Perl folks made.

> That should be past tense, "got" as the Python community has certainly learned its lesson.

Evidence of that to me would be a proposed fix and maybe I'm too out of touch with what's happening in the python world but from what I see the two main version still co-exist rather than that any attempt at unification has been made. Am I missing something major here? My understanding is that you migrate your project to 3.x or you're stuck in the past.

> Luckily it wasn't as bad a mistake as the Perl folks made.

That's up for debate.

Python 2 and Python 3 are different languages. The core Python team is working exclusively on Python 3. If you're using Python 2, you're using a deprecated language, with all the pitfalls that entails. It's much like using Visual Basic or Delphi in the mid-2000s: you can do it, you can build some fairly decent applications quite productively, but you aren't going to get any improvements in the future and you're not going to be at the leading edge of technology.

It sounds like you are still in the Denial phase?

No, I'm just lucky that I don't actually need anything that 3.x has to offer. The python code I wrote uses a bunch of modules that are not available for 3.x so that's one reason for being stuck (and they won't be ported either and I don't feel like doing it myself). The old code works and works well enough that I just set it up in a VM where it will happily work for decades to come. New stuff gets written in other languages, once burned, twice shy.

I loved the way I could use python to quickly prototype some GUI based program, but I don't love it that much over that I can't write the same program in a wide choice of different languages with other ideas about long term stability.

Before the tragedy of the Perl 5-6 transition, I don't think people realized how important backwards compatibility is. I understand your fear, but perhaps the most reliable team is the one that's already made the mistake and been burned.

Yes, you are missing that the str as bytes to str as Unicode transition was a tough problem. That's the major incompatibility. Once that decision (mistake?) was made, the break was irrevocable. Perhaps not in theory, but in the practice of how Python 2 folks have used strs.

Aiui the Unicode problem is even more complex than most folk realize for the reasons I explain in this comment.


On the final technical destination implied by Unicode:

The final technical destination implied by the Unicode standard is correct handling of "grapheme clusters".[1]


On the half way point at which we've arrived:

In the 80s, there were a bazillion incompatible "character" sets and encodings of those "character" sets. Unicode was a response to this.

The half way point envisaged by the original Unicode design was to firmly establish a single overall "character set" view with a handful of encodings.

Imo we've arrived at this half way point.


On the string types of languages:

Programming languages designed before Unicode emerged adopted the "byte=character" view to determine the character unit of their default string type, leaving higher level views to library code and functions. This is the level Python began at. But the more established Unicode becomes, the more important the higher level views become.

Newer languages have mostly been designed to optimize for Unicode's half way point (but not the eventual destination). They typically support the byte view in some form but adopt the "codepoint=character" view to determine the character unit of their default string type. Aiui, this is where Python 3 is at.

A handful of newer languages continue to support the byte and codepoint views in some form but have adopted the highest level "grapheme=character" view for the character unit of their default string type. I think the most visible new language that's been forward-thinking enough to adopt the final destination view (grapheme=character) is Swift.


[1] Folks trying to understand this point should find it helpful to start with the deceptively brief hint inherent in http://www.unicode.org/glossary/#Grapheme which defines "what a user thinks of as a character".

Next, I'll quote the perluniintro[2] document:

> A Unicode logical "character" can actually consist of more than one internal actual "character" or code point. For Western languages, this is adequately modelled by a base character (like LATIN CAPITAL LETTER A ) followed by one or more modifiers (like COMBINING ACUTE ACCENT ). This sequence of base character and modifiers is called a combining character sequence. Some non-western languages require more complicated models, so Unicode created the grapheme cluster concept, which was later further refined into the extended grapheme cluster.

[2] http://perldoc.perl.org/perluniintro.html

They got their transition better than the Perl 5 to 6 transition, and arguably better than the Ruby 1.8 --> 1.9 --> 2.0 transitions.

I moved last month and I have been using Python since 1.5.2

What prevents me from recommending it to everyone I pass on the street is that the PyPy 3.x line is back on 3.2.5 which makes it not compatible with many libs which have a 3.3 minimum, which is most. Yes, I give to pypy.org http://pypy.org/py3donate.html

Recommendations to folks

  * switch your daily interpreter to Python3
  * use tox, https://pypi.python.org/pypi/tox
  * use six, https://pypi.python.org/pypi/six
  * when you go native, go cffi, https://pypi.python.org/pypi/cffi

This kills it for me too. I rely on pypy, and I just can't see myself sacrificing my productivity by using Six.

Also, Python 3 neglected to fix one of the most annoying things about the language - default arg value

  def foo(x=[]):
      print x
  foo() # 1
  foo() # 1 1
Why is this still busted??

Are you referring persistence of mutable default args? Which then potentially leads to bugs when you treat it as a new variable on each function call?

This is intentional behavior. It's the result of one-time evaluation of default args, which is important for memoization. It's also really useful in combination with late-binding closures. For example, using a lambda as a generator function:

    def create_multipliers():
        return [lambda x : i * x for i in range(5)]
This doesn't work, you'll get all 8's. Instead you need to:

    def create_multipliers():
        return [lambda x, i=i : i * x for i in range(5)]
Late-binding closures and memoization strategies are pretty core language features; I wouldn't expect them to change (and many python devs would be pretty pissed if they did). Yes, this can be confusing with mutable default objects, but the alternative is to have disparate behavior depending on the mutability of the defaults, which would be an utter catastrophe.


Other languages seem to get it right. Even C++!

    #include <vector>
    static int foo(std::vector<int> x = {}) {
      return x.size();
    int main (int argc, char const *argv[]) {
      printf("%d\n", foo()); // 1
      printf("%d\n", foo()); // still 1
      return 0;

    def foo(x=[]):
      return len(x)
    print(foo()) # 1
    print(foo()) # 2? wtf?

C++ its own insane behavior, where default arguments are determined at the call site instead of at runtime. This means your function can get another function's default arguments.

    #include <iostream>
    struct Base {
        virtual void foo(const char *name = "base") {
            std::cout << "Base impl with " << name << " param\n";
    struct Derived : public Base {
        virtual void foo(const char *name = "derived") override {
            std::cout << "Derived impl with " << name << " param\n";
    int main(void) {
        Base *b = new Derived();
prints "Derived impl with base param"

Fortunately Python avoided that bit of silliness.

This behavior is pretty reasonable to me.

ES6 got this right too.

This design decision makes me die a little every time I have to do:

  def foo(x=None):
      if not x:
          x = []

That will get you into trouble as well, please use:

    if x is None:
        x = []
... etc.

Or even "prettier":

  x = [] if x is None else x

I'm partial to:

x = x or []

Only works when x is truthy for all legal x values, excluding x=0.

I'm not familiar enough with the innards of other languages to make any definitive comments on it, but I will say that the challenge is in combining a high-level language with late-binding closures with memoization. I'm not aware of a strategy for that which is always intuitive for every program you write; I think it's just a language design combination that has a very high probability of developing "gotchas" no matter what you do.

Also, on a related note, I really love late binding and memoization, so I'm definitely biased on the side of "I understand why the language designers made this decision, and though the ramifications bother me a little bit, I much prefer this way".

On a third note (because hey, why not), with type hints being a thing now, maybe it would be worth considering a default eval hint. It could default to current behavior, and therefore retain backwards compatibility, but you could add syntax to declare defaults as new variables. That might be a smart compromise.

Small tip: you can shorten your snippet by completely removing main's parameters and the return 0 and still have a valid c++ program.

I did pause to think about this, but sod's law dictates that if you remove the parameters to main, you will have to add them back in again. :)

It may be an easy mistake to make but using None is more consistent and efficient to detect. You can use "if x is None:" for any type; whereas, a function may or may not want to allow empty strings (or even empty lists) and it is easy to use a None check to decide when to apply the desired default.

Can you (or someone) clarify what problem you're talking about? This seems to work as expected for me:

    def foo(x=[]):
        sum = 0
        for a in x: sum += a
        return sum
    print foo([3,5])  # 8
    print foo(x=[3]) # 3
    print foo([]) # 0
    print foo() # 0
Is it the fact that kwargs are required to have defaults? Or what?

The problem is that the default argument is mutable.

  def f(a=[]):
    print a

  f(['ok']) # ['ok', 'v']
  f()       # ['v']
  f()       # ['v', 'v']

Ah, thanks.

Nah, do it like this

def foo(i,x=[]):

    return x


See what happens

Because nobody's made a Python Enhancement Proposal for what should happen instead.

Why's this downvoted, it seems like useful advice to me.

Also cffi is definitely awesome.

> While some people feel stuck in Python 2 at work and are "depressed" over it, others have reached the point of having transitioned their projects and accepted Python 3, both at work and in personal projects.

It is not as simple.

I would be simpler if it was 2006 again. Python would have less competition on being an easy to use language, batteries included, with good enough performance (SMP wasn't as popular then). Java had a bad perception -- too bloated, enterprise-y, Javascript on the server wasn't happening much. Go wasn't there. Scala not as popular. In that context, people would eventually have switched to Python 3 even if it took some pain (security maintenance, some newer features etc), because GIL wasn't a big deal and it sure wasn't going to be Perl again (maybe Ruby for webdev?).

The situation is more difficult today -- Python has more serious competition. Once a team has found the time/money/excitement to switch to Python 3, it is a high chance they'll start looking and evaluating other languages/platforms/libraries (and I speak as a person who loves Python and used it professionally for 12+ years). It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14. No matter how cool Python 3 is and what new features it brings, it will have a harder battle to fight just because of the current environment.

> It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14.

That would have happened no matter what. It's not really a bad thing, is it? I can't see how something better coming along is anything but great news.

If someone would make a Python 2.8 (or call it Python 5=2+3 if you like), I would pay for it. It must run basically every python 2.7 (or 2.6) program without changes. It should backport as many features as possible. The meaning of "str" and string literals (bytes or unicode) should be configurable on a per-file basis. It should have a print statement. Finally, it should be able to use all the binary modules from 2.7 (maybe after recompilation).

Currently at our work (in physics research), I have the feeling that people are giving up on Python. It's not worth the trouble to port to python 3 - especially because we can't switch all code at once; we'd have to support both versions for years. C++ which is also heavily used here is getting much nicer, with proper tooling it is even more pleasant to work in (code completion etc). We are mostly using Python for a) configuration files and b) quick and dirty scripts. For these cases, Python 2.7 will be good enough forever. If someone does something more complex, they'd probably use Python 3 - but they treat it like it's a completely different language rather than a new version of Python 2.

You'd have to pay a lot.

Jokes of mine like https://github.com/dutc/rwatch and https://github.com/dutc/didyoumean and https://bitbucket.org/dutc/astexpr are examples of adding major features to the CPython interpreter at run-time. It works fairly well in practice. It ends up looking like a `__future__`-import pragma that has interpreter lifetime scope rather than file scope.

`__future__` imports cannot be undone within the same file -- after all, they're pragmas scoped at the file level, not actual imports -- and it would make sense for these backported features to behave similarly, though at the scope of the interpreter's lifetime. That might not do what you want.

As you'd expect, the most superficial of Python 3's new features (syntax changes, `raise from`, &c.) would be easiest to backport. The bulk of the difficulty would be in merging those changes into Python 2.7. Making the result hot-loadable would be just as easy as in `dutc-rwatch`, though being able to toggle features on-and-off independently would require combinatoric effort with the naïve approach. You'd need to do something smarter.

All in all, it's wholly possible to do what you want, and you probably have the skill to even do it yourself.

I'd happily do the work myself, but only for lots of money. Unfortunately, the burden of both certifying the resulting interpreter as correct and of maintaining this fork would be large.

And then when you're on-boarding some fresh new post-grad you've just hired into your research group:

"Oh, we use Python 2.8 here. That's what we call Python 2.7 with an unsupported collection of modules that mutate the currently running interpreter to backport features from Python 3. Don't worry; it only breaks in ways that no one outside of this office will ever have a chance of ever understanding or helping you debug."

Oh, I should also mention that I found a way to embed Python interpreters within themselves. I have a couple of different ways to do it.

Here's one: https://gist.github.com/dutc/eba9b2f7980f400f6287

Here's another: https://gist.github.com/dutc/2866d969d5e9209d501a

I've given, like, a million conference talks about this.

Better than creating some horrible Python 2.8 hybrid would be funding (or convincing me to volunteer to do) the work of completing this bridge. This involves writing 2/3 and 3/2 PyObject shims and figuring out some GIL and GC issues.

Then, within a given host interpreter, you'd be able to execute modules within a guest interpreter of whatever Python version you want.

With the shims and the bridge work, you'd be able to interact with these objects seamlessly from host-to-guest and guest-to-host.

Aren't there issues at the bytecode level? I've been studying bytecode and ceval.c and I know there are significant differences in the bytecode set, as well as in the code object and function object. So what would a "Python 2.8" compile phase generate? Would the huge switch statement in ceval.c have to have a whole new layer of if-logic in it?

Likewise, one of my previous advisors has stopped doing any quick and dirty math in python. I am seeing Julia a lot more (for astrophysics and ML) and more C++ (physics) for the more important simulations.

Julia seems like an odd thing to bring up on the topic of "avoiding backward-incompatible language changes".

As much flak as ruby has gotten for its performance issues and "magic" (read: heavy preference for use of meta programming) ...

One thing I think the ruby core team has done a great job of in my opinion is always having some awesome new features in the next major release that make you say "hey thats awesome" and make you want to upgrade despite any headache it might cause and the headaches are generally minimal.

Its been interesting to see python so publicly and painfully fail at that.

A whole new language version release that as far as I can tell only changed internal plumbing and went so far as to make you have rewrite print statements?

I just don't understand how they could think people would want to move to that.

Maybe Python 2 to Python 3.0 didn't introduce many "awesome" features, but 3.4 and 3.5 have been great.

asyncio and pathlib are absolutely killer features for me. I never much cared about Python 3 until asyncio came on the field. As soon as I saw some example code Guido had written, I started using tulip, and have been a willing participant on the wild ride it has taken.

Pathlib lets me down a bit -- I find myself having to str() the Path objects and lose their benefits, because the rest of the standard library doesn't use pathlib.

But asyncio is great. It shows how much better asynchronous programming can be when you have support from the language itself. I can't imagine going back to a byzantine system like Twisted even if they did manage to finish their Python 3 port.

I'm a long time Python user and I still tend to go for 2.7 over 3. When writing packages, I'll add support for 3 as an afterthought, but that's it.

The main reason for this and the main reason I prefer Python to other languages is the excellent and complete ecosystem of libraries. I've never seen a single Python library I wanted to use in the last 8 years that didn't have support for Python 2.7. I still run into those that don't support Python 3 well. So, using Python 3 means being terrified that I run into a case later on where I need to use some package that only supports 2.7 and I get stuck. With all of the other potential problems to run into while coding, why would I want to add that extra one, especially for something work-related?

I imagine this will continue being an issue until some major packages stop supporting Python 2.7 entirely and force people to make a choice.

I was in denial stage until we picked Python 3 for a project at work. Then I went into anger and bargaining with the other dev on the project. Heh. blush

Being forced to come to terms with Python 3 has made me see the light, though. I still have a host of personal projects in Python 2, but anything new I write is in 3.

For me, it was concurrent.futures that pushed me into acceptance stage.

My experience was similar. An even bigger push in my case was the async stuff in 3.5.

It's going to be a little while before pypy supports 3.5, but once that happens, Python will be very well placed. Something like Tornado + pypy + py3.5 looks a lot like the app server stack of the future IMHO.

It has been a massive PITA, and probably could have been better handled, but hindsight is easy. The future looks bright.

The Python team is blind to the huge number of users that will never switch to Python 3. My industry (a large one) is currently standardized on RHEL 6 (and the CentOS equivalent), which uses Python 2.6! I'm not generally able to install arbitrary software or demand upgrades without a lot of beurocracy. I was looking forward to the day when we move to RHEL 7 with Python 2.7.

I've started to view the lack of changes in 2.7 as a feature. I would value the stability and lack of breaking changes. Knowing that Python 2.7 is on a limited timeline, and that the Python 3 folks (users and developers) are trying to kill off 2.7, I can't think of any reason not to switch to another language.

In short, my field is nowhere near "acceptance" of Python 3, and the "anger" stage will just make us (me at least) leave Python behind. I will never want Python 3.x, and I don't even want Python 2.8. I wish people would recognize that stability is a virtue for some of us...

> the "anger" stage will just make us (me at least) leave Python behind

By switching to Perl 4?

If you're hobbled by dumb policies that prevent an upgrade to 2.7 how are you going to transition to something "better".

The investment in porting your code from Python2 to Python3 should be many times smaller than porting to an entirely new language ecosystem.

Yea, but what's to stop Python 4 from doing the same thing as Python 3?

It's clear from the tone of this article that the python core devs haven't learned the value of backwards compatibility even after their huge python 3 failure. The author makes it seem like the problem preventing python 3 adoption was python users' unwillingness to accept the inevitable, rather than the poor migration path provided (especially initially).

In a massive, old production codebase, upgrading your platform is incredibly risky, even if no code changes are necessary. But when you have to make code changes in a dynamic language...

The thing is, not all systems have tests. Not all systems even have known, defined behavior. Upgrading these things can be a multi-month, very risky project.

Given that python 3's backwards incompatible changes were made for purely cosmetic reasons and there are few compelling reasons to upgrade, I totally get the anger people feel. I work in startups with relatively new codebases and good test coverage, but I can empathize with other types of organizations. I'm shocked that python devs don't, even after 7 years.

That is an incorrect characterization of the core development team.


> Yea, but what's to stop Python 4 from doing the same thing as Python 3?

If it's any consolation, Guido learned his lesson[1] and will avoid backward incompatibility for Python 4.

[1] https://mail.python.org/pipermail/python-dev/2015-December/1...

The reason it might be hard to port python 2 code to python 3 is because most likely it is buggy. Python 3 is much stricter about distinguishing bytes from strings.

Making python 3 code compatible with python 2 (2.7 and 2.6) is much easier, granted you don't use new features exclusive to 3.x.

That's what I did so far with multiple applications and have yet to find case where I couldn't make my code work on 2.x and 3.x with adding import _future_ and in some cases six module.

Oh, quite the contrary for any non-trivial application. If you were to rewrite your code in a totally different language, you would get most of that right on semantic level. But instead, you're going to need to adjust your code to a very similar language, which sometimes has totally different semantics. This is going to bite your ass every now and then, because it's very, very easy to miss some spots where you used old assumptions that don't hold in new environment.

If you want fully supported Python 2.7 or 3.x on RHEL 6 then have a look at Redhat Software collections.


I think the Python core team is very aware that there are some users that will never switch to Python 3. It's just not relevant to them. If you're never going to switch, and all future development is going to be on the Python 3 branch, they should just accept that you're not a Python 3 user, and can continue using the stable Python 2.6 deployments you already are.

In the very long term, it's likely that your industry will be replaced. It's unlikely that this will be because of Python, though. Rather, technological change will continue to outpace bureaucracy and eventually the gap will become so big that some startup comes up with a new way of doing things and uses whatever the technologies of the future are to build their system.

>> huge number of users that will never switch to Python 3

I don't believe this assertion.

All companies, projects, and developers have to move to the current version of Python eventually. Python 2 eventually is no longer supported. Python libraries will start to drop support for Python 2. Developers will see that for their career they need to have Python 3 skills. Jobs ads will require Python 3. Linux distros will drop Python 2. New libraries will start to appear that are only Python 3.

Don't know when, but Python 2 will suddenly look very old.

Given that there are significant differences between python2 and 3, such that porting large codebases is a nontrivial task, why do you assume people with automatically decide to proceed with python3 instead of some other language? At this point, python3 doesn't just have to be better than python2, it also has to be better than Clojure/Swift/Julia/C++14/name a modern language.

>> Given that there are significant differences between python2 and 3

Really? I don't think they are that different. If it was so different and hard to port then why have so many libraries been successfully update to support both http://py3readiness.org/

You can't generalise but for the most part updating from Python 2 to Python 3 is likely to be easier and less risky and costly than doing a full rewrite in some other language.

Some companies might rebuild in another language, that happens all the time - things get rebuilt from this to that.

But why would a company ditch Python entirely in such a situation?

> All companies, projects, and developers have to move to the current version of Python eventually.

Well, yes, and that "eventually" is somewhere in ten or fifteen years.

There's no way to know the timeframe but tipping points can come very suddenly.

I've seen a number of companies get stuck with a large code base built on old technology and it causes them huge problems, chiefly around finding people who can (and are willing to) program with an obsolete language. Companies with large Python 2 code bases will make things easier for themselves if they start making the transition sooner rather than later.

From LearnPythonTheHardWay.org:

> A programmer may try to get you to install Python 3 and learn that. Say, "When all of the Python code on your computer is Python 3, then I'll try to learn it." That should keep them busy for about 10 years. I repeat, do not use Python 3. Python 3 is not used very much, and if you learn Python 2 you can easily learn Python 3 when you need it. If you learn Python 3 then you'll still have to learn Python 2 to get anything done. Just learn Python 2 and ignore people saying Python 3 is the future.

Besides, judging from the number of DARPA-sponsored projects that require Python 2.7, the future is not going anywhere soon.

I'm so tired of seeing people act like LPTHW is some gospel. I tried using it to learn Python and absolutely hated it. It scared me away from Python. Especially the "don't ever use 3, use 2" thing.

And if you're going to assume that government entities use of a language is what is going to drive our market than you better stop using Python and hop back on to the ColdFusion and Fortran train.

It's not an act. LPTHW is my Bible. I retake the courses every Sunday. Saying that I should hop on the Fortran train for stating that even advanced fearless Python users opt for 2.7 is heretics!

All of my code on my machine is Python 3 and soon the python my OS is using will be Python 3.

Do we need to keep flogging this dead horse. Python 3 is here, it's big it's bold it's bloody working unless your doing the most arcanely version specific Python code.

If your extremely lazy, just use 'futurize -0 -w' and boom Python 2 and 3 compatible code 95% if the time.

My advice to new programmers is completely opposite this. I recommend that people learn Python 3 and use it until they have a specific reason not to. Most new programmers can work with Python 3 and never run into difficulty.

If someone needs to use 2.7 for a specific project, it's easy to adapt if you're well grounded in Python 3. There's no reason these days to tell beginners to start with Python 2.

I may be wrong, but IIRC, all Python code shipped by default in the next Ubuntu will be Python 3.

Now may be a great time to start with 3 and never look back.

Yes. 2.7 won't be in the default install of 16.04. Although 2.7 will still be installable of course.

So why is this good advice - a hard recommendation against something ideally comes with a reasonable explanation for why.

It is good advice, because it works. You can give reasonable explanations for Python 3, involving politics, the future, and dead languages, while still having your students be confused when Google releases TensorFlow with only 2.7 support. Hard language is his style. He loathes anything that gets in the way of being a productive programmer. See: http://programming-motherfucker.com/

So in general I should never move to a new software technology until all software with the old no longer exists?

You should do whatever makes you happy and productive. So install 2.7 and ignore the people who say 3 is the future (unless you can work better in 3).

Hang on, the advice given is not "do whatever makes you happy and productive.", the advice given is "never use Python 3 until every line of Python code on your computer is written with Python 3".

And please explain why to ignore people who say Python 3 is the future, I don't understand why.

I thought you were asking me a (loaded) question. The advice given was not "never use Python 3 until...", that was a suggested reply to people telling newcomers to go for Python 3. Perhaps too tongue-in-cheek, but coming from someone who taught millions of people Python, worth noting.

You ignore people who say that Python 3 is the future, because it will make you happy and more productive when you can run modern libraries that have a 2.7 requirement. Or you don't ignore them, and get into endless debates about how almost all libraries support 3, and that you need to spend just 1 in 20 days of your time worrying about getting your code to work, instead of... programming. But if that makes you happy, you should do that.

Personally, I can't wait to switch to python 3. I'm glad most of the community is finally getting behind it.

The cogs move slowly. We've only just upgraded our huge codebase to 2.7, but I expect we'll be on some release of 3.x in about two years. Almost all of our big list of dependencies now support 3.3+, so there is little reason to not upgrade.

One frustrating part of attempting to transition is that not all libraries make it clear which Python versions are supported. Usually it's Python 2-only code that isn't clearly labeled as such, but it happens with Python 3 code as well. Sometimes a library doesn't work under specific versions of Python 3. The result is that library research takes longer than it would.

Please include this in the README for any project, even if it seems obvious.

> I have yet to hear from someone who has used Python 3 that they think it is a worse language than Python 2

the maker of the most succesful Python projects: Armin Ronacher

Just ask him about unicode for LOLs.

Those of us running Arch Linux have to check the rearview mirror to see the transition... back in 2010 when "python" started defaulting to Python3.

It wasn't the smoothest thing, but the world didn't end, either. Lots can be learned from it if you're thinking of making the move.

Use 3. Unless you have a good reason to use 2, then use 2.

The world doesn't have to be perfect. Go make something.

Can someone explain aiohttp server vs wsgi? It seems wsgi is the old standard but now you can run a server right in python, kind of like node. Are people running aiohttp instead of wsgi?

It seems like aiohttp can do everything that Flask can do. Are more people using this?

I want to know too. I did a little toy project with websockets in aiohttp and it was extremely enjoyable.

It'd be reassuring to know that other people see it as "the future" too, so I could invest into it more heavily.

I've used aiohttp for a few things. Works fine, small, lightweight, easy to use.

I think one fundamental miscalculation went like this: early adopters will start writing new code in python3, and it will gradually become more normal for new code to be python3, and eventually it will snowball and python2 will become legacy.

The problem is that early adopters have a choice: python3, or a completely different language. And there are a lot of good languages to choose from, and early adopters are a more radical by nature.

That's why breaking backwards compatibility is almost always worse than it looks like on paper: the early adopters you are counting on to get you through the transition are also the most likely to leave your platform completely.

What I think people figured out is you can port Python2 code to PyPy4. Then for all the testing involved, you remove any potential CPU performance worries and the GIL in one fell swoop.

Those two perks are worth more than any piddly features of Python3, which were already resolved in the ecosystem. I ported to PyPy instead of Python3 and gained those advantages, but can always port to Python3 when/if it ever gains traction. But I'm just as likely to migrate new projects to something else than Python3.

I view this blog as a long-form denial letter to himself.

I like the fact that the 'python core' devs have moved on to doing something other than pointlessly fiddling with something that's reasonably good, ie python 2.7

As for 'lack of support', what are folks expecting support for? If there's a known security issue, somebody will write a patch.

I am on CentOS and honestly for most of our applications there is just no compelling reason to change. We can do everything with 2 that we need.

I have seen some async stuff that looks interesting in 3 but not enough to make me change.

I've always wondered why they couldn't package 2 and 3 in the same binary and let the user have a switch for which to use?

They don't need to. You can install all versions (at least tested 2.4 - 3.5) side by side without any conflicts, which makes the whole dispute which language version to use so silly.

>With five years left until people will need to pay for Python 2 support, ...

Is this some sort of thing?

It's just the author pointing out that the EOL for official support for Python 2 is 2020. After then, the Python dev team won't be providing any (free) support, so people who want to keep using it will have to pay for it in some way -- whether by investing time in maintaining a fork, or by paying someone else to do so.

The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all.

>The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all.

Or it's evidence that python 2.7 works just fine for many people.

Cent OS 7 will support Python 2 for free until 2024. I would assume most patches will work without changes on other OSes as well. But if you want someone to do the QA work (and compiling) for you and no independent Python 2 fork will arise, you'd presumably have to pay someone for that.

Beyond that, apparently you can pay Red Hat to big bucks to provide you with bug fixes for really old Red Hat releases in a similar way you can pay Microsoft big bucks to get patches for Windows XP even today. I'm sure other companies will do the same if they can find enough customers that find porting costs more than such a support contract.

Wait, Python 3 has existed since 2008 and the transition is at 20%? How is that any form of success?

He said that 20% have reached "Acceptance", the last of the 5 stages, and said that most people are in stage 4 (Depression). That'd put overall progress at more like 80%.

This article also wasn't actually talking about adoption of python 3, just applying the stages of grief to how people feel about it. IOW, adoption of python 3 might still be worse than 20%, but 20% have accepted that they will need to adopt it soon.

Anecdotally as a full time python guy, I'd agree with the OP. The number of people interested in Python 3 has been growing a bit every year, and it seems opposition to adopting it at all has finally mostly dwindled :)

According to the python community survey of 2014, 80% of users are still using Python 2. Of course, that was two years ago, but 2014 was still six years after Python 3 was released. (https://wiki.python.org/moin/2.x-vs-3.x-survey)

If the python developers were any normal business, they would have abandoned Python 3 and returned back to developing what their customers obviously prefer; the problem here is that the python developers essentially concluded that because users get python for free, they aren't customers at all and the python developers can do whatever they want and their users will eventually be forced to fall into line.

Except they won't...

Python users are refusing to migrate to Python 3 as a community. It's a shame that the python developers have decided to try to show their customers who is boss because it won't work--the developers will lose. Many people will continue with 2.7 forever and fork the language in 2020 like Perl 5 day. The rest, like me, have dumped python and moved on to greener pastures.

Yep, it's true. My company started a major python2.7 project in 2015.

Pretty smart bet. You have production-ready PyPy to migrate to, and can always port to 3.x if it ever takes over (looks like never).

What's "Perl 5 day"?

Sorry... s/day/did

Overall progress is not 80%, by any meaningful metric.

I mean sure, if we take the 'people who like python 3 and eat burgers but only use python 2 while drinking lemonade sampled from a group of 20 people down at the pub', I guess maybe we're at 80%, but now you're just inventing random statistics for no reason.

The only relevant statistic is people actually using python3, and that number is not 80% (and not even close).

If you read the wiki link in the article, the 'stages of grief' aren't stages that you progress through. They're different ways of dealing with loss, and you won't necessarily hit all of them or do them in any particular order.

More importantly, these stages have nothing to do with choosing either Python 2 or Python 3. Python is just a tool. If one doesn't work, you choose another, and if the developers of that tool decide to screw it up, you just move on.

I still find reasons to be pessimistic. OpenCV was really really late to support Py 3.

Every pypy release focuses on Py 2 only (and Py 3 releases are an afterthought). Really weird prioritization for me (I'm not interested in legacy-focused projects).

Backwards incompatible language changes having not killed the community and gaining traction after 7 years is pretty good going.

It isn't a success when compared with other languages, no. I myself am a bit guilty. I have nothing against Python 3, but it is almost as if I want to "stay out of it" so I tend to default to using 2.7.

I think my use of Python overall (like others in this thread have mentioned) has decreased as a result of the 2-vs-3 move. There are other reasons, but the 2-vs-3 added to the list of reasons to make me reexamine building new projects in Python.

20% is a really, really optimistic guess.

Extended the maintenance timeline from 5 to 10 years is not helping the transition at all.

The extended maintenance timeline is an implicit admission that nobody wants to hop on their wagon.

There was a Simpsons episode[1] back in 1991 in which Homer, after eating a poisonous fugu fish, must deal with the five stages. It's an all time classic:

   Dr. Julius Hibbert: Now, a little death anxiety is
   normal. You can expect to go through five stages.
   The first is denial.

   Homer Simpson: No way, because I'm not dying!

   Dr. Julius Hibbert: Second is anger.

   Homer Simpson: [furiously] Why you little... !

   Dr. Julius Hibbert: After that comes fear.

   Homer Simpson: [worried] What's after fear?
   What's after fear?

   Dr. Julius Hibbert: Bargaining.

   Homer Simpson: Doc, you gotta get me outta this.
   I'll make it worth your while.

   Dr. Julius Hibbert: Finally acceptance.

   Homer Simpson: Well, we all gotta go sometime.

   Dr. Julius Hibbert: Mr. Simpson, your progress
   astounds me.
If only this Python transition were so simple.

[1] https://en.wikipedia.org/wiki/One_Fish,_Two_Fish,_Blowfish,_...

Applications are open for YC Summer 2021

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact