
Why Is the Migration to Python 3 Taking So Long? - josep2
https://stackoverflow.blog/2019/11/14/why-is-the-migration-to-python-3-taking-so-long/
======
recursivecaveat
The simple reason is that there was no compelling feature to reward you for
upgrading. You'd spend a tremendous amount of effort for dubious return and
(until recently) a smaller ecosystem.

1\. Unicode support was actually an anti-feature for most existing code. If
you're writing a simple script you prefer 'garbage-in, garbage-out' unicode
rather than scattering casts everywhere to watch it randomly explode when an
invalid byte sneaks in. If you _did_ have a big user-facing application that
cared about unicode, then the conversion was incredibly painful for you
because you were a real user of the old style.

2\. Minor nice-to-haves like print-function, float division, and lazy ranges
just hide landmines in the conversion while providing minimal benefit.

In the latest py3 versions we've finally gotten some sugar to tempt people
over: asyncio, f-strings, dataclasses, and type annotations. Still not exactly
compelling, but at least something to encourage the average Joe to put in all
the effort.

~~~
takeda
> Unicode support was actually an anti-feature for most existing code. If
> you're writing a simple script you prefer 'garbage-in, garbage-out' unicode
> rather than scattering casts everywhere to watch it randomly explode when an
> invalid byte sneaks in. If you did have a big user-facing application that
> cared about unicode, then the conversion was incredibly painful for you
> because you were a real user of the old style.

Actually that's the behavior of python 2, it works fine, until you send
invalid characters then it blows up.

In python 3 it always blows up when you mix bytes with text so you can catch
the issue early on.

> In the latest py3 versions we've finally gotten some sugar to tempt people
> over: asyncio, f-strings, dataclasses, and type annotations. Still not
> exactly compelling, but at least something to encourage the average Joe to
> put in all the effort.

That's because until 2015 all python 2.7 features were from python 3. Python
2.7 was basically python 3 without the incompatible changes. After they
stopped backporting features in 2015. Suddenly python 3 started looking more
attractive.

~~~
pmontra
> In python 3 it always blows up when you mix bytes with text so you can catch
> the issue early on.

This is definitely the case. I've been wrestling with bytes and strings all
the time during the port of a Django application to Python 3 for a costumer. I
can see myself encoding and decoding response bodies and JSON for the time
being. For reasons I didn't investigate I don't have to do that with projects
in Ruby and Elixir. It seems everything is a string there and yet they work.

~~~
CaptainMarvel
I’ve worked in a variety of Django codebases, and the last time I had trouble
with string encoding/decoding was with Python 2. Since moving to Python 3, I
have rarely needed to manually encode or decode, and I genuinely can't
remember the last time I did.

Perhaps there’s something about a port that requires encoding/decoding
bytes/strings?

~~~
takeda
The encoding/decoding is heavy in codebases that have to run on Python 2 and
Python 3 at the same time, and authors are worried about handling unicode
correctly on python 2.

Ironically when your python 2 app doesn't care about unicode, the porting to
python 3 is actually much easier.

------
downerending
Because there's been not enough carrot and too much stick.

The only real killer feature of Python3 is the async programming model.
Unfortunately, the standard library version is numbingly complex. (Curio is
far easier to follow, but doesn't appear to have a future.)

On the down side, switching to Unicode strings is a major hurdle. It mostly
"just works", but when it doesn't, it can be difficult to see what's going on.
Probably most programmers don't really understand all of the ins and outs. And
on top of that, you get weird bugs like this one, which apparently is simply
never going to be fixed.

[https://github.com/pallets/click/issues/1212](https://github.com/pallets/click/issues/1212)

~~~
semiotagonal
I wonder how many people feel that migrating to Python3 would have been worth
doing in the absence of being forced to do so.

Dropbox invested three years of work, actually hired Python's creator, and are
still not done. What are they getting out of it that they wouldn't have gotten
if Python2 simply had been maintained?

~~~
ogre_codes
This is backwards thinking.

Yes, it's expensive to upgrade from Python 2 to Python 3, but it's also
expensive for the Python project to maintain 2 versions of Python
indefinitely. If someone wanted other than the core Python team wants to step
up and maintain Python 2, they are free to do so, it's open source. But
failing that, expecting the Python team to support the older/ less functional
version of the code indefinitely is unrealistic. Corporate owned languages
have even shorter lifecycles for exactly this reason.

~~~
coldtea
> _This is backwards thinking._

And the alternative is cargo cult "newer is better".

> _Yes, it 's expensive to upgrade from Python 2 to Python 3, but it's also
> expensive for the Python project to maintain 2 versions of Python
> indefinitely._

On the other hand, they could progressively enhance upon a backwards
compatible single 2 version. JS manages to do that just fine, as does Java...

~~~
jnxx
> On the other hand, they could progressively enhance upon a backwards
> compatible single 2 version. JS manages to do that just fine, as does
> Java...

Common Lisp has a backwards compatibility which goes into decades, and
implementations like SBCL had no difficulties at all to absorb Unicode.

Racket even supports different language standards and completely different
languages (such as Scheme and Algol) running on the same runtime. And both
SBCL and Racket are compiled languages with a high-end GC which should make
such things more difficult than CPython, which is purely interpreted and has a
simpler GC.

But the incompatibility between Python 2 and Python 3 is perhap s only a
symptom of a larger problem. The Python developers have decided that backwards
compatibility is not that important any more. This is not a problem for
companies like Dropbox, or small start-ups, from which 95 % will not even
exist in five years on. It is, however, a huge problem for domains like
scientific computing, where most code has no maintainers and even for very
important code there is no budget or staff for maintenance:

[https://blog.khinsen.net/posts/2017/11/16/a-plea-for-
stabili...](https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-
the-scipy-ecosystem/)

~~~
acqq
> But the incompatibility between Python 2 and Python 3 is perhap s only a
> symptom of a larger problem. The Python developers have decided that
> backwards compatibility is not that important any more.

Exactly: and that was a wrong decision for anybody but the developers of
Python.

Everybody else prefers having something that works: "The improvements are
welcome, but please allow us to to run our old programs too, thank you, and
allow us to use that new feature only once we need it."

It's an obvious expectation. We would also _hate_ a new version of a word
processor which wouldn't open our old documents. Or a new version of Photoshop
which wouldn't open our old pictures. Or a new version of the browser where
only the newest web pages are visible.

It follows that it was absolutely technically possible to have a new version
of Python in which the old programs still work. It's the failure of the
developers that they haven't made it.

Compare that decision of theirs with the policy of Linus Torvalds who insists
that the old user programs should never break on newer kernels.

~~~
jnxx
And the reason why I think there is a deeper problem is that the issue is
clearly not solved with the python2/python3 transition:

Why do some python developers have to maintain installations of a whole
handful of python versions just to ensure that their code is working? Why all
the mess with pyenv, virtualenv, and so on? If the python developers, as well
as the library developers would support backwards compatibility, this would
not be necessary at all.

~~~
acqq
> the issue is clearly not solved with the python2/python3 transition

Exactly. The incompatibility mess continues when using different 3.x versions.

------
twblalock
I recently went through a fairly large upgrade from JDK8 to JDK 11 and it was
a bit of a pain -- lots of dependencies to update, etc. But very few code
changes were required, and the static type system made it pretty clear when
the codebase was broken -- it just wouldn't build. It still took my team
several weeks.

Migrating from Python 2 to Python 3 is way worse than that -- code changes are
required, and because Python is a dynamic language you may not notice bugs
until you actually run the code (or even worse, until after you release it to
production and some code branch that is rarely invoked somehow gets
called...). In other words, the tooling and the type system are not
confidence-inspiring and it's really hard to verify that you migrated without
breaking stuff.

~~~
LIV2
Will this will make people less likely to use Python in future for some
projects? I'm no developer or manager but I figure there'd be a thought in the
back of my head thinking "what if we need to rewrite this all again for the
next major release"

~~~
twblalock
What I've been seeing is people using Python 3 for new projects but leaving
their older projects on Python 2. As a result of that, Python 2 will continue
to be supported internally at a lot of companies, even though it is
"officially" end of life. Nobody wants to rewrite the stuff they finished
years ago that is now in maintenance mode.

~~~
JohnFen
I considered doing this myself, but decided that I didn't want the hassle of
having two versions of Python hanging around.

If/when the day comes that using Python 2 isn't realistic, I may go with 3, or
I may choose a different language, depending on the project. I'll cross that
bridge when I come to it.

------
Animats
First off. Python 2.6 and 2.7 supported Unicode just fine. I had a large all-
Unicode system in Python 2.6. You had to write u'word" to get a Unicode
constant, and use a "unicode(s)" function here and there. Also, the part that
remained "compatible" was that "str" remained an array of bytes, even though
there was also a type "bytes" and a "bytearray".

Early Python 3 was hell for conversion. The syntax was changed for no good
reason. u'word" became illegal. (That later went back in.) The "2 to 3
converter" was a joke. I didn't have the "print statement problem" because my
code called a logging function for all debug output.

Many of the P3 libraries didn't work. (The all-Python MySQL connector failed
the first time I tried to do a bulk load bigger than a megabyte, indicating
that nobody was using it.) It took _years_ before the libraries were cleaned
up.

Python 3 got some really weird features, such as type declarations that don't
do anything. I can see having type declarations, especially for parameters,
but they need to be used both for checking and optimization. CPython boxes
everything, which is terrible for numerics and is why most serious math has to
be done in C libraries. My comment on that was "Stop him before he kills
again."

~~~
viraptor
> Python 2.6 and 2.7 supported Unicode just fine

It did, but in a way that chainsaws support sculpting just fine. Technically
possible. Very advanced people will know how to handle it. Everybody else is
just going to injure themselves randomly.

Most people writing py2 got the text/binary processing working on accident.
Things appear to work until you throw actual Unicode into parameters and then
nobody knows what happens. There's a number of "what does this decoding
exception mean" questions on stackoverflow every day. They're often actual
bugs people could ignore before. Now they're told immediately and I believe
that's better.

~~~
user5994461
Pictures of chainsaw sculptures for reference:
[https://www.google.com/search?q=chainsaw+sculptures&tbm=isch](https://www.google.com/search?q=chainsaw+sculptures&tbm=isch)

Some are quite good and finely detailed in my opinion. It's really nothing
like what you'd expect after hearing _chainsaw_. There actually are small
chainsaw, maybe even one-handed ones, to do that.

~~~
jeltz
I think ice sculpting on the other hand is usually done with a normal sized
chainsaw.

------
m45t3r
Any non-trivial piece of code will probably be a pain in the ass to port for a
more recent version of the language, and the level of PITA depends on the size
of the project X size of the changes in the language.

Case in point, I worked in a project using Ruby. When we migrated from Ruby
2.4.0 to 2.4.6 (yeah, a minor upgrade), it broke spectacularly. Trying
multiple Ruby versions, the change was actually introduced in Ruby 2.4.1.
After some investigation, a change in Net::HTTP library from stdlib had a
change that broke a dependency from a dependency. The fix was just a line of
code (we just need to change the adapter used for HTTP communication), however
it was two days of work for a minor upgrade.

My current job tried to migrate from Java 8 to Java 11. It also broke multiple
services. This one is still in progress, months later.

Python 2 to Python 3 is bigger than both of those version changes (however it
is equivalent to Ruby 1.8 to 1.9 changes), so yeah, it does take more time.
And like some projects that are forever running Ruby 1.8 or Java 8 (or even
worse, Java 6), we will have projects forever running Python 2 too.

------
pmoriarty
Another data point:

According to my highly unscientific survey of the packages in Gentoo's package
repo, there are roughly:

\- 2500 packages that work with Python 2 or 3

\- 1350 packages that work with Python 2 only

\- 350 that work with Python 3 only

My methodology:

[http://dpaste.com/1M0TCV7](http://dpaste.com/1M0TCV7)

~~~
stevesimmons
I bet many of the Py2-only ones are old legacy packages that have been
superceded by newer better options.

------
guardiangod
>Why Is the Migration to Python 3 Taking So Long?

For the same reason why migration to IPv6 is taking so long.

Both technologies don't solve immediate problems end users are facing. Instead
they solve 'nice to fix' problems that few people care about.

------
at_a_remove
In my current job, I can point out something that might be a contributing
factor:

I work in an industry where there is basically one 800lb gorilla of a vendor.
They update rarely, because their product is a mission-critical, life-or-death
sort of thing. Their current product is heavily, heavily integrated with x.y.z
version of software from a different vendor in a different segment, but also
weighing in at 800lb. Yes, they specify x.y.z, not just x or even x.y. That
software comes bundled with a Python 2.7.5 distribution.

Imagine my woes trying to get pip running, which unhelpfully suggests I
upgrade Python. Cannot seem to find any other path to even get pip going
because of what I call the "lol just upgrade n00b" factor. Perhaps that
information once existed but I cannot find it.

So, I am stuck on this version because of some pretty tight integration, at a
couple of removes. I think the vendor-linkage can cause some "drag" that folks
who work in a greenfield environment might not be thinking about. It can be
unfortunate but there it is.

~~~
user5994461
Surprised it's including pip, if it's really this old or tightly coupled, it's
common to not have pip at all, or the old pip version simply can't run.

If it can help you. The trick I use is to install a normal python 2.7
interpreter with pip. Then you can use it to install software to any
directory, including the one from the other application. There are flags to
specify what to install, from where to where, internet or not, something like

    
    
        pip install packagename --target=/to/app/lib

~~~
tingletech
I came across a system where pip had broken the other day. easy_install to the
rescue!

------
upofadown
Py3 is for all practical purposes a language fork of Py2. So it doesn't really
make sense to talk of a "migration". If Py2 becomes unworkable somehow then
people will rewrite stuff. Some of it might even be in Py3.

Considering all the stuff that is written in Py2 I really don't see it being
out and out abandoned. That wouldn't really make any sense. With computer
languages stuff never goes away.

~~~
carapace
Py2 is the new Fortran, I like to say.

~~~
sigjuice
And they both use actual Fortran :)

    
    
      $ python3
      Python 3.7.4 (default, Sep  7 2019, 18:27:02) 
      [Clang 10.0.1 (clang-1001.0.46.4)] on darwin
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import numpy
      >>> 
      [2]+  Stopped                 python3
      $ lsof -c Python | sed -n "/fortran/s/$USER/<redacted>/gp" 
      Python  35190 <redacted>  txt    REG    1,4  1550456 12887664541 /Users/<redacted>/Library/Python/3.7/lib/python/site-packages/numpy/.dylibs/libgfortran.3.dylib

------
linsomniac
What do you mean taking so long? The original target date, proposed by Guido,
was Jan 1, 3000. Looks like we're 980 years ahead of schedule! :-)

I say this in part because comedy, but also because it was anticipated to be a
long project. It was originally called "Python 3000".

------
Groxx
Because it's not just syntactic changes, it's implied-semantic changes too.
You can't mechanically transform a project and know that it'll work.

And you can't do it gradually, so it's all-or-nothing. (yes, "six" exists, but
you still execute one way or another)

And you'll have to change the versions of all your libraries, which is not
usually a smooth experience in the Python ecosystem. (this is another place
where it's "all or nothing", since six can't help you if your dependencies
don't all use it + use it correctly)

\---

It's a _huge_ risk with _huge_ cost for already-working, running code. For new
stuff, sure, write it in 3, but 2.7 works fine and has the added benefit of
being _very_ well understood by this point.

~~~
mixmastamyk
You can definitely do it gradually with a few bumps but there are categories
of large apps that are more difficult than average.

------
soyiuz
In the academic data science-y world, the transition has passed the hump. In
other words, where I was reluctant to switch a few years ago because some of
my basic tooling was still in the 2 land, today the critical mass of important
libraries has been ported and updated. Most of the cutting-edge docs and
tutorials (Google's GPT2 for example) default to python3.

------
musicale
Because they broke backward compatibility in very annoying ways without
providing a fallback mechanism.

I still haven't forgiven them for killing the print statement, which could
have peacefully coexisted with a print() function.

~~~
mixmastamyk
Bugged me for a while five years ago. Then I moved everything to logging and
use an editor snippet: pr<TAB> —> print(‘foo:’, %cursor%)

------
lasermike026
Management wants new features not porting. They will only port when they
absolutely have to.

~~~
war1025
Even if management wanted porting, the python 2 -> 3 migration path is very
painful in the details, while on the surface not having a lot to offer at the
other end in terms of new capability.

~~~
lasermike026
Roger that.

------
paulie_a
Simple answer. Updating legacy code. If it works now why upgrade? Stability is
more important than some new features of the language. Granted that has its
downfalls via security issues but stability wins

If you have a hole it's hard to dig yourself out of it. This is why I prefer
modular apps instead of monolithic codebases. You can upgrade piece by piece.
Otherwise it's all or nothing and dangerous

------
JohnFen
I haven't migrated to Python 3 simply because I see no reason to go the the
pain and hassle of porting my existing Python code. Python 3 doesn't give me
anything that I want or need badly enough to put that much work into it.

------
digitalsushi
I never knew if this was a cynic's answer or truthful, but I was told by my
manager at one point that RedHat's OS is superglued to python2 and spends a
lot of money to keep python2 in good working order. I highly expect it's a
cynic's response and please read it as such until someone in-the-know can
retort my post.

~~~
navigatr
Red Hat has long support contracts for their server OSes that shipped with
Python 2 when it was still kosher to do so.

That means they'll patch Python 2 should vulnerabilities be found on their OS.

~~~
ianai
It’s a pain from the outside looking into their business. Their contracts,
though, are their business.

~~~
int_19h
Those support contracts aren't cheap, though. I don't think they're getting
the short end of the stick here. If anything, the companies that run lots of
Python 2 code in the house that haven't migrated yet - which is not uncommon
in the enterprise - have the incentive to run that code on RedHat, if they
aren't doing so already.

I wouldn't even be surprised if they provide support contracts specifically
for Python 2 on their newer OSes. ActiveState announced that they'll continue
providing paid support for Python 2 on all platforms, so some people at least
feel like there's a business model there...

[https://www.activestate.com/company/press/press-
releases/act...](https://www.activestate.com/company/press/press-
releases/activestate-offers-extended-support-for-python-2-beyond-eol/)

------
KaiserPro
I now work on a python 3.6 codebase. Something like 100k lines of code. in
practice 80% of the code was written for 3.5.

However, barring speed improvements, there isn't much to offer, apart from
unicode, f strings and annotations.

If python 3 had proper multithreading, that might have been worth breaking
backwards compatibility for.

------
NelsonMinar
The key inflection point was around Python 3.3 when enough bridging
technologies and tools came along to either migrate code or else write code
that supported both languages. Things like adding the u'string' syntax to
Python 2, the creation of six.py, all the various features in the future
package. That gave a much smoother transition path and enabled crucial
libraries to work in Python 3, which then let everyone else migrate too.

------
neilobremski
The upward migration is imminent for ALL CONNECTED applications and not just
Python 2 to 3. An issue I've seen with PIP (that must be relevant to other
platforms as well) is that version-locked packages are for software that can
no longer communicate with the actual SaaS/API because THAT layer has changed.
It's really THIS that is forcing conversion of Python 2.7 to 3 because API
vendors will stop supporting old software while they continue breaking their
own interfaces. The alternative is the end user or FOSS picking up the slack
but that's only going to happen for SOME of the API's. In the end it will be
cheaper (albeit still painful) for companies to upgrade their code to Python
3.

I have a lot of Python 2.7 code that I wrote years ago which has been running
smoothly and my team is generally going to rewrite rather than "convert"
because I really don't trust conversions. I'd rather see all bugs upfront
rather than hidden in the fog.

------
jeltsin1234
The thing is python3 IS a better language. Unicode is very important for me,
as i deal with a multitude of languages. For a good comparison, see how PHP
failed unicode with PHP6 and we still deal with an insanity that is the mb_
functions. In python this is a non issue, and its a very nice language to work
with.

------
klyrs
In my experience, the biggest issue that I face is "what do you mean by Python
3?" I count 4 minor versions which aren't fundamentally broken, and I
encounter them all on a regular basis.

A lot of my code is performance critical, and, for example, I'm still salty
about dictionary operations taking O(log(n)). But the proliferation of active
minor versions makes it very difficult to write portable, performant code.

It's become a sticky wicket. I want to migrate to Python 3 (and, by and large,
I have in most of my projects). But what version do I target? Will my
dependencies make the same choice? Or does "migration" turn into a sisyphean
task? It's becoming burdensome enough that I'm contemplating abandoning the
language for something more stable.

~~~
user5994461
If it helps you, you can consider that no version below 3.5 is worth thinking
of. It's the edge when python added back enough features to ease the migration
and many libraries started being ported.

Current version is 3.7. If you expect your migration work to take a year, you
should consider going for 3.7 and above only, because the previous minor
versions will be dropped by the time you're done.

~~~
klyrs
Thank you (and it's good to say so if another reader doesn't know), but yeah,
the 4 versions I was referring to were 3.5-3.8... but my point is that it's
now a perpetually moving target.

And fwiw "3.7 is the current version" doesn't help my users.

~~~
user5994461
I think pip can select the version automatically as long as you provide the
right metadata with the package. Shouldn't matter much really.

Maybe write in the README that the package is only tested on 3.7 and above so
users can be aware of that and check. Bet they struggle to figure out what
version to upgrade to as well.

~~~
klyrs
> I think pip can select the version automatically as long as you provide the
> right metadata with the package. Shouldn't matter much really.

I don't think pip can install new versions of python. This becomes burdensome,
especially for novices.

> Maybe write in the README that the package is only tested on 3.7 and above
> so users can be aware of that and check. Bet they struggle to figure out
> what version to upgrade to as well.

I must disagree with this philosophy. As a package maintainer, I think it's my
responsibility to "struggle" so that my users don't need to. And fwiw, my
employer also thinks that this is part of my job.

------
drdeadringer
About 10 years ago I started learning Python. I double-timed across both
versions. I decided to land on v3 for good. I haven't looked back. Granted
that Big Corp cannot have such flexible actions. I reserve sympathy.

------
jdhawk
The same reason there is still a ton of legacy PHP 5.6 code.

Migration in interpreted languages that implement major breaking changes is
really tedious.

------
cygned
> most large organizations, outside of the hype cycle of technical news posts,
> move much more slowly than the press or blogs would have you think

That’s the reason I am so upset with today’s JavaScript ecosystem - things
move so fast that good technology is being deprecated and changed constantly
which breaks all kinds of things in other places.

------
izolate
As somebody who only occasionally uses Python, the fact that the default
`python` binary on my system resolves to Python 2.x and I need to specify
`python3` to invoke 3.x, means I am quite often mistakenly using Python 2
instead of 3.

How can we expect Python 3 to become the default if Python 2 still asserts
such dominance?

~~~
jonfw
That's not a python thing- that most likely has to do with your package
manager.

In my archlinux installation, python resolves to 3, and I have to use python2
if I want 2

~~~
izolate
Well, PEP 394 suggested it be this way, so Python is also a bit complicit.

~~~
bscphil
As the history section of that PEP notes, it was actually written in part as a
response to distributions like Arch changing the default Python to Python3.
Although it's true that the PEP originally said that python should point to
Python2 (it doesn't any more), it also said that code requiring a specific
version of Python shouldn't assume anything about the system default Python,
instead using either python2 or python3 - and requests that distributors of
Python provide these commands.

This suggestion was adopted by nearly everyone, including Arch Linux. If
you're writing a script that calls Python, you probably still want python3 as
your command for compatibility across all current distributions.

In other words, the intention of the PEP was not to ingrain Python 2 as the
"default" Python, it was to get people to stop making assumptions about what
Python version they would get and use the python2 and python3 commands
instead. I do agree that it has had that effect.

~~~
int_19h
The original recommendation had to be made, because at the time, there was a
lot of software from pre-Py3 days that would have #!/usr/bin/env python, and
that was broken badly if it was pointing at Python 3. They have changed the
PEP now that Python 2 is reaching EOL, since it wouldn't make sense to demand
"python" to be an alias for an unsupported version.

------
leetrout
Or, in the case of my employer (and new ecosystem for me), we are using Jython
which AFAIK has no clear plans or path to support Python 3 at this time. Iron
Python has started work to support 3 but it's still in development and seems
to be a ways off.

------
MaulingMonkey
We're still waiting for Maya to switch. It's still using Python 2, and
Autodesk keeps putting off the transition, partially because it'll break the
scripts of all their downstream users.

~~~
pfranz
I think that's partially correct. If you follow
[https://vfxplatform.com/](https://vfxplatform.com/) it says Python3 was
pushed from 2019 to 2020 to allow everyone to upgrade Qt and PySide (which are
prerequisites). The reason it was deprioritized in previous years were similar
transitions like C++14, gcc, etc.

I've been meaning to dig into Maya, Houdini, Nuke's Python 3 transition plans.
I know Houdini will offer a Python 3 option with Houdini 18 (shipping in the
next month or so).

I don't think the reason was because of downstream users. Python 3 was an
inevitable change. Previously, they swapped out PyQt for PySide which wasn't a
forced change, but required everyone to update their Python scripts.

------
just_myles
Personally, I don't see the need to migrate entire code bases if there is no
need. I think that much is obvious. Perhaps the focus should be put onto
pivoting instead. That way it doesn't leave your dev team on the hook to go
back and change existing code bases in python 2.

------
mbparks
In my organization, business needs overrule technical details. Management
didn't want to spend the resources until forced to do so, rather focus on
revenue-generating work. Can see it both ways I guess. We have recently
started upgrading our codebase to support 3.X

------
xvilka
Find a few remote code executions in Python 2 after Jan 1 2020, and migration
will be faster.

------
goatinaboat
The 2to3 tool should add .decode(‘utf-8’) to every string manipulation, even
better Python 3 should have a flag to make that the behaviour and _even
better_ that should default to on.

So much effort wasted doing this in a large codebase. And what do you get for
it? It’s just not worth it. Nobody actually needs Python 3, it was foisted on
them by the developers. What everyone really wanted was Python 2.8.

~~~
joshuamorton
Please no! Text is text. Bytes is bytes. Convert to the correct form on input
boundaries, and that's it. Don't switch back and forth internally, it's overly
complicated, error prone, and slower.

~~~
test7777
unix pipes (stdin, stdout) are bytes, files are bytes, filenames are bytes.
yet, for some reason python3 thinks al of those are text. its not the coders
that are wrong, it is the language.

~~~
kstrauser
No, it's Python 2 that thinks those are all text. Python 3 makes you
explicitly say "convert this stream of bytes into a text encoded string", and
from then on it's a str object.

Python 2 was happy to (try to) let you call text methods on a JPEG. Python 3
draws an appropriate distinction between the two.

~~~
test7777
Python 2's "str" is a bytestring. All of those are bytestrings in Python 2 as
they should be. Python 3 makes them text and suddenly i can not output data
over stdout anymore without extra steps of switching the mode of stdout from
some arbitrary text default.

------
qwerty456127
What are some Python 2 features that make it hard to transpile to Python 3
automatically?

------
sigjuice
Are there any examples of rewrites of large code bases from one language to
another?

~~~
yjftsjthsd-h
Mercurial, perhaps?

------
m463
For me, it was that python on macos was 2.7

------
munherty
This is like why are hedge funds still using excel to model. Also why is SAS
stilll used

------
alexhutcheson
[copying comment from an older HN thread, not speaking on behalf of any
employer, opinions my own]

I think many people underestimate the challenge that the 2 to 3 migration
presents for large enterprises. The core issue is that even though the
migration for any given module is normally really easy, the total effort
required to migrate is still essentially O(n) in module count/file count,
because even with current tooling you still need to have an engineer look at
every module to do the change safely. Even if it only takes ~5 minutes per
module to make the changes and validate that it works correctly, this becomes
a giant undertaking when you have tens of thousands of files to migrate.

The fact that it takes a long time also creates other problems. Your business
isn't going to hit "pause" on other development, so there will be changes
constantly introduced into modules you've already "swept". It's going to be
hard to make sure 100% of your engineers and code reviewers are knowledgeable
about the specific requirements to make sure the code works in both 2 and 3,
so you would really like some automated safeguards to make sure they don't
introduce anything that won't work in 3. Pylint helps with this, but won't
catch everything. Unit tests are obviously essential, but:

1\. Even a well-tested project won't have tests that cover 100% of code paths
and behavior.

2\. You're stuck running the tests on both python2 and python3 for the
duration of the migration, which doubles the resource (compute, memory, etc.)
cost of your Python CI and regression testing infrastructure for the duration
of the migration.

Most big companies have passionate Python advocates who really want to be on
Python 3, but the scale of the problem and the lack of tooling to tackle it
with a sub-O(n) amount of effort make the overall project risky and expensive
for the business.

~~~
kevin_thibedeau
The way to start a migration is to first get the 2.7 code as forward
compatible as possible. Bring in the future imports. Make sure equivalent
iterators are used where 3.x mandates them (xrange, iterkeys, etc.). Make sure
Unicode is managed correctly on I/O. Explicitly call out truncating division.

This doesn't require parallel testing. These all improve the quality of 2.x
code even if you never make the leap to 3.x.

Once this is done you can use 2to3 to mechanically fix the remaining
differences. Anything else that remains broken can be special-cased in the 2.7
code until 2to3 works without intervention.

~~~
coldtea
2to3 never worked and will never work "without intervention".

That's why six and manual changes are always needed...

~~~
kevin_thibedeau
Using six is intervention. You end up with code that has one foot stuck in the
past.

------
kissgyorgy
I honestly don't care as long as I don't have to deal with Python 2 code bases
anymore. The important point is that all of the popular open source libraries
and frameworks are ported.

------
UptownMusic
Simple. Python 3 is theoretically better but not in fact better, at least for
most applications of Python.

~~~
lkbm
In terms of feature-richness, I'd choose Python 3 over 2.7. Pre-3.6 it was
slower, but at this point it's more memory efficient and generally faster, so
it is in fact better in basically all applications.

The problem we have where I work is some very clever 2.7 code that isn't easy
to redo in Python 3. For any new project I do, I use Python 3.

------
m4r35n357
It _is_ happening, but moaning about it won't make it happen any more quickly.

~~~
eej71
I think the intent is less moaning about it - and more learning about the
blocking points so they can be better surmounted. At the very least, perhaps
the blocking points can be avoided in the future. Those who do not learn from
history... etc etc.

------
SQueeeeeL
Python, unlike a C executable, could hypothetically stop working tomorrow.
Researchers who have code that works/businesses don't give a shit about the
esoteric differences between py2 and py3, they just want their stuff to keep
working. This is similar to the banks still running Cobal backends, I don't
know why everyone cares, I guarantee multiple servers out there are running
executables that couldn't be rebuilt, but those don't curse you out on
terminal every time I turn one on

~~~
sansnomme
Because software that isn't protected by walls of legislation value features
and the end user UX. i.e. "evergreen". Your Nokia N900 running Meego may be
sufficient for certain subsets of HN readers but your average user wants
Material Design and cannot care less about ideological purity or "if it ain't
broke don't fix it".

~~~
FpUser
"average user wants Material Design" \- is that a fact or just a wishful
thinking from people trying to push said design?

EDIT: Personally for me while I find for example older windows interfaces ugly
they were very consistent and functional. In modern designs I sometimes could
hardly find what is clickable/actionable. It is not interface working for me
but the other way around

