
Ride the carousel of folly with a Python migration - SeanBoocock
https://mortoray.com/2017/01/24/ride-the-carousel-of-folly-with-a-python-migration/
======
deathanatos
> _The only somewhat problematic code was this:_
    
    
      for line in iter( handle.stdout.readline, '' ):
        self.output.put( ( handle, line ) )
    

> _This no longer terminated an emitted lots of odd values (one number
> repeatedly). It turns out it returns an empty b '' now when empty, as
> opposed to a ''. This could have been avoided had this API sensibly returned
> None instead of an empty string._

For any observers, this is much more idiomatically written:

    
    
      for line in handle.stdout:
        self.output.put((handle, line))
    

That is, the default iteration over a file-like object is line by line.
Frankly, I wonder how many Python programmers are aware that iter() has a two-
arg form; this is the first time I've ever seen it used (and I had to look it
up myself).

> _[PyPI]_

His solution of maintaining separate code bases is … unusual. I feel like a
cursory glance around the library landscape shows that most of the projects
out there are maintaining a single codebase that supports 2.7+. six greatly
simplifies most of this. I'm not aware of _any_ projects that are supporting
Python 2 and 3 concurrently by separating out their code completely.

> _I presumed that I’d just open a “Migration from Python 2 to 3 Guide” and
> follow the instructions. Alas, there is no such guide. There appears to be
> no central resource whatsoever on the porting of code from version 2 to
> version 3._

An I'm Feeling Lucky search on _that exact quote_ pulls up the migration guide
_in Python 's official documentation_.[1]

[1]:
[https://docs.python.org/3/howto/pyporting.html](https://docs.python.org/3/howto/pyporting.html)

~~~
agf
In addition to what you've pointed out, he claims that 2to3 didn't do anything
to his code, and yet he had to fix print statements. 2to3 definitely fixes
print statements; he likely didn't read the usage or docs at all. By default
it prints a diff, you have to use 2to3 -w if you want it to modify files on
disk -- I certainly don't want tools modifying files in-place by default, so
that seems like the right API to me.

~~~
mortoray
The simple syntax changes I did prior to running 2to3 as they came up on the
first run I tried. The semantic changes that didn't come up in 2to3 is what I
was more bothered about.

~~~
agf
Then you should update your post to say that, rather than saying 2to3 didn't
do anything to your code.

------
pdw
He claims there's no migration guide, but when I google on the phrase he used,
"Migration from Python 2 to 3 Guide", or any variation of that phrase, the
first hit is
[https://docs.python.org/3/howto/pyporting.html](https://docs.python.org/3/howto/pyporting.html)
I wonder why he didn't find it.

~~~
freyir
And when 2to3 didn't work, he didn't attempt `2to3 --help`?

I guess if everything went smoothly, he wouldn't have had a blog post to
write.

------
zzleeper
In my opinion/experience, there are some important hidden costs of _staying_
in py2:

1) Hard-to-read or quirky code. Unicode handling, args, harder unpacking, etc.
The more time it passes, the more updates py3 gets, and the more outdated the
py2 code looks.

For instance, after a year almost exclusively in py3, I have completely
forgotten about iter(), unicode gimmics, etc. and also rely more on stuff that
is just too slow on py2 (like ordered dicts). With 3.6, I can start assuming
dicts are ordered, which will create further breaks for py2 users (if they
want to backport anything).

2) Outdated libraries. As the other comments say, epydoc is abandonware. Had
he migrated to py3 earlier, he would have stopped investing in that tool
earlier and switched to the standard (sphynx).

3) Harder to interact with others in the community.

Although the folks at google might think otherwise, most people are now using
py3, and most code being developed right now is on py3 (at least open source
code).

If you are still on py2, it will be harder to hire (it hints at huge technical
debt, and a tedious working environment), harder to sell to others (e.g. I
don't use google compute because of that), and harder to collaborate on (for
open source projects).

All in all, the more that py2 and py3 diverge, the harder it will be for
legacy (py2) users to migrate (this was not the case a few years ago, when
there were still important libraries on py2 only).

~~~
rtpg
Nitpick, but you cannot assume dicts are ordered in Py3, even in 3.6.
CPython's 3.6 implementation will make it so, but it's totally a CPython
implementation detail. Please use ordered dicts when appropriate (they'll be
dicts in CPython anyways)

------
lowmagnet
I ran a 2to3 last week on a codebase I've been working for the past six
months. Took me one try to get that I was missing a flag. 'git diff' cured me
of thinking it wrote anything, and I found '-w' shortly after.

I guess a masochist could do something like pipe 1 to 'patch -p0' or
something…

I ran into the same unicode issues as the author, and dumped all the code that
handled it because I didn't need it in 3.

I also reduced the whole codebase by some 150 lines because of not needing to
do a bunch of extra work I originally designed in when working through the
code. This was more due to fresh eyes.

ETA: the code was on 2.7 because a library I needed wasn't yet available under
python 3 (FuelSDK from SalesForce) until November, and I completely missed
this. 1.0 version supports py 3, but it wasn't in pip.

------
pdonis
This statement in the article...

"there is nothing in the PyPI package description that actually indicates what
version of Python is being targeted"

is incorrect. There are Trove classifiers[1] for both versions, and indeed for
every minor version number of both versions. There are even Python 2 and 3
classifiers that end in "Only", which look like they would exactly fit the
author's use case.

[1]
[https://pypi.python.org/pypi?%3Aaction=list_classifiers](https://pypi.python.org/pypi?%3Aaction=list_classifiers)

~~~
mortoray
Okay, they are classified but I don't see that anything in the setup, or pip
package management, seem to actually use them. Are they somehow used in actual
package management, or just search qualifiers?

~~~
di
You are correct, the classifiers are just search qualifiers.

However, since pip 9.0.0, it will respect the Requires-Python[0] metadata,
which can be specified via your setup.py file.

Since PyPI allows only a single source distribution per release, though, this
will still not get you separate modules for separate codebases as it seems you
originally desired. As mentioned in a previous comment, it's generally
preferred to support multiple versions in a single codebase via six.

[0]: [https://www.python.org/dev/peps/pep-0345/#requires-
python](https://www.python.org/dev/peps/pep-0345/#requires-python)

------
bigmanwalter
Python 2 > 3 is the next Y2K. My colleague and I have started offering Python
2 > 3 consulting services as some of our clients weren't prepared to make the
migration themselves. With Django announcing that they will drop Python 2.7
support, this might become one of our more popular services!

~~~
tingletech
We have "until at least April 2020" before django 1.11 (the last that works
with 2.7) is EOL. Python 2.7 is EOL in 2020 as well.

------
viraptor
I'm glad someone actually wrote up their migration. Even with some weird
elements ("missing" guide?) it still contains good links and may be useful for
others. Hope to see more of those, as opposed to the usual FUD and ideological
arguing about py3.

------
viraptor
Re. documentation: unfortunately epydoc is pretty much dead. No updates since
2008 and doesn't run on py3 itself.

Sphinx is close to a standard nowadays.

------
Animats
That doesn't sound so bad. When I ported a medium sized system to Python 3 two
years ago, I found serious bugs in four library packages. It took me about a
month to get everything working. He's not reporting anything like that.

------
dagenleg
Author, please stop responding to every negative comment. That looks pretty
pathetic, especially when you just expose your lack of expertise this way.

