
Migrating to Python 3 with pleasure - arogozhnikov
http://github.com/arogozhnikov/python3_with_pleasure
======
re
I've been toying around with Python 3 and using it for most of my
personal/hack projects, but I somehow missed the unpacking improvements:
[https://www.python.org/dev/peps/pep-0448/](https://www.python.org/dev/peps/pep-0448/)

In particular, being able to create an updated copy of a dict with a single
expression is pretty cool:

    
    
        return {**old, 'foo': 'bar'}
        
        # Old way
        new = old.copy
        new['foo'] = ['bar']
        return new

~~~
est

        return {**old, 'foo': 'bar'}
        
        # Old way
        return dict(old, foo='bar')
    

Not much difference if you ask me.

~~~
orf
It's twice as slow, doesn't work with more than one dictionary, you can't
easily control the merge prescience and you can't (easily) use
expressions/variables in the keys:

    
    
        {**x, 'fo'+'o': 'bar', **y}

~~~
Doxin

        return dict(x, **{'fo'+'o': 'bar'}, **y)
    

The new syntax is some mild syntactic sugar. (Which isn't a bad thing IMO)

~~~
kamaal
This is the most unreadable code I've seen so far.

Just skip all this and use Perl instead. You can write far more idiomatic,
succinct readable code in Perl than you can in Python.

The whole point of Python is to not write code this way.

~~~
vgy7ujm
This is what I have been saying for a long time. Readability in Python is an
illusion.

------
cup-of-tea
It's so strange to me that data scientists would need to be convinced to move
to Python 3. It's superior in every way to legacy Python. I can understand
maintaing Python 3 compatbility for legacy systems if you don't want to have
Python 3 as a dependancy, but data scientists will be writing mostly ad hoc
code and using Jupyter notebooks. The people around me are not allowed to use
Python 2, in fact they're generally required to use the latest version of
Python 3.

For anyone having trouble with maintaing multiple Python versions, I recommend
Pyenv. You can install multiple local versions and switch between them. The
selected version then uses the standard commands "python" and "pip", etc.
which you can use to make your virtualenv from.

~~~
newen
For me, it's as simple as python is not a moving target. Whatever new changes
there are in python 3 doesn't affect me since I won't be using any of the new
features even if I were to use python 3.

~~~
cup-of-tea
Why on earth would you not use the new features?

~~~
newen
Tradition? ;) I only use python for deep learning, which is not really that
intensive in terms of code. Also most of the code out there for deep learning
works on both python 2 and 3.

Also, your phrasing invites the question. Why on earth would I use a new
feature in a turing complete language? Unless using the new feature results in
tangible improvements in my code, I don't see a reason to use it.

~~~
koshak
I really like this one. “So you already have your turing complete language!
What else do you need?”

~~~
newen
That's not the point. The point is there is no reason to use an extra language
feature unless it provides a tangible improvement in the code.

In fact, there are lots of reasons not to use an extra language feature that
doesn't provide tangible improvements in code, including maintainability, ease
of reading code, and portability in python's case.

------
AlexCoventry
I've moved to python 3 over the past couple of months, after resisting for the
better part of a decade. I like it.

One surprising thing I learned from this document is that dicts now iterate in
assignment order, not hash order. That's going to break some code for people.

~~~
VHRanger
Dicts are UNORDERED associative containers. If youre depending your app on
implementation defined behavior, that's on your developpers shoulders. Stuff
like that shouldn't pass code review

~~~
djrobstep
Not for much longer, as of 3.7 the ordering is a language feature:
[https://mail.python.org/pipermail/python-
dev/2017-December/1...](https://mail.python.org/pipermail/python-
dev/2017-December/151283.html)

It's mad that it ever wasn't this way. Mapping-with-ordered-keys is such a
useful and pervasive data structure (all database query result rows, for one)
that an ordered dictionary should be a fundamental part of a language.

It has been so much more pleasant to write python since ordering was
maintained by default.

~~~
larrykwg
I have the opposite reaction to it, it seems insanely idiotic to have a
associative array with ordered keys. It can only make sense to someone who
doesn't know anything about fundamental data structures and a language that
caters to people like that in spite of the performance penalty is just
strange.

but hey, its Guido, I still can't fathom that he moved reduce into functools.

~~~
goerz
I vaguely remember that the change to ordered keys in 3.6 was actually a _side
effect_ of making the dict implementation more efficient!

[https://mail.python.org/pipermail/python-
dev/2016-September/...](https://mail.python.org/pipermail/python-
dev/2016-September/146327.html)

------
makecheck
I’m a little surprised at this point that Apple still doesn’t include a
default Python 3.x on macOS. It’s the single thing keeping me from moving (as
there’s a big difference between “just run this” and “first download this,
then run this”).

~~~
rajacombinator
Downloading it is definitely not the biggest hurdle to moving to python 3.

~~~
semi-extrinsic
I infer GP was talking about his users on OSX, who would have to
download/install python3 to use the distributed software.

------
amyjess
I'd been a 2.7 holdout for ages, but when f-strings were greenlit for 3.6, I
decided then and there that all my new personal projects would be written in
3.

I'm glad I did. F-strings are wonderful, as is pathlib and the enhanced
unpacking syntax.

Since I started my current job, I've also been writing as many scripts as I
can in Python 3 as well (and Docker has been a godsend for that because I can
now deploy 3.6 scripts to the few servers we have that are still running RHEL
6).

~~~
memco
Could you provide more info on your setup for this? I work on some EL 6
servers and would be interested in using this setup.

~~~
Leynos
Not the OP, but replying to offer a suggestion in this respect.

I'll often do something similar to this, where I have a CLI tool that I'm not
ready to deploy server-wide yet, and has hefty dependencies.

The pattern I use is to have a wrapper shell script that calls:

    
    
        docker run -it --rm --volume "$(pwd)/$1:/file_to_process:z" --user $(id -u) container-image /opt/command_to_run /fileToProcess
    

This runs "/opt/command_to_run /fileToProcess" inside a container as the
current uid, mounting the parameter to the shell script as "/file_to_process"
inside the container.

The :z mount parameter may or may not be needed depending upon whether you
have SELinux enabled or not (by default, SELinux prevents countainers
accessing any file on the host, and :z changes the SE context to allow
access). I don't know if this is the case with EL6 tho.

The -t parameter shouldn't be used if your script is running in a pipeline (it
creates a pseudo-tty). So it may be worth having some kind of conditional to
remove this.

The wrapper I use also has a conditional to add the "$(pwd)" prefix to the
call parameter only if the parameter is a relative path.

------
pletnes
Several posters indicate that they’ve stuck to python 2.7 even for small side
projects until now. I cannot understand why? Python 3 seems to have been
technically superior for a few years, and side projects must surely be good
for learning something new?

~~~
CJefferson
I distribute python programs. Macs only have Python 2 by default.

~~~
wyclif
But surely that's not stopping anyone. As someone above said, just use pyenv
to run Python 2.7x and Python 3. It's not as if anyone has to settle for using
only legacy Python.

~~~
CJefferson
At the moment my instructions are "grab this script and run 'python
script.py'". That is ok for about everyone.

I don't want to have to start teaching pyenv to every academic or student I
want to send a script to.

------
eindiran
If you would like to write Python 3 but need to maintain support for any
particular version of Python 2, something that I can personally recommend is
using the Coconut transpiler: [http://coconut-lang.org/](http://coconut-
lang.org/) The Coconut language is a superset of Python 3, so any valid Python
3 is valid Coconut. But the advantage of the transpiler extends beyond the
language itself, in that it can target any version of Python from 2.6 onwards
on the Python 2 branch and 3.2 onwards on the Python 3 branch.

It has been really useful for me in that I want the advantages of type
annotations and f-strings and other python 3.5+ features but I have to support
running in an environment with only 2.6 installed. So when I target a 3.5+
version, all of those features are maintained, but when I target 2.6, the
transpiler does all the work in converting to running 2.6 code for me.

------
breatheoften
Didn’t know about the enforce library
([https://github.com/RussBaz/enforce/blob/master/README.md](https://github.com/RussBaz/enforce/blob/master/README.md))
— but have been wanting something like this.

Thanks for the useful list!

------
coppolaemilio
the Path thing is incredible! I wasn't aware of it. I'll definitively be
replacing my os.path.join calls for a more readable version of it.

~~~
ioddly
Wow, I switched to Python 3 really early on and never knew about pathlib.

~~~
sametmax
And the multiprocessing and threading pools.

And collections.ChainMap.

And f-strings.

And yield from.

And type hints.

And statistics.

And ipadress.

And secrets.

And matmul.

And subprocess.run.

Come on, Python 3 is packed with awesomeness !

~~~
dozzie
Every time somebody mentions how awsome f-strings are makes me laugh. It was
around ten years ago when Python community was looking down at Perl and shell
with their string interpolation, but now that Python got pretty much the same
it's suddenly _not considered_ a misfeature.

~~~
has2k1
Have you considered that maybe the language had not evolved enough for them to
be appealing. With b'' and u'' string syntax coming into the language, there
is a realisation that string interpolation can be an opt-in feature among
other reasons.

What you have is decision making analogous to the function

    
    
        evaluate(string_interpolation, python_ecosystem)
    

which is different from

    
    
        evaluate(string_interpolation)

~~~
dozzie
> With b'' and u'' string syntax coming into the language, there is a
> realisation that string interpolation can be an opt-in feature among other
> reasons.

This argument won't fly. Did you know that string interpolation in Perl and
shell was _always_ an opt-in feature? And despite that it was frowned upon by
Python community.

------
ggm
Came here to validate my own "I moved" experience: learned stuff I hadn't
checked on. TL;DR its never too late to learn what you can do, once you can
deprecate the past.

------
minimaxir
I'm not sure why this is targeted toward data scientists; these tips are
useful for any Python user.

~~~
tomlock
I think it's because the remaining python 2.7 users are mostly people using
the packages that were unsupported in 3, which I think were mostly packages
used in data science afaik

------
iandanforth
Is the np.dot -> @ tip reliable? I thought these were fairly different in
practice.

~~~
gh02t
What do you mean? `A.dot(B)` and `A @ B` are the same thing for NumPy arrays.
You might be mixing it up with the weirdness of `array` versus `matrix`, but
that's totally separate.

~~~
pletnes
The @ operator is the same as np.matmul which is different from np.dot for
matrices of rank >=3.

~~~
gh02t
Ah you're right, I had that mixed up. matmul should generally be the desired
behavior anyway I'd think, the previous behavior of dot was a bit weird IMO,
especially the behavior with a scalar. It's not backwards compatible, but I
think it'd be better to prefer @/matmul in the future anyway.

I almost think it'd be nice to make matmul undefined for ranks higher than 2,
since it's not really matrix multiplication and if you want to do that (or the
previous behavior of dot) it can be achieved with einsum, with the advantage
that you have to be a lot more explicit about what sort of tensor
multiplication you want. That's probably a bit too purist though.

------
jtchang
I had so many issues trying to install python3 in an existing server that I
ended up having to go back. pip kept complaining and it was just really
annoying. Then some libraries were not compatible and it felt like it wasn't
worth it.

~~~
setr
Isn't python 3 usually installed alongside python/pip 2.7 as python3/pip3, and
everything kept seperately?

~~~
wyclif
If you use pyenv, it is.

~~~
fastball
Even if you don't, any OS-level package manager should easily install Python3
alongside whatever the base install is without any conflicts as `python3` and
pip as `pip3`.

Homebrew, apt, pacman, etc. all have one-line python3 installation.

------
Noumenon72
Is there some kind of slash operator making this statement work? Is this a way
to concatenate things?

train_path = datasets_root / dataset / 'train'

~~~
necavi
Python allows the overriding of just about every operator. For Pathlib they
overrode the division operator to instead perform path addition in a platform
agnostic manner.

~~~
glandium
> Python allows the overriding of just about every operator.

Except the boolean operators (and, or, not). For instance __and__ overrides
the binary and (&), not the boolean and.

~~~
Cyphase
That's True, but you _can_ override __bool__.[1]

[1]
[https://docs.python.org/3/reference/datamodel.html#object.__...](https://docs.python.org/3/reference/datamodel.html#object.__bool__)

------
sdsad46464
Type hinting isn't part of Python 3, it's part of Python 3.5+.

~~~
ubernostrum
Syntactic support for annotating functions and exposing the annotations
existed as of 3.0.

The 'typing' module in the standard library was new as of 3.5.

Syntactic support for annotating variables was new as of 3.6.

Support for delaying resolution of annotations is new in 3.7 with a __future__
import.

Originally the annotation feature was seen as a possible way to add type hints
to Python, but other potential uses were envisioned and no immediate
preference was given to types over other uses of annotations.

~~~
viraptor
With type comments you can use it in 2.7 as well:
[https://www.python.org/dev/peps/pep-0484/#suggested-
syntax-f...](https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-
python-2-7-and-straddling-code)

Mypy supports it:
[http://mypy.readthedocs.io/en/latest/python2.html](http://mypy.readthedocs.io/en/latest/python2.html)

------
brett40324
I realized, just today, that the secrets module is new to 3.6 after trying to
pip install it. This being provided directly by the language is a game
changer, IMHO.

~~~
orf
It's interesting for sure, and I'd like to see where it goes, but right now
there isn't much to it:
[https://github.com/python/cpython/blob/3.6/Lib/secrets.py](https://github.com/python/cpython/blob/3.6/Lib/secrets.py)

------
gesman
<3 python 3 :)

------
donarb
Another feature is that the mock module is now part of the standard library,
no need to install.

------
julox
also you can use pythonconverter.com

