
Python 3 at Facebook - wyldfire
https://lwn.net/SubscriberLink/758159/f1f631e1535ab9d6/
======
mark-r
> When they ran the resulting code, they found it was 40% faster and used half
> the memory. This points to a persistent myth that Fried has heard: Python 3
> is slower than Python 2. That may have been true for earlier releases of
> Python 3, but it is definitely not true now, he said.

I wonder why that is? My theory is that it has to do with the new Unicode
encoding; if 99% of your strings are ASCII compatible, the new encoding will
be twice as efficient. Seems unlikely that would account for all of it though.

~~~
pletnes
Optimizations of the dict class (both memory and cpu wise) in python 3.5+ are
pretty important, as it’s used a lot internally. E.g class members and methods
are stored in a dict. This basically affects all python code.

~~~
rectangletangle
The dictionary implementation changes are the biggest speed up I can think of.
They touch nearly everything in the language.

One of the interesting side-effects of this, is in 3.7+ dictionaries are now
ordered by insertion, which I'm sure will help fix tons and tons of subtle
non-deterministic bugs (and probably expose a few too).

Although I'm not 100% on the performance characteristics of the new unicode
implementation. The unicode first string handling is a _massive_ improvement
when it comes to non-English languages. So IMO it's worth a hefty performance
hit, based solely on it's own merits. In practice, it hasn't seemed to make a
meaningful difference to me. So I'd wager it's good enough, unless you're
hitting some sort of use case specific bind. In which case, it's probably time
to leverage the C bindings anyway.

~~~
danso
Is there a good, in-depth technical write up of the changes made to dict
implementation?

~~~
philipov
Not a write-up, but how about a pycon talk from one of the senior core devs?

[https://www.youtube.com/watch?v=npw4s1QTmPg](https://www.youtube.com/watch?v=npw4s1QTmPg)

~~~
danso
Thanks, I remember Hettinger's video making the rounds last year but never got
around to watching it. Guess it's about time to make time for it.

edit: Ah, I remember. His talk uses slides that were created in
sphinx/readthedocs, but IIRC, the link to those docs never worked for me:
[https://twitter.com/raymondh/status/867021035719110656](https://twitter.com/raymondh/status/867021035719110656)

------
eiji
> He wanted to tell the "story about how I and couple of engineers used our
> free time, with no authority whatsoever, and made Python 3 the dominant
> version at Facebook."

Isn't it great when engineers upgrade your codebase in their free time.

~~~
alexgartrell
I'm actually fairly certain it was "within working hours" free time. Facebook
doesn't have 20% time or anything like that, but, in practice, there's lots of
extra time for whatever you want. This was my experience as an Engineer at
Facebook and I certainly "pay it forward" as a manager.

Probably worth adding that we ultimately created a python foundation team, of
which Jason is a member.

Here's something dumb I created in my "within working hours" free time that
Facebook let me open source:
[https://github.com/facebookresearch/py2bpf](https://github.com/facebookresearch/py2bpf)

~~~
_sdegutis
I've always had the drive to do things like this in my spare time. I'd love to
get paid for solving technical problems for an employer. I think besides the
obvious, I'd make a good candidate for an employee at something like Facebook.

More companies should encourage hiring developers who would solve problems in
their spare time. When I applied for an apprenticeship at 8th Light, they had
me write a tic tac toe game, and I wrote it in 4 or 5 different languages,
just for fun. I think that's why they hired me.

Man, it would be good to have an employer. Any takers?

------
bertr4nd
Ha! The 2013-era linter he mentions that required “from __future__” imports
was actually one of my bootcamp tasks! Hilarious and kind of gratifying to
stumble across it in a random HN post all these years later.

------
aidos
That’s a pretty large juggernaut to try to change. I’m surprised and pleased
they managed to make it work.

I wasn’t aware that Facebook used python so heavily. Presumably much of their
code is in Js these days?

~~~
holografix
I’d love to know what an entry level Python coder would earn and need to know
to get a job at Facebook.

I’m so tempted to try and get a job in software eng sometimes, it’s by far
what I have the most fun doing.

~~~
kajecounterhack
FWIW larger companies don't hire "x coders" where x is some language. They
look for CS fundamentals, mastery of programming paradigms, and then you work
in whatever language you need / makes sense given the task.

~~~
praneshp
> then you work in whatever language you need / makes sense given the task

I think this simplifies the truth too much. At least in my limited experience,
you work in whatever language your team is working in.

~~~
joshuamorton
Depends, I'm currently coding almost entirely in python, the guy behind me is
using almost solely C++, and the person to my right is working solely in Java
right now. Last week I was split heavily between python and javascript, but
will likely write very little JS in the near future. At that point, another
person on my team was working heavily in Go, although that is also finished
now.

Its not just "what your team works on", but what fits the task/what the
existing tooling is in. If you're only working on a single product/project ("I
develop the android app for XYZ product"), you likely work in the same
language or two as the rest of your team. But that isn't always the case.

------
NelsonMinar
I was surprised there was no mention of six.py. In my experience it's been a
key enabler for 2->3 transitions. But maybe not at Facebook?

~~~
cup-of-tea
There is also python-future which does similar things, I think.

~~~
andrewshadura
Yes, and python-future does it in a more Pythonic was, it seems to me

------
metabagel
I don’t believe it’s kosher to provide a subscriber link to everyone on HN. I
believe it is intended to be used to communicate with a few colleagues, not to
be posted on a news aggregator site.

Per LWN.net:

“The "subscriber link" mechanism allows an LWN.net subscriber to generate a
special URL for a subscription-only article. That URL can then be given to
others, who will be able to access the article regardless of whether they are
subscribed. This feature is made available as a service to LWN subscribers,
and in the hope that they will use it to spread the word about their favorite
LWN articles.

If this feature is abused, it will hurt LWN's subscription revenues and defeat
the whole point. Subscriber links may go away if that comes about.”

~~~
dmurray
Sharing a single subscriber link and having it hit the front page of HN is
unquestionably good for LWN and its revenue. Which is, as your quote says, the
whole point.

~~~
xapata
There's probably some parabolic curve of revenue vs frequency of sharing
subscriber links.

------
hyperbovine
To save you the trouble of Googling: this is _not_ the Jason Fried that more
or less co-invented Rails.

~~~
cityzen
I don't think Jason Fried co-invented Rails, I think that was all DHH.

------
johnramsden
Repost of
[https://news.ycombinator.com/item?id=17413350](https://news.ycombinator.com/item?id=17413350)

~~~
trevor-e
Why does it say this was posted 2 hours ago and yours 1 hour ago?

~~~
sctb
[https://news.ycombinator.com/item?id=17419144](https://news.ycombinator.com/item?id=17419144)

It's rare that we rescue an older story and while it's waiting to get re-upped
another repost gets traction, but that's what happened.

------
trumped
Python is really slow, but luckily, most python code can be optimized by
improving the interpreter... maybe Facebook will release a better interpreter
(there is already some faster interpreters, not sure why Python doesn't adopt
them)...

~~~
blake_himself
There was an article somewhere recently, of someone proposing changes to
CPython for just that reason. Guido had already resisted several attempts at
the kind of thing he was doing, from a desire to keep the CPython
implementation simple.

