
Optimizations which made Python 3.6 faster than Python 3.5 [video] - mmastrac
http://pyvideo.org/pycon-us-2017/optimizations-which-made-python-36-faster-than-python-35.html
======
std_throwaway
More important than the optimizations themselves is the instrumentation they
built up to reliably measure and monitor the performance.

One thing to keep in mind is that python 3 is still considerably slower than
python 2 ( _edit_ ) in some areas such as startup times while it has gotten
faster in most other benchmarks.

~~~
joshuamorton
Its worth watching the video in its entirety. That's not true. Python 3.6
beats 2.7 in most benchmarks, its just that there are a few places where 3 is
slower (like startup time) that make a lot of things look bad, but that don't
actually affect "speed" as most people would consider.

~~~
dman
The startup time fiasco was one of the reasons why Java never took over for
GUI apps. Even though its unscientific, time to first usable interaction goes
a long way in establishing "speed" in the users mind. You should see the hoops
that Chrome jumps through to excel on this metric.

~~~
kstrauser
True, but:

    
    
      $ echo exit | time python2
    

gives an average time of 0.015s on my laptop, and

    
    
      $ echo exit | time python3
    

works out to about 0.039s. If you're spamming hundreds of processes from a
looping shell script, the Python 3 overhead would probably start to grate. For
anything manually launched, 40 milliseconds is still substantially close to
instant.

~~~
nomel
I think the problem is pep8. For some reason, pep8 says you load your modules
at the top of the file rather than when needed. Giant libraries often follow
pep8, meaning something like "import pandas" loads hundreds of python files,
which gives the absurd startup times.

I've seen some large code bases take 4 seconds to import, > 80% being unused
code.

~~~
smitherfield
But couldn't that inefficiency be optimized away in the implementation? (Load
imports lazily).

~~~
marmaduke
It can't be done automatically because imported can execute code. Since that's
part of the semantics, you have to be lazy by hand.

~~~
deckiedan
Only on the first load ever. When modules are compiled to their byte code,
they could get a "clean" flag if they don't execute code, which means they get
loaded lazily next time.

~~~
marmaduke
Ok but figuring out what executes code might require executing code: consider
non trivial use of meta classes, where an inconspicuous subclass actually
invokes e.g. A registration process like a Django model class.

~~~
smitherfield
Sure, but, just spitballing here, could a substantial fraction of imported
code benefit even with a very conservative heuristic?

In any case, given your points, it seems like a future Python version ought to
introduce a new version of the import statement with lazy semantics (which,
besides eliminating dead code, is also IMO the more correct/explicit behavior
when importing symbols).

~~~
marmaduke
It'd be pretty easy to do the experiment, since you can override the import
logic and do lazy imports, eg.

    
    
        class LazyModule:
            def __init__(self, name):
                self.name = name
            def __getattr__(self, key):
                mod = {}
                exec('import %s as mod', mod)
                self.mod = mod['mod']
                return getattr(self.mod, key)
    
        class LazyImporter:
            def get_module(self, name):
                return LazyModule(name)

------
rplnt
More reading linked at the end of the slides: [https://faster-
cpython.readthedocs.io/](https://faster-cpython.readthedocs.io/)

------
DonbunEf7
Ah, yes, the "rewrite it all in C" theory of optimization. Meanwhile, PyPy
continues to be a thing, writing most of its core in RPython and spanking
CPython in benchmarks. Why do we keep doing this?

~~~
ciupicri
From [https://faster-cpython.readthedocs.io/pypy.html](https://faster-
cpython.readthedocs.io/pypy.html) (emphasis mine):

> \- _Bad support of the Python C API_ : PyPy was written from scratch and
> uses different memory structures for objects. The cpyext module emulates the
> Python C API but it’s slow.

> \- _New features are first developed in CPython_. In january 2016, PyPy only
> supports Python 2.7 and 3.2, whereas CPython is at the version 3.5. It’s
> hard to have a single code base for Python 2.7 and 3.2, Python 3.3
> reintroduced u'...' syntax for example.

> \- _Not all modules are compatible with PyPy_ : see PyPy Compatibility Wiki.
> For example, numpy is not compatible with PyPy, but there is a project under
> development: pypy/numy. PyGTK, PyQt, PySide and wxPython libraries are not
> compatible with PyPy; these libraries heavily depend on the Python C API.
> GUI applications (ex: gajim, meld) using these libraries don’t work on PyPy
> :-( Hopefully, a lot of popular modules are compatible with PyPy (ex:
> Django, Twisted).

> \- _PyPy is slower than CPython on some use cases_. Django: “PyPy runs the
> templating engine faster than CPython, but so far DB access is slower for
> some drivers.”

~~~
eat_veggies
A lot of the db drivers are written in C so that would make sense. It's really
just the same problem.

