

Maciej Fijalkowski's view on PyPy's future - pieceofpeace
http://lostinjit.blogspot.com/2011/12/personal-view-on-pypys-future.html

======
drats
PyPy will likely be complete enough to use by the end of 2012, if not sooner.
By that I mean most of the major libraries will work with it and people will
ask themselves "why wouldn't I want a massive speedup?". The one reason why
they might not is memory usage: it can be 10 times as much. That's significant
in quite a few applications.

As it's beyond my ken I would like to know from someone capable of answering
if this is going to be a constant constraint, stemming from the design of the
project, or if memory usage is likely to go down at some point.

~~~
viraptor
I believe the memory usage has been adressed lately (either last release, or
shortly after). There was an article about it on HN, which i can't find atm.
Do you mean some specific use cases?

~~~
drats
<http://news.ycombinator.com/item?id=3349429>

If you search for my username in that thread I asked for someone to run the
code from the story on a similar dataset with PyPy. Twice as fast but x10
memory usage. But as I said elsewhere a bit of searching suggests it won't be
a showstopper.

~~~
reitzensteinm
Be careful extrapolating from such small examples, where the memory usage is
dominated by fixed overhead. Running Django or something similar is more
likely to give a real world result, and even then, a bunch of tests is far
better.

3mb vs 30mb might be par for the course for smaller programs, but either is
negligible.

------
binarycrusader
In my opinion, they could get more contributors if they simplified the rather
byzantine build process for pypy. It's especially annoying for developers
attempting to port to other platforms.

(No incremental build, it takes roughly three hours on a high-end nehalem
workstation, and if it fails for any reason, you get to start all over again!)

------
spenrose
What should we make of the failure of PyPy's users to fund Maciej's work? The
value of PyPy seems much, much greater than one engineer's salary.

~~~
drats
I find it puzzling as well. Google heavily uses python, as do a number of
other web companies. Further, Ubuntu and Redhat have it as a system
administration language. It's almost the default language for O'Reilly books
that aren't language specific. Given all that you'd think there'd be a few
more corporate contributions now that a x5 speedup has already been proven and
it seems to be just a matter of polishing it up.

A bit of searching and it seems the memory problems I raised in my other
comment aren't so drastic after all. Theoretically it can use less memory in
many operations and the current blowouts are not as high as I thought (I was
going from one benchmark a HNer, brianh, was kind enough to run for me[1]).

[1]<http://news.ycombinator.com/item?id=3357160>

~~~
cpeterso
> I find it puzzling as well. Google heavily uses python, as do a number of
> other web companies.

Especially after Google invested engineers on Unladen Swallow, which fizzled
out. I've read that Google likes to max out their servers to the point where
OOM is not unlikely, so PyPy memory usage might not be worth the runtime
performance gains.

~~~
cdavid
More likely, google requires compatibility with its set of C extensions. At
least when unladen swallow was announced, this was one of the major
requirement. I would expect the problem to be similar in most large corporate
environments.

------
ramanujan
This might be a contrary view, but Cython has been ready for production for
quite a while while PyPy has still been getting its boots on:

<http://cython.org/>

It actually works right out of the box. You can download it and get real world
examples to work in five minutes. It works with C++. It works with numpy.

<http://wiki.cython.org/WrappingCPlusPlus>

<http://wiki.cython.org/tutorials/numpy>

It's the real deal: ridiculously easy to use as a drop-in replacement for slow
functions. It works on problems that are not toys. It allows you to use high
performance C and C++ libraries, it's used in a large and well maintained
project (sage), and development was at least partially funded by Google.

With all due respect to Maciej for his work, PyPy has been in the works for
many years now, and projects like that don't tend to ship -- or to meet
expectations if they actually do ship.

~~~
rdtsc
I think you are being unfair. PyPy is something you can use on your code
without modifying your code. That's a big thing. And by big I mean huge.
Paying someone $70/hour to modify code is expensive. Switching "python"
executable with "pypy" is a lot cheaper than profiling code then re-writing it
in Cython. Yeah you might still end up having to do that but who knows, you
might not have to or you might even be running faster than C (yes in theory
jit compiler could do that).

------
madarco
I've tried Pypy to speed up a background worker that create collages of
300-500 images, this is my experience:

\- The JIT optimization doesn't survive between different runs of the script.
Since I run a different script for every collage, I haven't see a noticeable
speedup.

\- The script also download the 300-500 images from facebook with curl, this
module is not supported yet by pypy

\- Since the most of the time is spent on images resize (PIL.resize) which is
in C, the JIT can't do many optimizations.

------
thealistra
It's similiar to how clang tries to be a gcc replacement. I think clang got
started in ~2008 and now you can build most of the open source projects with
it. But still some packages use GNU extensions or they depend on some gcc-only
features.

I think the reason it works for clang is because they're really trying hard to
be drop-in compatible with gcc. I remember reporting some minor thing like:
command line flag -- in gcc generated a warning, but it didn't return an error
code on exit and it did in clang, so they've changed it, because it caused
crashes some project builds.

Not sure how pypy handles the compatibility now, because I couldn't compile it
yet, tried like 4 times already and it always crashes randomly or it gets out
of memory or something.

------
Gerdus
I don't think speed is a big enough pain point for the python community at
large to justify the pain of moving from cpython.

PyPy needs to use their great implementation stack to develop their own killer
feature. Like for instance a small and fast python for embedded scripting or
compiling normal python program to a small and optimized native executable.

~~~
fijal
So far just faster python have been a killer feature for some people :) True,
it does not fit anyone, but hey, nothing does.

