
PyPy gets faster - r11t
http://www.ripton.net/blog/?p=59
======
fauigerzigerk
Not that I'm a big fan of Jython, but including startup time in a benchmark is
only useful for very short running command line tools. It says nothing about
the speed of the JIT or the code that's being tested.

~~~
mzl
On the other hand, there are a lot of small scripts and utilities that are
written in Python, so it is a useful metric for quite a lot of use-cases.

~~~
fauigerzigerk
I don't see why I would want to benchmark those kinds of utilities.

~~~
mzl
Because the start-up time of the interpreter will affect you every time you
run one of those utilities.

~~~
fauigerzigerk
Sure, and I did mention command line utilities. The issue I see with the
benchmark is that it compares apples and oranges. For some programs it tests
JIT performance and for others it tests startup time. If I want to test
startup time, I just use a hello world program. To test JIT/interpreter
performance, I try to exclude startup time.

~~~
derefr
Hello world programs only ever rely on the bare runtime, though; real
utilities have to load libraries _after_ the runtime loads, which incurs
further delays (which might depend on JITing/interpreting speed if they're not
pre-compiled.)

------
kwellman
Has anyone done any recent memory benchmarks with PyPy?

I'd like to see magnitude of the memory trade-off for using a JIT compiler. As
a web developer my programs are mostly IO-bound, not CPU-bound. I'm also
bootstrapping and trying to squeeze as much as I can out of my 512MB linode.

~~~
silentbicycle
Comparisons with LuaJIT (<http://luajit.org/>) would be particularly nice.

~~~
metamemetics
LuaJIT is a nigh untouchable work of art, we might never see a faster dynamic
language JIT.

PyPy is significant because of the nature of Python itself: the language
specification is much more complex than Lua and there are currently many more
users and applications.

[the more complicated the language specification, the harder it's going to be
to prove things about, the harder it's going to be to write a compiler]

~~~
Raphael_Amiard
> LuaJIT is a nigh untouchable work of art, we might never see a faster
> dynamic language JIT.

Well Mike Pall disagrees in that he says there are still a lot of possible
optimizations. Also all the process of by hand optimization for specific
architecture can theoretically be automatized and decoupled. You're totally
right about Python complexity though.

------
jnoller
Competition in the python-vm space is _fantastic_ and I'm actually really,
really happy to see PyPy maturing.

------
Raphael_Amiard
That's really nice for PyPy. Although Eulers are mainly numerical tests. Most
of the work done in many programmers "real world" are work on strings, which
CPython is very good at since all its strings libs are implemented in C. So
i'd like to see a comparison with a better benchmark i guess :)

------
CWuestefeld
I'd like to see IronPython included in the comparison. (do they handle 2.7?)

~~~
dripton
Yes, I'll include IronPython next time.

IronPython is at 2.6. PyPy and Jython are at 2.5. psyco is at 2.6.

Most of my Euler solvers are compatible with Python 2.5. The ones that don't
work in 2.5 end up getting excluded from the benchmark, because the benchmark
only shows programs that worked and finished in less than a minute on every
tested Python.

------
random42
Just to be clear, what are the specified numbers? execution time in seconds?

~~~
dripton
Yes, execution time in seconds. And I throw out any program that fails on any
Python, or takes longer than a minute on any Python.

------
snissn
PyPy needs epoll!

~~~
kingkilr
epoll was new in CPython 2.6, PyPy currently targets Python 2.5, we have a
branch where we're working towards Python 2.7 support (we skipped a step),
that will include epoll support.

~~~
snissn
Ah, okay, that totally makes sense!

------
mikeklaas
Years and years later and they are finally able to (very slightly) beat
psycho. Sigh.

~~~
kingkilr
Sure, but we have a 64-bit backend ;) After years and years still no one wants
to write one for psycho.

Edit: Psycho also broke Python's semantics in a few very subtle ways, in that
sense it isn't a truly fair comparison.

~~~
mikeklaas
Hey, don't get me wrong, it's great that you're making strides, and I still
hope the project is a success.

My viewpoint comes as someone who was very excited initially at the thought of
a drop-in replacement for CPython whose goal was to be "faster than c". It's
been a very long time and it seems like pypy still has a long way to go to
achieve production-readiness, so it's hard to continue being excited about the
project.

~~~
andybak
Aside from the rapid performance gains in recent months and the fact that it's
twice as fast as CPython...

~~~
mikeklaas
... _in these benchmarks_ , which are mostly tight-loop math code, from what I
understand.

Being a drop-in replacement also includes things like virtual crashproofness,
full stdlib support, runs major third-party packages flawlessly, etc.

