

PyPy IO improvements - jerven
http://morepypy.blogspot.com/2014/11/pypy-io-improvements.html

======
arthursilva
kudos to PyPy team, always delivering the goodies. I just wished it was more
CPython (extension) compatible.

~~~
jerven
I hope that the "pinning" work in the GC code is a first step for better
CPython extension support. Classic C interop requires that some memory pointed
from the C side in the Python heap stores the same thing. i.e. is not moved.
Pinning is one way to do this.

Otherwise the same idea that Jruby/Graal did with co interpreting C code can
work for PyPy as well
([http://www.chrisseaton.com/rubytruffle/cext/](http://www.chrisseaton.com/rubytruffle/cext/)).
Although that would need someone to step up and implement a RPython C
interpreter, which to be honest is not a trivial exercise.

~~~
masklinn
That's not really a solution when using precompiled native extensions. Not
much of a solution either when the native extension (precompiled or not) is a
wrapper against a separate native library (which may or may not be coded in
C).

Basically, it only works with "native accelerators", libraries which provide a
pure {language} implementation as well as an optional self-contained C version
of a subset of the library. Which is a case where you'd hope a high-quality
JITed implementation would have a good speed on the "pure" implementation (and
would probably prefer work be done on improving the JIT to make the pure
version fast)

~~~
chrisseaton
It doesn't work against precompiled native extensions, but it does work
against wrappers against separate native libraries, like a database driver or
libpng or something like that. The C interpreter can interface with native
code, and the JIT has special support for making those calls very fast [1]
(it's not like JNI, for example).

So the only thing it doesn't work for is precompiled native extensions where
you don't have these source code - so maybe a proprietary C extension. Do you
get many of those in practice?

[1]
[http://dl.acm.org/citation.cfm?id=2500832](http://dl.acm.org/citation.cfm?id=2500832)

~~~
masklinn
> So the only thing it doesn't work for is precompiled native extensions where
> you don't have these source code

Or you can't build it because you're on windows, or the extension was
installed precompiled by the package manager (on Python, that's exactly one of
the use case for wheels, I don't think wheels ship C code ever), or the
extension is coded in non-C (which is very much explicitly supported by
CPython:
[https://docs.python.org/3/extending/extending.html#writing-e...](https://docs.python.org/3/extending/extending.html#writing-
extensions-in-c))

~~~
maxerickson
Microsoft has done work to make the Windows situation at least incrementally
better:

[https://mail.python.org/pipermail/distutils-
sig/2014-Septemb...](https://mail.python.org/pipermail/distutils-
sig/2014-September/024885.html)

(I'm crassly attaching a bit of information to your post more than trying to
respond to anything in it)

