
PyPy.js: A fast, compliant Python implementation for the web - doublextremevil
http://pypyjs.org/
======
btown
This is glorious. From
[https://github.com/rfk/pypyjs/blob/master/CONTRIBUTING.rst](https://github.com/rfk/pypyjs/blob/master/CONTRIBUTING.rst)
:

> We have the following major components in the PyPy repo:

>> An "emscripten" build platform definition, which teaches pypy's rpython
toolchain how to compile things with emscripten:
./deps/pypy/rpython/translator/platform/emscripten_platform/.

>> An rpython JIT backend that emits asmjs at runtime:
./deps/pypy/rpython/jit/backend/asmjs/.

>> A "js" builtin module for the resulting interperter, to allow interaction
with the host javascript environment: ./deps/pypy/pypy/module/js/.

>Along with these wrappers to help working with the resulting interpreter:

>> A wrapper to load up the compiled VM and expose it via a nice javascript
API: ./lib/pypy.js.

>> A script for bundling python modules into an indexed format that can be
easily loaded into the browser: ./tools/module_bundler.py.

In
[https://github.com/rfk/pypy/blob/master/rpython/jit/backend/...](https://github.com/rfk/pypy/blob/master/rpython/jit/backend/asmjs/assembler.py)
alone, there are 2887 lines of custom Python code to emit asm.js for function
and block JITing.

And at 3.26 seconds to initialize from cache on a 4-year-old machine, 5MB
gzipped (not too much worse than your average high-quality gif) and only 1.5x
slower than CPython... this is actually approaching viable if you needed to
run Python business logic in the browser. Which I never thought I would see.

~~~
mutagen
Glorious indeed. At least three levels of abstraction, two JITs, and a host of
supporting code. Can we add some more layers like an emulator or get recursive
by getting something like Pyasm running inside it?

~~~
nudpiedo
I work on a machine that has two hypervisors, one third in plan, and more
abstraction layers than a lasagna software.

It is still very very efficient.

~~~
wfunction
How can you have multiple hypervisors...? Do they both run in ring -1?

~~~
mschuster91
Heh, that could be a good trick against hypervisor malware. Just run inside
your own hypervisor...

------
brachi
I watched the presentation in Pycon 2015[1], and I really liked how
honest/aware they are about the challenges and tradeoffs (so far, the
performance it's pretty bad compared to javascript, and the download size is
still an issue). At the end is answered why python can't be directly included
in Firefox or other browsers(the presenter/main developer works for Mozilla)

[1][https://www.youtube.com/watch?v=PiBfOFqDIAI](https://www.youtube.com/watch?v=PiBfOFqDIAI)

~~~
vog
_> python can't be directly included in Firefox or other browsers_

So Python is a bad candidate, but there are better candidates out there. For
example, jsofocaml (using OCaml in the browser) looks quite promising.

In the end, we need a lightweight yet properly defined and highlevel language,
and the ML languages as well as the LISP languages are great candidates for
that.

~~~
maximilian
Purescript is a Haskell-like language that is designed to output to
javascript. It looks quite nice and has seen a lot of attention lately. It's
even in GSoC.

~~~
dheera
The problem I have with "output to Javascript" languages is that debugging
becomes hell. Line numbers no longer correspond to each other, for example.

A slightly less-hellish but still annoying problem is that I often poke at
things directly from the console when bugs happen, and this helps me a lot in
debugging. However poking at the console requires using JavaScript, which is
fine, but it would drive me nuts to have to code in one language (e.g.
CoffeeScript, TypeScript, Python) and debug in another (JavaScript) and switch
my brain back and forth every few minutes. So I've ended up largely sticking
with writing code directly in JavaScript.

Any good solutions to this? Are there replacements to the Chrome JavaScript
console that supports those other languages?

~~~
Psyonic
Are you using source maps? If so, you can actually put breakpoints in your
other source and Chrome will still break on it. I've even done this with
Scalajs.

------
bayesianhorse
Besides the big download size, it does several things very "right": Full
Python data model, access to the full javascript model (albeit slowly
currently).

Also PyPy is the right choice as the basis for this (as opposed to CPython)
because its ecosystem is generally more "pure" Python than CPython's. C
Extensions would only get in the way.

~~~
fijal
it's systematically translating the PyPy interpreter to the Javascript, so it
_has_ to get semantics right. However I'm not sure if the drawbacks can be
addressed in the current JS model, notably: very slow warmup, huge download
size (that's primarily PyPy fault though), the lack of good JS access.

~~~
jychang
> However I'm not sure if the drawbacks can be addressed in the current JS
> model, notably: very slow warmup, huge download size

Only for the first time loading it.

It would be great if some big organization (looks at Google) hosts the
"standard" version of the file, much like jquery, so that it would be cached
and ready to be used on the local machine most of the time.

~~~
fijal
I believe this only addresses the download size and not the crazy warmup (you
still reparse it etc.)

~~~
TazeTSchnitzel
Not quite. At least Firefox caches compiled asm.js.

~~~
fijal
You can't cache JITted code (so JIT produced by pypy.js) because it'll be
different each time.

~~~
TazeTSchnitzel
Ah, sure.

------
jedberg
So does this mean that since I'm better at Python than Javascript, I can now
use this to make my web frontends in Python? Because that would be pretty
sweet.

~~~
riffraff
weren't you already able to do it with pyjamas/pyjs?

~~~
jedberg
Quite possibly. To be honest I never looked into it.

~~~
StavrosK
Being a Python guy myself, the problem was never the lack of Python in the
browser, it's always the lack of a Python _ecosystem_ in the browser. If I'm
going to use Python and awkwardly interface with JS, I might as well just use
JS.

It makes me want to gouge my eyes out, but it's not really that hard to get
productive in. CoffeeScript is even better, if you aren't averse to adding
another dependency on top of everything else.

------
nosir33
Sample code to execute some Javascript code inside PyPy.js. I couldn't find
any examples.

    
    
        import js
        js.eval("console.log(\"hi!\")")

~~~
fragmede
It's a bit messy, but:

    
    
        js.eval('var div = document.createElement("div"); div.style.width = "100px"; div.style.height = "100px"; div.style.background = "red"; div.style.color = "white"; div.innerHTML = "Hello"; document.body.appendChild(div);');
    

manages to append a div.

\--

Directly modifying document.body.innerHTML in any way seems to hang it. Both:

    
    
        js.eval('document.body.innerHTML += "<div></div>";');
        js.globals.document.body.innerHTML += "<div></div>"
    

Cause it to hang, though the javascript console shows it was added (or using
an img tag instead).

(ChromeOS v43 beta)

~~~
batbomb
It's more fun doing this, since jQuery is already on the page:

    
    
        import js
        jq = js.eval("$")
        first_line = jq(jq(".container p")[0])
        first_line.text("jQuery was here")

------
Nitramp
I think there is no point in trying to emulate different language semantics on
top of JavaScript.

asm.js is a weird hack that lives in a totally different world than actual
browser APIs (e.g. the DOM), and you have to emulate your entire runtime and
environment to get anything useful, which means your binary is going to be
huge (as in this example, but similar things apply).

Emulating better semantics than JS based on JS usually leaves you in this
uncanny valley where you either trade off performance and size for nicer
semantics (e.g. killing the null/undefined dichotomy, or 64 bit integer math),
or you end up with odd semantics that don't quite fit the language you're
compiling from, or a mix of both, which is a usability disaster.

So the only real chance to improve on the state of JS is being a syntactical
shim on top of JS and help users with better syntax, static analysis, and
compile time tooling. That'd be TypeScript.

~~~
fragmede
> That'd be TypeScript.

Or CoffeeScript. Or Dart. Or this. In fact, here's a whole big long list of
other languages. Most of which don't use the asm.js "weird hack".

[https://github.com/jashkenas/coffeescript/wiki/List-of-
langu...](https://github.com/jashkenas/coffeescript/wiki/List-of-languages-
that-compile-to-JS)

~~~
alexweber
_one weird hack_ to run assembler-like code in the browser!

 _developers hate him!_

------
Tossrock
Looks like there are some issues to iron out...

    
    
      >>> from datetime import datetime
      >>> datetime.now()
      Traceback (most recent call last):
        File "<console>", line 1, in <module>
        File "/lib/pypyjs/lib_pypy/datetime.py", line 1548, in now
          return cls.fromtimestamp(t, tz)
        File "/lib/pypyjs/lib_pypy/datetime.py", line 1522, in fromtimestamp
          result = cls(y, m, d, hh, mm, ss, us, tz)
        File "/lib/pypyjs/lib_pypy/datetime.py", line 1459, in __new__
          hour, minute, second, microsecond)
        File "/lib/pypyjs/lib_pypy/datetime.py", line 312, in _check_time_fields
          raise ValueError('microsecond must be in 0..999999', microsecond)
      ValueError: ('microsecond must be in 0..999999', 1716000)

~~~
caryhartline
That works now.

~~~
Tossrock
Does it?

    
    
      >>> from datetime import datetime
      >>> datetime.now()
      Traceback (most recent call last):
        File "<console>", line 1, in <module>
        File "/lib/pypyjs/lib_pypy/datetime.py", line 1548, in now
          return cls.fromtimestamp(t, tz)
        File "/lib/pypyjs/lib_pypy/datetime.py", line 1511, in fromtimestamp
          us = _round(frac * 1e6)
        File "/lib/pypyjs/lib_pypy/datetime.py", line 28, in _round
          return int(_math.floor(x + 0.5) if x >= 0.0 else _math.ceil(x - 0.5))
      ValueError: cannot convert float NaN to integer

------
nighthawk454
Sadly, "import antigravity" doesn't work

~~~
e12e
I was more disappointed (but not surprised!) that "import pip" didn't work.

If it did (and it has big ramifications for how much of "python" (for some
definition of "python" that includes most of the standard library) works. Eg
sockets, interacting with some sort of "local" "file-system" etc) one could
(this is handy if working on windows, and running python via cmd-r
python.exe):

    
    
      import pip
      # pip.main expects a list of arguments, hence .split() 
      pip.main("install ipython".split())
         # Only needed first
         # time, as it installs ipython under site-packages.
         # Only works if the user has write access, obviously
    
      import IPython
      IPython.start_ipython()

~~~
toyg
That would be a total security nightmare and hopefully it's never going to
happen.

I'd be happy enough with serving .py files with the exact same restrictions
and privileges that JS libraries have now.

~~~
fragmede
> That would be a total security nightmare and hopefully it's never going to
> happen.

Why? You're assuming a lot of things about a hypothetical implementation that
is going to have heavy restrictions on in anyway since it's running in the
javascript sandbox.

------
dulvac
I can imagine a way to include language extensions in the browser and have
them loaded up lazily in a sandbox. And a checkbox that says "Allow websites
to run code" or something. Then you could develop an extension for your
favourite language and either install them manually or have some popular ones
bundled with the browser.

If we're going to run everything in the browser, why use javascript as the
low-level language that stuff compiles to?

I know portability and existing standards are a limitation, but those can be
influenced by the 3 (?) major browsers if needed.

I think asm.js is an awesome cute little thing, but the fact that it's taken
seriously is worrisome to me.

Am I completely crazy here?

------
bootload
If software is eating the world, JavaScript is eating the languages.

~~~
zo1
Only because the browser vendors are too scared (for whatever reason,
legitimate or otherwise) to add extra languages to the mix, like python.

~~~
bootload
_" vendors are too scared (for whatever reason, legitimate or otherwise) to
add extra languages to the mix, like python."_

Tried python server-side, mod_python and got lots of issues with formatting.
Not syntax or mixed tab/spaces but one-off spacing errors. I realised then,
Python used this way has problems. Could be fixed though.

~~~
zo1
One-off spacing errors? That doesn't sound right at all... Do you have an
example?

~~~
bootload
ages ago @zo1, I do see similar problems now if I shift from *nux/vim
toolchain to win/idle which by default uses 'Python standard: 4 spaces!' but
still throws random format error parsing on default code edits.

I don't like the fact python code stops working randomly if I write it in one
environment then touch (edit) in another. Never have this problem with C or
JS.

~~~
techdragon
I have only ever had issues like that when I worked on an actual Python
checker where I had to keep BAD code in my tests to ensure the checks worked.

I've never ( read 99.999% of the time I've performed the task ) had issues
with Python formatting that weren't my own fault, everyone makes mistakes
after all. I routinely edit the same Python files in Vim, Nano, Sublime Text,
Atom, and PyCharm, and even jumping between all those editors, they have never
"damaged" the formatting and created syntax errors in my code.

I can only surmise that some of the things you used or even just your vim
config was setup poorly for Python work.

~~~
bootload
_" I routinely edit the same Python files in Vim, Nano, Sublime Text, Atom,
and PyCharm, and even jumping between all those editors, they have never
"damaged" the formatting and created syntax errors in my code."_

Do you routinely change OS?

------
henryaj
Please humour my lack of knowledge: is there much difference between something
like this and Opal[0]?

[0] - [http://opalrb.org](http://opalrb.org)

~~~
Veedrac
PyPy.js is not a transpiler, it is a JITted interpreter. This has correctness
advantages since the major work is upfront, and potential throughput
advantages, but is results in huge upfront download and warmup costs.

------
pc2g4d
Not working for me:

"Cross-Origin Request Blocked: The Same Origin Policy disallows reading the
remote resource at [http://10.30.229.27:15871/cgi-bin/blockpage.cgi?ws-
session=1...](http://10.30.229.27:15871/cgi-bin/blockpage.cgi?ws-
session=18446744072937828300). This can be fixed by moving the resource to the
same domain or enabling CORS."

Firefox 37.0.1

------
acron0
Obligatory link to Atwood's Law: [http://blog.codinghorror.com/the-principle-
of-least-power/](http://blog.codinghorror.com/the-principle-of-least-power/)

Although I appreciate that this isn't technically "written in JavaScript" but
it follows the principle.

------
MBlume
I'm curious -- since the custom emitter is tied to the RPython JIT and not to
the Python interpreter, could one easily plug in the existing Ruby or PHP
interpreters written in RPython and get similar performance?

------
estomagordo
Unfortunately, this is of little use to me. Why? Well, with my Swedish
keyboard layout, some alt-combinations are crucial (mainly square brackets,
[]). And these keypresses do not seem to work.

~~~
Veedrac
You can try `vm.eval` from the JS console.

------
hurin
>>> def foo(x,y):

... return x,y

debug: OperationError:

debug: operror-type: SyntaxError

debug: operror-value: ('EOL while scanning string literal', ('c callback', 1,
12, "r = c.push('def foo(x,y):\n", 0))

>>>

------
acbart
I use Skulpt for a lot of my research - I'd love to see a comparison against
it in terms of speed, accuracy, etc.

------
gdw2
Can the client-side python access the DOM?

~~~
venti
yes it can: import js js.globals.document.getDocumentById("...")

------
dlitz
Is there a Python 3 version of this?

~~~
venti
Yes, in his talk at
[https://www.youtube.com/watch?v=PiBfOFqDIAI&t=1355](https://www.youtube.com/watch?v=PiBfOFqDIAI&t=1355)
Ryan Kelly already showed a Python 3 version of pypy.js

------
forgottenuser
How is this different than Brython or Skulpt?

Also python implementations that run in the browser.

[1] skulpt.org [2] brython.info

------
xiaq
The next level is to run a Python implementation of JavaScript inside that...

------
nirai
Doesn't seem to work well on chrome on a modern tablet.

------
raymondh
This is pretty cool.

------
retrogradeorbit
6.6 Megs of binary blob download (pypy.vm.js.mem). Plus 593K of index.json.
Yuck.

Curious. Interesting tech. But hardly practical.

------
morekozhambu
This is super awesome.

