Hacker News new | comments | show | ask | jobs | submit login
PythonJS now faster than CPython (pythonjs.blogspot.com)
138 points by tyrion on June 8, 2014 | hide | past | web | favorite | 60 comments

Can it run real programs (i.e. not benchmarks)?

If so I will be impressed. Glancing over the code, it looks pretty short, and I can imagine benchmarks will run, but not real programs.


It's true that most programs don't use all the dynamism of Python, but they probably depend on something that does (e.g. Django). Python is more dynamic than JavaScript, in that it has __getattr__, __setattr__, __getitem__, etc.

If not, it's not fair to compare it to PyPy... PyPy actually runs arbitrary Python programs.

Javascript (6) has those in the form of Proxies [0]... Coming soon! I think that they cover the set of features you indicate anyways.

However, you can already hack them in and there are multiple js projects that let you compile ES6 down to ES5, proxies included.

As other commenters noted, python is not "more dynamic" than javascript. You can easily transform python code that does unusual metaprogramming stuff to javascript code that does similar metaprogramming stuff but looks significantly different; basically all languages that support runtime-eval will let you imitate all other metaprogramming features in a somewhat verbose fashion.

[0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

> However, you can already hack them in and there are multiple js projects that let you compile ES6 down to ES5, proxies included.

Of course, but will it still be as fast? For example we can trivially turn a Python expression "a + b" into a Javascript expression "a + b" to get a great speed benefit. It may even give correct results in benchmarks, but in general it breaks the semantics and therefore any frameworks which rely on it.

FYI in Python "a + b" is equivalent to "a.__add__(b)", and hence the correct Javascript would look more like "a.__add__(b)", but this requires a property lookup and a function call so a lot of the speed gains may be lost (although some benefits may be regained by JITting).

Considering that the parent pointed out how small this project's codebase is, it does give legitimate concern about how many rules there are for mapping Python's semantics to Javascript's. For example, the ShedSkin project has been around for years and gets very impressive benchmark results since it compiles Python to C++; however, it tries to map Python classes directly to C++ classes, which breaks the semantics immensely.

It's even more complicated than a.__add__(b), due to __radd__: https://www.inkling.com/read/learning-python-mark-lutz-4th/c...

Which is only the beginning of a full mapping since the method is actually a descriptor. The method lookup has to go through a bunch of rules that mean we get the correct descriptor behaviour in a given context.

> Python is more dynamic than JavaScript, in that it has __getattr__, __setattr__, __getitem__, etc.

No, it's not. You can do equivalents to all of those in Javascript. Javascript objects are all mutable with a few exceptions. They are just maps of properties, very similar to Python's. Properties can be added or changed at runtime.

For example, setattr(obj, item, value) in python is basically just obj[item] = value in js.

__getattr__ and friends allow to customize the getters and setters.

JavaScript is in the process of getting similar capabilities with Harmony proxies, but the spec has yet to be finalized: http://wiki.ecmascript.org/doku.php?id=harmony:direct_proxie...

I'm slightly confused by your wording but I think the functionality you're referring to is already available via Object.defineProperty's "get" and "set" options: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Object.defineProperty allows to overload get/set for a single key at a time.

Harmony proxies catch everything, and really overload the getter/setter mechanism.

No, JavaScript does not have Python's __getattr__, and obj[name] is not a substitute for it.

obj[name] works if you have already either assigned a value for that name, it exists in the prototype chain, or you have defined a getter via Object.defineProperty(). It's truly no different than obj.name[1] - `name` has to already exist in some form.

Python's __getattr__ [2] takes the attribute name as an argument, so it can be used to implement property access for any name, even those not known ahead of time.

Ruby has something similar with method_missing, and even more with define_method and instance_eval. Dart has noSuchMethod.

These are possible to implement in compile-to-JS language, but tricky to make fast. Dart's static analysis looks for all names that could be called on objects with a noSuchMethod, tries to figure out the most specific class it can add them to, then adds stubs to the JS prototypes to redirect to noSuchMethod(). A compile-to-JS Ruby could implement define_method easily enough, but method_missing is would be impossible to do the way Dart does it due to the open classes: you could dynamically add method_missing and not have the stubs available. If you want full Ruby semantics you have to abstract away method calls, which will be slow. I'm guessing the same problem exists for Python, so I wonder the same thing as the grandparent post: did they implement __getattr__ fully?

[1] With the small difference in the names that are allowed. [2] https://docs.python.org/3/reference/datamodel.html#object.__...

PythonJS implements __getattr__. See these regression tests: https://github.com/PythonJS/PythonJS/blob/master/regtests/cl...


__getattr__ and __getattribute__ are implemented in the special __get__ function defined in: https://github.com/PythonJS/PythonJS/blob/master/pythonjs/ru...

My JS knowledge isn't great. In python, I can do operator overloading such that `obj[item] = value` does something completely different than assignment. Is that possible with JavaScript?

You can set a custom getters and setters, but I wouldn't call it operator overloading. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...


    > var x = {}
    > Object.defineProperty(x, 'value', { get: function() { return 1 }, set: function() { console.log('set') } })
    > x.value
    > x.value = 2
    > x.value

I think he meant that you can overload operators in python and not javascript... but yeah I agree with you. At the end of the day javascript is still just as, or more, dynamic than python.

pythonjs.py is not the main part of the translator. PythonJS translates in two steps, most of the translation happens in the first stage, in python_to_pythonjs.py


I also wrote an interpreter, and it's 500x faster than CPython or PyPy.

(As it happens, my interpreter can only run a single fixed program, but I'm not worried: most people won't read past the headline when I announce it.)

>I also wrote an interpreter, and it's 500x faster than CPython or PyPy

That's great!

Even if your new interpreter doesn't run all the code that's out there it can serve as a base for further work, it's a nice proof of concept and it's just plain cool hacking. It certainly beats all those people merely posting snarky comments on project announcements.

>(As it happens, my interpreter can only run a single fixed program, but I'm not worried: most people won't read past the headline when I announce it.)

With sarcasm off now, yeah?

They worked on something and they are enthusiastic about the results they get. Perhaps they want to oversell them, in a headline. So what? If people "won't read past the headline" then they also won't be using this PythonJS. So no harm done to them.

At worst, you'll be lured to read an interesting article. And if you have any understanding of compilers/interpreters at all, you'll know before the end of it who real-world-usable it would be or not.

This kind of thinking generally hurts more than it does good. Large amounts of people will register it in their memory as CPython bad, PythonJS, or really, anything else good.

Don't know why, don't know why it's FUD, etc. Next time when they get a choice, they'll bash CPython. Not figuring why everyone uses CPython probably.

And you're encouraging that. Probably just thinking it puts you on a higher pedestal or something of the sort. I think its pretty shitty.

Well, CPython is embarrassingly slow.

The damaging thing is claiming things like PythonJS are implementations of Python, when they merely share syntax and some semantics. They might be fast, but that's not very relevant.

Runtimes like PyPy is where we should be looking, since they implement the full Python language and are fast.

>Well, CPython is embarrassingly slow.

Compared to what? Please be specific in your definition of "slow."

>Runtimes like PyPy is where we should be looking, since they implement the full Python language and are fast.

Why should 'we' be looking to PyPy? Have you run a non-trivial application with pypy (v. standard cpy) and found it to be significantly faster?

Even if your code would benefit from JIT, and your code is compatible for running on PyPy, does it benefit more from a compiler switch than CPython + Pandas?

CPython is very slow at various things that are common, like function calls and attribute lookup. There is no good reason for this and few runtimes are quite this slow. This sort of overhead adds up and is often annoying to deal with.

Yes, I have run non-trivial applications on PyPy and they were significantly faster. Its only downside is libraries that depend on CPython extensions, but there are few of those left.

Pandas doesn't do much for removing function call overhead. I don't necessarily have large mathematical datasets, I just want my web service to be easier to write, by not having to care about what CPython is terribly slow at.

Your first point is particularly interesting to me. It was my understanding that Javascript was impossibly slow at a few very key operations, like array lookups. (Since it didn't have true numerically backed arrays and instead did hashtable lookups for those types of arrays.) C should be blowing away Javascript in pure array lookups. I'm astonished it's not. What do you suspect is the cause of that?

Perhaps you misread.

C most likely is, or at least more reliably. JS engines specialise to proper arrays most of the time, though. They're quite remarkably fast, much like PyPy.

CPython, the original Python interpreter written in C, is not particularly fast at anything.

Interesting. Did you profile your code to see where your hot spots were? Whenever I've run web-services on PyPy and got a nice little speed bump, I've found that I could have also made small changes to optimize and get near the same result.

Yes, the slow bits are often part of complex validation, directly proportional to the number of function calls. Sometimes in arbitrary computation, but that is not so common.

I could've manually inlined lots of functions to avoid the overhead on CPython, but I'd have a horrible code base. On PyPy I can simply not care about this aspect, just like in most other languages.

Did you have a point?

The other interesting and recent "Python in the browser" project is RapydScript - https://github.com/atsepkov/RapydScript

Unlike PythonJS, it does not have the goal of being 100% Python compatible, but instead takes the CoffeeScript approach. So, like Coffeescript, you get (mostly) clean Javascript output, no huge library dependency, runs nearly as fast as Javascript, etc.

Check out the "ants" demo - http://salvatore.pythonanywhere.com/RapydBox/default/editor

I've been playing around with both PythonJS and Rapydscript, and I like both. Different goals. PythonJS can output to Lua, too - so there's the whole Python in LuaJIT angle which could be very interesting as it matures.

Still, it's written in JavaScript. Not aiming full compatibility is not only humility. They can't do it.

According to the footnote he tested with pypy 1.9. There was just a post here a day or two ago that pypy 2.3.1 was release. Seems like one should at least use the latest version when testing. Pypy 1.9 was release 2 years ago.

Yes, but am quite happy and impressed to see how well Pypy is holding up.

Something that I have learned about reading benchmarks is to pay attention to who is coming consistently second. Usually that is the most informative part of a benchmark.

This just translates Python syntax into JavaScript, assuming a direct translation is possible.

Neat, but a more compelling post would compare with CoffeeScript and check build times, high-level syntax, and features like generators, list comprehensions, itertools, functools, and so on.

Edit: why the downvote? Am I incorrect about something? If so, I'm genuinely curious to know what I'm overlooking here.

Edit 2: I went over to the github page and it is in fact a 'transpiler', though it can run self-hosted in node, and the features list includes both list comprehensions and yield (i.e. generator functions). It looks like the itertools and functools libraries aren't supported yet, but doing so really wouldn't be too difficult. CoffeeScript is a direct output option too.

Don't worry about the downvotes. Things have gotten pretty bad around here lately, especially after the whole "downvote if you disagree" policy was encouraged.

A lot of totally correct and perfectly valid commentary ends up getting downvoted these days, often with no attempt made to explain why. It's particularly likely to happen when members of the Mozilla, JavaScript or Rust communities are involved. They're particularly sensitive to anything that might contradict their beliefs, and apparently they're particularly eager to downvote, for whatever reason.

These days, many of us have started to read all of the downvoted comments. Aside from occasional spam, they're often among the most insightful comments in any given thread.

> often with no attempt made to explain why. It's particularly likely to happen when members of the Mozilla, JavaScript or Rust communities are involved

Your repetitive comments about Rust's instability almost always receive a lot of feedback about why they are incorrect and/or invalid.

The Rust mis-votings are quite absurd.

In no way can Rust be considered "stable". The most recent version is 0.10 (that is, it's still very pre-1.0), it's undergoing significant breaking changes on an ongoing basis, and even its home page warns that "Rust is a work-in-progress and may do anything it likes up to and including eating your laundry."

Anyone who points out that Rust is just not stable yet is absolutely correct. Anyone who points out that it's looking unlikely that it will be stable before the promised end-of-2014 deadline for 1.0 is also correct.

It doesn't matter who points out these facts, or how many times they do so. The facts are still facts, and whoever expresses them is totally correct.

My suspicion is that the inappropriate downvoting is just immaturity on the part of the Rust developers. I realize that it can hurt to face valid criticism of one's work, but an important part of being a professional adult is to consider such criticism rationally, and to learn how to improve based upon it. Engaging in censorship of facts, however, completely contradicts with self-improvement.

It renders the voting system useless when it's incorrectly used to censor others, due to a lack of emotional control on the part of the person misusing the system.

It's one thing to downvote blatant commercial spam that has no inherent value. It's harmful to downvote perfectly legitimate commentary that you may just happen to disagree with, however.

It has been demonstrated time and time again why your comments are either incorrect, or a mischaracterisation of the situation.

It's clear that Rust isn't stable yet, and won't be for a long time. It's impossible for me or anyone else to be incorrect when we point out that very real fact.

If you feel the need to vote down perfectly legitimate comments merely expressing a truth that you dislike, so be it. I can't stop you, and the others you've chosen to target can't stop you, either. However, reality will be reality, regardless of the voting. Rust will still remain unstable no matter how many down arrows you click.

The problem is not saying that Rust is currently unstable, it is you taking the leap to saying that Rust will never be stable.

Well, the trend so far has been a serious lack of language and library stability, and there's very little to suggest that the situation will change any time soon.

Stability isn't something that happens overnight. It takes a lot of discipline, and this discipline has to be ingrained within the very project itself. I don't see this as the case with Rust.

If you were following Rust at all you would know that they are aggressively rejecting requests for new features in order to produce a stable release, and have five full-time developers working on clearing release blockers.

Python has bignums by default. I was curious how PythonJS implemented bignums, so I tried it, and the answer is: it doesn't. It just uses ordinary JavaScript doubles.

This is a critical difference that ought to be called out, especially in the "add" micro-benchmark, which is measuring two very different things in the two implementations.

PythonJS has support for 64bit integers using the Long.js library. You just need to type your variable as "long". Numbers larger than 64bits could also be supported with another library and simple modifications to the translator.


If this project gains more traction, maybe it will get more people interested in Dart. (which is essentially javascript with a standard library reimagined for the things people are using javascript for today: async APIs, stream processing, mapreduce, webapps, etc.)

Nothing wrong with Dart (other than lack luster adoption) but I'm not sure a surge in interest of people using it as a dropin backend for compile-to-javasvript targets is really going to drive the sort of developer adoption you'd really be looking for.

Near as I can tell, because CPythong is so tied to native code at every turn, none of the other VMs have any real chance of success without bridging to them.

Nice to see I'm not the only one who has a habit of accidentally adding a "g" on the end of "python". Still no idea why that happens though...

Also a big Ruby problem.

cffi has been getting more popular lately, though, and it works on both CPython and PyPy.

Hopefully this will slowly fix itself.

Ah yes, the old X is faster than Y pointless nonsense. Let's ask the basic questions that always come to mind:

- Does X do all or at least most that Y does?

- Are the benchmarks sane? I.e. not the pointless arithmetic fibonacci that nobody really uses, but the useful stuff such as regexps etc.

Python in the browser would be pretty fun. Congrats on the speed improvements.

PyPy uses JIT to generate optimized code during runtime. As many comments noted, PythonJS may not support all Python features but still don't understand why PyPy performing worse than PythonJS.

You know JIT? I mean does it scale to run the JIT in the browser? I will say maybe. All people that do compiler stuff I asked the question think that it's not.

Because PythonJS is not an implementation of Python, it merely looks like Python. It's easy to make a significantly simpler language faster.

The comparisons to PyPy would be much more meaningful if they weren't based on a version of PyPy that's over two years old: PyPy 1.9 was released in 2012.

I've posted updated benchmarks with the latest PyPy 2.3.1 http://pythonjs.blogspot.com/2014/06/pythonjs-faster-than-cp...

You mean 1.9 is not the last stable PyPy?

I'd be interested to see how it stacks up against other python-to-js converters in terms of both speed and supported python features.

It'd be interesting to see a comparison with CPython with the 'numpy' backend using a vectorised code.

Here is a new benchmark testing PythonJS using SIMD vectorized code in the DartVM vs CPython with NumPy. http://pythonjs.blogspot.com/2014/06/pythonjs-simd-vectors.h...

Nitpick: vertical text is really hard to read, sp if the image is compressed like that. Next time please use some bars instead of columns!

Asides from that, this looks quite promising!

PythonJS might win importance if there was a framework to build webapps with it. Otherwise it's only a curiosity...

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact