
Announcing Topaz: A New Ruby - jnoller
http://docs.topazruby.com/en/latest/blog/announcing-topaz/
======
Pewpewarrows
Completely unscientific, but if these outputs are any indication, this is
going to be great news for Rubby users in the future...

    
    
      $ time ruby -e "puts 'hello world'"                                                                                                                           
      hello world
    
      real    0m0.184s
      user    0m0.079s
      sys     0m0.092s
    
      $ time ~/Downloads/topaz/bin/topaz -e "puts 'hello world'"                                                                                                    
      hello world
    
      real    0m0.007s
      user    0m0.002s
      sys     0m0.004s

~~~
grn
In order to eliminate the potential bias of startup time I run similar test in
many iterations. Here's what I got:

    
    
        $ time ruby -e "10000.times { puts 'hello world' }" > /dev/null
        real    0m0.102s
        user    0m0.096s
        sys     0m0.005s
    

and

    
    
        $ time ./topaz -e "10000.times { puts 'hello world' }" > /dev/null
        real    0m0.098s
        user    0m0.071s
        sys     0m0.026s
    

Any idea why I don't see such big difference?

~~~
fernandezpablo
what is your 'ruby'?

~~~
onedognight
Good question. How many shells is rbenv/rvm executing? Do you get similar
results with an absolute path?

------
vidarh
I'm all for new Ruby implementations, but be very careful about judging
performance of an incomplete implementation.

It's "trivial" to make a fast language that looks quite a bit like Ruby. It is
a lot harder that make a language that remains fast in the face of handling
all the quirks of the full Ruby semantics, though, such as selectively
handling the risk of someone going bananas with monkey-patching core classes
that could happen at any "eval()" point.

~~~
kingkilr
What you just wrote is exactly what I've been telling people about every new
"Python" implementation for the last 3 years. Believe me, I understand this
argument completely, that's why I made sure we had all the hard bits (monkey
patching core classes, eval, etc.) before I released this.

~~~
surrealize
callcc?

~~~
kingkilr
no call/cc yet, I do have a branch with fibers, but I'm waiting for an RPython
branch to land for that first.

------
draegtun
For those with long memories there was a (in)famous _Topaz_ project in the
Perl world. This was an attempt to implement perl5 in c++. It was abandoned
however it did help kick-start Perl6 & Parrot.

refs:

\- <http://topaz.sourceforge.net>

\- _Topaz: Perl for the 22nd Century_
<http://www.perl.com/pub/1999/09/topaz.html>

\- _Historical Implementations/Perl6_
[http://www.perlfoundation.org/perl6/index.cgi?historical_imp...](http://www.perlfoundation.org/perl6/index.cgi?historical_implementations)

------
lost-theory
I will be the first to ask: why did Alex / the pypy devs start this project? I
thought the three community funded ideas were great (STM, Python 3, and
Numpy). What prompted this new project? Is it meant to attract new developers
to pypy as platform for implementing languages? Is it a side project? Is it a
long term project meant to become the premium implementation of ruby as pypy
is becoming for python? Etc.

Not criticizing at all by the way, just curious about the motivation /
background / context for the project, which is missing from the docs.

~~~
kingkilr
Hi to be clear, this is project undetaken by me (Alex) independently, it
wasn't funded out of the PyPy funds or anything like that. There were 3
primary motivations:

a) Because it's fun

b) To prove RPython is a great platform

c) To mess with people's heads, it's crazy!

~~~
lloeki
Hey Alex, let me tell you I completely understand points a) b) c) and more. It
seems we started our competing Ruby on PyPy implementations at about the same
time [0]! My goal was to build a robust and documented Ruby parser, and hope
PyPy make it magically fast. To that end I embedded PLY and modified it to not
require kwargs. Then a thunder of work fell from the skies, and I was just
recently unearthing PyPy 1.9 to put the thing back on track.

Congratulations for the release :-)

[0]:

    
    
        $ ls -l playground/pypy
        total 29160
        drwxr-xr-x  12 lloeki  staff       408 18 Feb  2011 ply-3.4
        drwxr-xr-x  15 lloeki  staff       510  3 Apr  2012 pyby
        drwxr-xr-x  23 lloeki  staff       782 27 Mar  2012 pypy-pypy-2346207d9946
        drwxr-xr-x  18 lloeki  staff       612 27 Mar  2012 pypy-tutorial
        -rw-r--r--   1 lloeki  staff  14927806 27 Mar  2012 release-1.8.tar.bz2
        drwxr-xr-x  10 lloeki  staff       340  3 Apr  2012 unholy

~~~
lloeki
Holy snakes[0]. It seems you pursued the exact same approach[1] as me :-)

[0]:
[https://github.com/topazproject/topaz/blob/master/topaz/lexe...](https://github.com/topazproject/topaz/blob/master/topaz/lexer.py#L6)

[1]: <https://github.com/alex/rply>

------
hemancuso
It would be nice to see Rpython/pypy turned into a project not unlike WebKit
where many different projects benefit from the same open-source core. If pypy
is flexible enough to implement all of ruby and all of Python, has a
reasonable amount of maturity, and is already rather fast it seems like a good
candidate for such a thing.

~~~
lloeki
Quick how-to:

1\. (optional) install PyPy "JIT compiler" binaries [0]

2\. get and extract the PyPy source[1] from bitbucket release packages, which
includes the RPython translator, written itself in Python and, although
working on CPython, best run under PyPy as installed on step 1.

3\. write a (R)Python module including a _def target_ returning the entry
point function[2], and call the translator upon your module, like so:

    
    
        python ./pypy/pypy/translator/goal/translate.py example2.py
    

I wish getting the translator was more 'packaged' and did not involve getting
the whole PyPy source but I hear this is in the works, and it is reasonably
easy already. The hardest part is actually figuring the translator is not part
of PyPy binary release, and that it's available straight from bitbucket.

[0]: <http://pypy.org/download.html>

[1]: <https://bitbucket.org/pypy/pypy/get/release-1.9.tar.bz2>

[2]: [http://morepypy.blogspot.fr/2011/04/tutorial-writing-
interpr...](http://morepypy.blogspot.fr/2011/04/tutorial-writing-interpreter-
with-pypy.html)

~~~
fijal
yeah, working on it, sorry :/

------
nlh
Forgive me for asking what may be a very stupid question: Can someone explain
to me the need for a Ruby interpreter built on top of Python? He mentions
performance in the announcement, but is this really going to be faster than,
say, MRI? Thanks in advance.

~~~
llimllib
Think of Pypy as somewhat like LLVM, except the language it's built on is a
subset of the python language, called RPython.

Although the most well-known language implemented in the pypy "vm" is python,
the toolchain is completely language agnostic.

So, this is more akin to Apple writing a C interpreter on top of llvm than it
is to building a ruby interpreter on top of python.

~~~
fijal
PyPy is totally not like LLVM in many regards. It targets a very different
demographics. Some differences:

* RPython comes with a good garbage collector

* the language where you specify what's going on is RPython in which you write an interpreter. Then you get a JIT using a few hints. This is difference than "interpreter in C + compiler to LLVM" scenario by quite a bit.

* RPython comes with a set of data structures that are higher level (lists, dicts, etc.) and is a GCed language. JIT is also well aware of those.

~~~
fusiongyro
Do you know what an analogy is?

~~~
fijal
I'm actually not a native speaker, so indeed, I might not know. I checked in
the dictionary though. This is an analogy, but not a very good analogy,
because of the reasons that I pointed out. Are you disagreeing with any
particular reasoning there? Or should I just say "they're similary, but"?

~~~
sirclueless
An analogy does not require that the items involved actually be similar, only
that the relationships in each are the same. For example, "Gasoline is to
cars, as sunlight is to trees" is a perfectly good analogy, even though
gasoline is nothing like sunlight, and cars are nothing like trees.

In much the same way, comparing "Apple writing a C interpreter on top of llvm"
to "Alex Gaynor writing a Ruby interpreter on top of RPython" makes a _very
good_ analogy, even though Apple is not much like Alex Gaynor, Ruby is not
much like C, and PyPy is not much like LLVM.

~~~
gojomo
A good explanation, and further, many times a good analogy _relies_ on the
things being very different in every way _except_ for the one way it intends
to emphasize. An analogy is often a way of saying: "Let's focus on this (often
very abstract) aspect of interesting similarities. Yes, you can easily point
out many other dissimilarities. That ease means I'm not talking about those
aspects, so if we understand each other those don't even have to be
mentioned."

------
andrewgodwin
I'm curious to see how this compares to MRI/Rubinius/JRuby once it's feature-
complete; the PyPy vs. CPython benchmarks these days are pretty convincing.

~~~
kingkilr
One of the major launch criterion was having "enough of Ruby that the
performance won't change". What does that mean in practice? We have
exceptions, we have bindings, and we have all manner of other obscure corner
cases. Charles Nutter (JRuby dev) writes about this:
[http://blog.headius.com/2012/10/so-you-want-to-optimize-
ruby...](http://blog.headius.com/2012/10/so-you-want-to-optimize-ruby.html)

------
hencq
Does Topaz benefit at all from the work that's been done already on Rubinius?
I seem to recall that Rubinius implements as much of Ruby in Ruby as possible.
It seems like this would help Topaz as well in implementing the standard
library. Or are the implementations different enough that this doesn't work?

~~~
lobster_johnson
One of the things to come out of the Rubinius project is RubySpec, which is
pretty much a language spec in unit test form, and which be very useful to
test Topaz.

------
semanticart
I'm probably in the minority but I would love to see a dialect of ruby that
has significant whitespace. Death to unnecessary 'end's

~~~
josh-j
> Some observers objected to Go's C-like block structure with braces,
> preferring the use of spaces for indentation, such as used in Python or
> Haskell [braces are optional in haskell]. However, we have had extensive
> experience tracking down build and test failures caused by cross-language
> builds where a Python snippet embedded in another language, for instance
> through a SWIG invocation, is subtly and invisibly broken by a change in the
> indentation of the surrounding code. Our position is therefore that,
> although spaces for indentation is nice for small programs, it doesn't scale
> well, and the bigger and more heterogeneous the code base, the more trouble
> it can cause. It is better to forgo convenience for safety and
> dependability, so Go has brace-bounded blocks.

<http://talks.golang.org/2012/splash.article>

~~~
pacaro
This.

Especially when writing a code generator. Targeting a significant whitespace
language is potentially more tricky than one with explicit delimiters. I love
the Haskell approach to this.

~~~
dbaupp
(For those who don't know, Haskell has an optional whitespace-significant
syntax (which most humans use) as well as a brace-delimited syntax, and these
are exactly equivalent.)

------
blissofbeing
Sweet, I'm very interested in this. I assume this implementation has a GIL?
Any word on how threads will be implemented?

~~~
kingkilr
No GIL. Also no threads yet :) We're going to wait to see how Armin's work on
STM support pans out.

------
Millennium
Implementing other languages on top of the PyPy toolchain is an interesting
concept, and I'd like to see it happen more often. But calling this "Ruby in
Python" is sure to invite flamewars from all sides. The actual Topaz site
seems to be careful not to do this, but if this thread is any indication, a
fair number of people are already mistakenly calling it that.

~~~
draegtun
There are already a few other language _implementations_ on PyPy.

For e.g.

\- Javascript | <https://bitbucket.org/pypy/lang-js>

\- Smalltalk | <https://bitbucket.org/pypy/lang-smalltalk>

\- Schema | <https://bitbucket.org/pypy/lang-scheme>

\- Io | <https://bitbucket.org/pypy/lang-io>

~~~
masklinn
\- Converge |
[http://tratt.net/laurie/tech_articles/articles/fast_enough_v...](http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time)

------
amalag
[http://blog.headius.com/2009/04/how-jruby-makes-ruby-
fast.ht...](http://blog.headius.com/2009/04/how-jruby-makes-ruby-fast.html)

At least once a year there's a maelstrom of posts about a new Ruby
implementation with stellar numbers. These numbers are usually based on very
early experimental code, and they are rarely accompanied by information on
compatibility. And of course we love to see crazy performance numbers, so many
of us eat this stuff up.

~~~
acdha
Given that the original article thanks the author of that post (Charles
Nutter) by name, I think it's likely that Alex is aware of it.

------
nathansobo
If it doesn't have all of Ruby's weird features, claims of speed don't matter,
because it's those weird features that make Ruby slow and hard to optimize.

------
sophacles
Could this lead to a runtime in which I can use really cool ruby modules from
python and really cool python modules from ruby? I mean, seriously, these two
languages both have some pretty awesome stuff, and regularly I say about one,
"man I sure wish I could use $module from the other".

------
jamesbritt
Any reason for not targeting Ruby 2.0?

------
VeejayRampay
Excellent news. I wish this project the best of luck. More competition in the
Ruby implementation field is always a good thing for end users, it fosters
innovation and provides choice and a good kick in the ass to everyone.

------
ggchappell
I've been wondering why the tools the PyPy people have been building are not
used to implement more languages. It's nice to see this happening. And it will
be interesting to watch where this leads.

------
irfn
Doesnt mention what the current status of topaz is with respect to
<http://rubyspec.org/>

------
tonycoco
I mean, it's cool that people want to expose Python's version of Rubinius as a
building block to yet another Ruby interpeter, but, come on. Why is this a
thing?! Why not waste time improving the JVM, contributing to Rubinius, patch
the MRI, or a million other open source projects already tackling Ruby being
"slow". By the way, has anyone else realized that, while Ruby is slow,
_computers_ are getting faster for us too!

------
denysonique
Sounds really interesting, but we already have a dozen of incomplete
alternative Ruby implementations...

------
evolve2k
Just curious, for this project did you set out to write it in idiomatic ruby
or idiomatic (R)Python?

------
steeve
So Ruby on top of RPython on top of PyPy? Why not, but how does it compare to
JRuby and Rubinius?

~~~
masklinn
> So Ruby on top of RPython on top of PyPy?

No. It's Ruby in RPython, where PyPy is Python in RPython. The pypy project is
currently in the process of splitting "RPython" (the VM-development framework)
from PyPy (the Python VM) to make that clearer.

RPython is a general purpose, language-agnostic (ish?) core for implementing
JIT-ed garbage-collected languages, it's kind-of similar to LLVM being a
framework for implementing compilers.

(now because rpython is a proper subset of python you can run your rpython VM
on a python interpreter without translation, but that doesn't give you any
specific python bridge, in the same way coding your VM in C doesn't give you a
c FFI for free)

~~~
kzrdude
But RPython used to be the reduced version of the Python language. What name
do you give to the jit-compiler generator?

~~~
masklinn
The same, as far as I've seen. RPython is both the VM-framework and the Python
subset which exists within (and for) that framework (RPython is not considered
and developed as a general purpose language, but as a language for building
VMs within the RPython system/framework/toolset).

The important distinction really is between RPython (a language and framework
for building virtual machines) and PyPy (a virtual machine implemented in/on
RPython)

------
jrochkind1
So... GIL?

------
DaNmarner
RPython is the new C?

~~~
luckystarr
In addition to fijal's comment: I also heard somewhere that RPython is a) not
strictly specified and b) may change when needed without notice. So I would
stay clear of it. :)

~~~
fijal
we removed that from the docs, it's kinda stable by now (but it's still an
atrocious language to use, just beats C/C++ for that particular task by a lot)

~~~
luckystarr
Out of curiosity, whats so atrocious about it? The slow translation step or
language idiosyncrasies?

~~~
kingkilr
Slow compilation, undocumented semantics, and most importantly: atrocious
error messages..

------
peteretep
Topaz as a name? Seriously? (<http://topaz.sourceforge.net/>)

~~~
veemjeem
I find it odd that you link to a project that hasn't seen an update in the
last 12 years. How old does a project have to be before it is considered
"dead"?

~~~
bryanlarsen
The project is certainly dead, but it's also historically significant.

------
rmoriz
Why not implemented on Erlang OTP? Ruby and Python share the same scaling
issues, so I don't see a benefit it combining them or migrate from one to the
other.

I also dislike the "high performance" claims without showing a single
performance comparison. Not to mention the state of implementation
completeness of the language.

~~~
fijal
I think you're missing the point. It's not written _in Python_. It's written i
RPython, which is vastly different. Technology behind this and PyPy is the
same, however there is no Python usage here.

Additionally, erlang has avoided scaling issues by being functional. This
might be good or bad, depending on your viewpoint, but just implementing a
ruby interpreter in erlang won't cut, because you're not answering any of the
hard questions about the state. You might answer "meh, ruby is just a broken
language", but well, this has nothing to do with that article.

~~~
kristofferR
"Topaz is written in Python on top of the RPython translation toolchain (the
same one that powers PyPy)." [1]

"Topaz - An implementation of the Ruby programming language, in Python, using
the RPython VM toolchain." [2]

[1] <http://docs.topazruby.com/en/latest/blog/announcing-topaz/> [2]
<https://github.com/topazproject/topaz/blob/master/README.rst>

~~~
awj
As RPython is a subset of Python, that statement is technically correct but
misleading. Stating that Topaz is "Ruby written in Python" leads to the idea
that it's written in / runs on _CPython_ , which is really not the case.

~~~
kingkilr
Well, it can be run on CPython if you wanted (that's how we run many of our
tests). It's just crazy slow.

