
Optimizing Ruby lazy initialization in TruffleRuby with deoptimization - kipply
https://engineering.shopify.com/blogs/engineering/optimizing-ruby-lazy-initialization-in-truffleruby-with-deoptimization
======
pjmlp
Nice to see some TruffleRuby love.

Ruby could have been Swift, if MacRuby had not been killed (well at least it
lives on as RubyMotion).

~~~
RussianCow
Ruby is inherently much too slow to have ever been a Swift. All of its dynamic
runtime features give it a pretty low performance ceiling compared to a
statically typed, compiled language with very limited runtime
introspection/reflection.

~~~
speedgoose
While it's still very slow, modern Javascript VMs are fast enough for many
usages. Ruby is slow, but it could be a lot faster with a lot of time and
money.

~~~
bnt
But developing in JS is consideray slower than in Ruby, and time to market
should be king.

~~~
tln
On a mobile device, efficiency should be king... that's not exactly Ruby's
forte

I'm skeptical of your first point as well

~~~
bnt
How does backend development reflect on mobile?

~~~
tln
"Ruby could have been Swift"

Swift's raison d'etre is mobile development

------
jrochkind1
> Benchmarking isn’t a perfect measure of how effective this change is
> (benchmarking is arguably never perfect, but that’s a different
> conversation), as the results would be too noisy to observe in a large
> project.

So... does that not mean that any micro-improvement would also be too small to
be significant on any large project?

~~~
trimbo
This was my question as well. If the change can't be measured even with a
benchmark, what is the goal?

~~~
kipply
The goal of the blogpost was less about talking about this optimization and
more introducing TruffleRuby and making changes to programming language
implementations at a high level understanding. The goal I had when I started
working on the feature, was to introduce myself to TruffleRuby and was
actually my first PR. It's not an impressive change, but it helps get into a
lot of the important aspects of how TruffleRuby is engineered.

------
ksec
Hopefully with Shopify's help we are closer to running Rails on Truffle.

------
The_rationalist
I wonder if ruby could eventually officially migrate to Graalvm

~~~
riffraff
that is unlikely, matz mentioned a few times how adopting something developed
elsewhere as a core vm was something he didn't want to do.

IIRc the reasoning goes along the lines of not wanting the risk of something
being discontinued by a third party and having to shoulder the extra labor of
supporting something with which the devs are not familiar.

------
ericb
I remain skeptical of JRuby. As of about 2-3 years ago, there were many
benchmarks touting it's performance, but in our large publicly deployed web
app, it always underperformed and switching to MRI on Phusion more than
doubled our performance on half the servers. We also spent like a developer-
year on issues that accompanied being off the beaten path with JRuby, like
various nokogiri complications.

I even contacted the devs at one point with a benchmark that ran jruby
directly on strings for several minutes. The answer I got was always that it
would eventually perform better than MRI with more warmup. Spoiler: it didn't.
Well, maybe I was supposed to run it for another month and it would have.

~~~
MoOmer
This article is about TruffleRuby, not jruby.

~~~
kipply
Sorry to be a bit overbearing here~

It's worth thinking about JRuby-performance-skeptical-ness when thinking about
TruffleRuby due to some similarities. Not going to outline in a comment, but
suffice it to say that TruffleRuby was once called JRuby+Truffle.

TruffleRuby has the power to work through JRuby shortcomings. With GraalVM,
TruffleRuby gets to have C extensions! Graal is also open source, so we get
more control of what we want it to do and more understanding of what it does.
In theory, that gives TruffleRuby more room to get faster.

It's also worth noting Charles Nutter's comment on the post, that mentions
that Hotspot C2 already does this optimization! (though they wouldn't have
this kind of control over branch profiling)

~~~
sdegutis
IIRC TruffleRuby is implemented very differently than JRuby and has a lot more
potential. So I'm not sure any comparisons can really be made.

~~~
ericb
source:
[https://chrisseaton.com/truffleruby/](https://chrisseaton.com/truffleruby/)

> TruffleRuby started as my internship project at Oracle Labs in early 2013.
> It is an implementation of the Ruby programming language on the JVM, using
> the Graal dynamic compiler and the Truffle AST interpreter framework.
> TruffleRuby can achieve peak performance well beyond that possible in JRuby
> at the same time as being a significantly simpler system. In early 2014 it
> was open sourced and integrated into JRuby for incubation, then in 2017 it
> became its own project, and now it is part of GraalVM. Since 2019 Shopify
> has sponsored development.

> In early 2014 it was open sourced and integrated into JRuby for incubation

Additionally, we also tried Truffle when it was JRuby, or in JRuby, however
you want to put it, but without a lot of success. Things could be different
now.

~~~
sdegutis
I interpreted that sentence as meaning it was integrated into the JRuby
project as a "home" for the project, similar to being under the management of
a particular github org. Not that its implementation was in any way changed or
integrated into JRuby.

~~~
ericb
Your interpretation is understandable, but incorrect.

[https://www.jruby.org/2016/05/03/jruby-9-1-0-0.html](https://www.jruby.org/2016/05/03/jruby-9-1-0-0.html)

~~~
eregon
Concretely, TruffleRuby never shared much in terms of the core language
implementation with JRuby (essentially the parser and encoding stuff, never
core methods/operators/etc).

------
kipply
(btw I am the author of this post, thread questions if wanted. Trying to avoid
draining myself addressing all the comments. Take care in these strange times
<3)

~~~
joelbluminator
So what's Shopify's goal in supporting TruffleRuby ? isn't Ruby 3x3 promise
good enough? Is Shopify looking to optimise memory footprint or speed or both?

~~~
kipply
As a general comment about production applications, memory is less important
than speed since computers are cheap compared to having poorer user
experience. Also, with JITs, consuming less memory isn't that realistic. You
can find "complaints" about Pypy and Java and consuming a lot of memory.

~~~
joelbluminator
> All in all, the goal is most accurately described as "research"

So what is a successful outcome you're hoping for with this project? Or is it
purely academic?

~~~
kipply
Not really sure there is a good answer for this. A past successful outcome was
that we could actually get TruffleRuby to run an entire production application
without code-changes. I suppose a future one could be that it's easy, fast and
good to develop and ship an application running on TruffleRuby.

~~~
saagarjha
How close to that goal is TruffleRuby right now?

~~~
kipply
oof that's the big question isn't it. Would take a lot to describe, as well as
a press-check since all my information / context about this is specific to
Shopify applications

~~~
saagarjha
Fair enough :)

------
rvz
>'TruffleRuby has high potential in speed, as it is nine times faster than
CRuby on optcarrot, a NES emulator benchmark developed by the Ruby Core Team.'

Really? But now the latest results are in apparently and judging by these
benchmarks[0], I don't know what to make of this claim given how slow
TruffleRuby is when compared to other similar languages or even Ruby. But its
clear that its LLVM-based cousin Crystal still blows it out of the ocean in
every benchmark if you're really talking about 'high performance'...

[0]
[https://github.com/kostya/benchmarks](https://github.com/kostya/benchmarks)

~~~
kipply
There are four benchmarks in there that compare CRuby and TruffleRuby and two
of them are slower. TruffleRuby is 2x faster on bench.b (brainfuck
implementation), 3x faster on matmul, 3x slower on base64 and 7x slower on
json. All four of these are pretty specific benchmarks. Optcarrot has code
that is much more representative of what a Ruby program might look like.

~~~
tom_mellior
In addition, it's interesting to see that TruffleRuby does better than CRuby
on the longer running benchmarks, and worse on the shorter ones. The page
claims that JIT warmup "is applied when necessary", but there are no further
details. If I had to bet, I would guess that in the case of TruffleRuby maybe
not enough warmup is being done in these experiments.

~~~
eregon
Seems there isn't any warmup accounted for:
[https://github.com/kostya/benchmarks/issues/246](https://github.com/kostya/benchmarks/issues/246)

