
Benchmarking JS - luu
http://mrale.ph/talks/goto2015/#/
======
inglor
Can always trust Vyacheslav Egorov to pump out super high quality stuff about
v8.

His blog [http://mrale.ph/](http://mrale.ph/) is full of hidden goodies like
Object.create vs function constructor [1] and hidden classes vs jsperf [2].
Lots of good stuff on his SO too.

It'd be great if he (hi if you're reading this btw!) could write a bit about
TF and the changes being made to v8 internally from an overview point of view.

[1] [http://mrale.ph/blog/2014/07/30/constructor-vs-
objectcreate....](http://mrale.ph/blog/2014/07/30/constructor-vs-
objectcreate.html) [2] [http://mrale.ph/blog/2013/08/14/hidden-classes-vs-
jsperf.htm...](http://mrale.ph/blog/2013/08/14/hidden-classes-vs-jsperf.html)

~~~
inglor
And, since you're reading this (in the other comments) I just want to point
out that the presentations you're giving got a lot better (the deck). This one
is a lot more witty than the old ones. Props on getting better at presenting -
a big step from your old presentations and blog posts, the English is good too
here :)

~~~
gnrlbzik
Agree : ) Deff. awesome experience watching him present. Such talks stick with
you through out.

------
barsonme
Does anybody happen to have the video for the talk? I'd be much obliged. Just
the slides leave a bunch of holes I want to fill in.

I can only find the videos from 2014.

~~~
mraleph
It was recorded but I don't know when it will be publicly released.

This talk is a combination of two separate talks I have given before plus two
new examples, based on my recent endeavors.

Talks merged here are LXJS 2013 one[1] where I talk about microbenchmarking
pitfalls and WebRebels 2014 one[2] where describe how bugs in VMs can affect
benchmarking results.

If you have any questions about the slides just ask them here or send me an
email --- I will try to answer as soon as I can.

[1]
[https://www.youtube.com/watch?v=65-RbBwZQdU](https://www.youtube.com/watch?v=65-RbBwZQdU)

[2] [https://webrebels.23video.com/crooked-mirrors-of-
performance...](https://webrebels.23video.com/crooked-mirrors-of-performance-
by)

~~~
randomfool
Your talks always frighten me when it comes to VM's. I just want predictable
performance- is that too much to ask? (Semi-serious question).

edit: to clarify. In C++ I write dumb but readable code knowing that the
compiler will always perform specific optimizations. In JS it seems that there
are many factors resulting in different optimization levels. How do I ensure
that the VM does not execute my 'dumb but readable' code literally? Without
that guarantee it seems that I have to frequently resort to optimized yet
unreadable code.

~~~
mraleph
I obviously craft/collect examples of V8 going off the rails for these talks
just to show that VMs are software and all software has bugs --- and those
bugs don't necessarily manifest as crashes and incorrect results - they can
lead to worse performance and developers must be ready for this: must be ready
to diagnose these issues, report them and work around them.

For a single test case where V8 goes off the rails there are millions of lines
of code across the globe which V8 optimizes correctly.

On a funny note: I actually do have a version of this talk where I show GCC
going slightly of the rails and producing a code that is 3 times slower than
it should be because it hits an (infamous) partial register dependency stall
--- see StackOverflow question[1] for the gory details.

> How do I ensure that the VM does not execute my 'dumb but readable' code
> literally?

Well, as I do say in the talk: reasonable code should be reasonably fast. If
it is not the case --- file bugs with VM vendors.

Keeping your code relatively static / monomorphic is the best way to achive
performance in any language.

In any case I think it's much much much more important to optimize algorithms
not their concrete implementations.

[1] [http://stackoverflow.com/questions/26585977/64-bit-code-
gene...](http://stackoverflow.com/questions/26585977/64-bit-code-generated-by-
gcc-is-3-times-slower-than-32-bit/26588074#26588074)

------
reimertz
I found this very interesting since I'm working on something similar where I
need to be able to measure how long certain operations take.

And I had no clue that js-engines has come this far, optimizing code during
runtime this efficiently(if you ignore the bugs.. :)).

~~~
DannyBee
It's actually really well written

Some of the optimizations compilers/jits will perform would blow your mind.

LICM is a very simple thing to do.

There are more advanced things, like
[http://www.cs.rice.edu/~keith/512/2011/Lectures/L06CLZ-1up.p...](http://www.cs.rice.edu/~keith/512/2011/Lectures/L06CLZ-1up.pdf)

etc

That's just on the code movement side too.

Most good static compilers can symbolically evaluate even linear systems of
equations/recurrences/etc that are solvable, and transform loops that
calculate them into constants, etc.

~~~
mraleph
> LICM is a very simple thing to do.

Yeah, it sometimes surprises me how much value the most simple optimizations
have and how much they break people's attempts to measure performance.

There is another side to this medal which is best expressed in a quote I
picked up from a relatively old paper:

"A survey of the literature on optimization uncovers many techniques that are
hard to understand, harder to implement, more general than necessary and of
marginal value. Our approach was to do the most profitable machine independent
optimizations in the most profitable way" (A Portable Optimizing Compiler for
Modula-2 by Michael L. Powel)

In some sense the more microbenchmarks an optimization breaks the bigger its
impact on real world code is :)

~~~
DannyBee
While that paper makes a generally true statement, trying to do the most
profitable optimizations in the most profitable way is what dead-ended gcc for
many many years :)

At some point, you have to step back from what you are doing and actually
design and engineer optimizations coherently, not just "do this little thing
and that little thing"

------
gnrlbzik
This was a great talk and funny one at that : ) Great perspective on what to
really be concerned with. There were other great talks at #GOTOChgo but this
one is one of the best ones : )

------
z3t4
You should never "optimize" JavaScript code, like objects vs arrays or methods
vs functions. But instead try to optimize your algorithms, like leaving out
square root when comparing witch line is longer.

I did optimize JavaScript in the past, but it always worked different in
different JS implementations and could change in different releases of the
same JS engine. So I gave up on it, and it actually makes me more productive
not having to think about with method is faster.

~~~
fridek
That is simply not true. To write fast JS you need to understand the basics of
the engine, like what a hidden class is, what exactly garbage collector does,
how to keep your methods monomorphic or how types work in arrays. I was
stunned when recruiting people to see how many just didn't understand why the
prototype chains they build are actually creating a separated method instance
per each object they allocate. You can see that easily when serious people
suggest that an object constructor should return another object with methods
on it. Sure, it's valid JS, it has it's advantages and compiler MAY be able to
optimize it (last time I checked it didn't), but there will be a limit in what
it can do and performance will be impacted.

You should perhaps not optimize trivial things like methods vs function,
because there is no good underlying reason for them to be much different. You
should know when to optimize and what. I love the longer version of Knuth's
quote - "We should forget about small efficiencies, say about 97% of the time:
premature optimization is the root of all evil. Yet we should not pass up our
opportunities in that critical 3%.". The overall architecture, the way you
write and organize your code, in my opinion, and in case of JavaScript is
likely to be in the 3%.

But I also agree not caring too much about it improves productiveness. Most
applications really don't need to worry about performance in such depth. Even
when they do, it often boils down to keeping one or two core components
optimized.

The only worry I always have about it, is that when you notice your code style
and architecture doesn't play well with JS engines, it's often too late to
change it. A friend called me a day before a prod release asking what to do
about his program taking a huge chunk of memory for what the profiler
described as a "compiled code". It was a large corporate application, with
dynamic code loading. We digged into the framework internals just to notice
there was no built-in way of deregistering a class once it's created. Since it
didn't play nicely with the JS engine, there were many more hidden classes
than business models in the system, and most calls were polymorphic. All this
compiled code had to be kept somewhere. At this point there was no way he
would replace the framework and even quickly fixing it was a challenge.

~~~
amelius
> To write fast JS you need to understand the basics of the engine

The problem is that there are many different engines out there.

Unless of course you are talking about server-side code. But for the server-
side, there are more efficient solutions than just javascript.

~~~
fridek
Sure they are, but they employ mostly the same techniques. The implementation
may differ, but the ideas remain the same, at least to the extent where well
organized code optimized for one engine is unlikely to be slow in another.

------
imaginenore
You should optimize your images. These are 6000x4500px 2.4MB PNG images:

[http://mrale.ph/talks/goto2015/images/intro-
slide.png](http://mrale.ph/talks/goto2015/images/intro-slide.png)

[http://mrale.ph/talks/goto2015/images/last-
slide.png](http://mrale.ph/talks/goto2015/images/last-slide.png)

~~~
mraleph
Oops, sorry for that. Forgot to shrink them before publishing. Will fix as
soon as I get to a place with a stable internet connection - traveling right
now.

upd. done, scaled down to 1024x768

------
xenofiend
How is the acronym LICM pronounced?

~~~
mraleph
I always pronounce it as /līkm/, but I am not a native speaker so I can't be
sure about correct pronunciation.

