
Octane benchmark retired - natorion
https://v8project.blogspot.com/2017/04/retiring-octane.html
======
franciscop
This actually makes sense. When the _low level_ stuff is at a terrible level
they made Octane to make that better. Once that was fixed, optimizing for it
is counter-productive.

This has a big resemblance IMO to most formal study methods including
University and natural languages. Once you know the basics you can optimize
for learning or for grades, depending on your goals. They _are_ correlated
(but not strongly correlated), but the time investment is not negligible so
you'll have to choose either of them.

------
skrowl
They probably got tired of FF & Edge beating them in their own benchmark. Can
you imagine the conversation with some middle level manager when you were
asked to explain that?

~~~
pizlonator
WebKit is currently king on Octane fwiw.

We still think it's a great benchmark. V8 made the mistake of focusing _only_
on this benchmark, so they didn't know that some of their Octane optimizations
were bad for other scenarios. We didn't have that problem because we have
always used a broader spectrum of benchmarks.

~~~
bterlson
Agreed, Octane was good as far as benchmarks go but you have to balance with a
much larger set of benchmarks and always ground your work in real-world
scenarios (something we focused heavily on from day one e.g. when we realized
having an interpreter really helps startup on lots of real-world apps but
doesn't do much for Octane score).

That said, FWIW I think Edge/Chakra gets the crown on Windows!

(I work on Chakra)

~~~
vanderZwan
Out of curiosity: does only having to target one OS make a difference here? Is
it "easier" to get it to run fast? Any worries that you _don 't_ have compared
to Firefox and Chrome, even though you purely work on the JS side of things?

~~~
bterlson
I'm sorry I missed your comment earlier, but hopefully replying later is
better than never :)

ChakraCore is cross-platform now so we no longer get the benefit of a single
platform. However back when we were only Windows there were advantages,
certainly, if only because all the work spent getting everything working on
different OSes can instead be invested in improving performance.

------
bozonil
First sunspider, now octane.

This shows that browser vendors are not really good at writing JS benchmarks
representative of real world, which is not surprising considering that they
are not doing real web development.

Ideally benchmarks would come from developers of popular websites like
Facebook, Twitter, etc.

~~~
ajross
That's just a truism about benchmarks. Once all the low hanging fruit is
picked they stop showing interesting progress and just become a garden for
crazy microoptimizations. The same thing happened with C compilers 10-15 years
ago.

Basically, Javascript performance is "done". It's not going to get much faster
at this point, and higher performance will come from paradigm shifts (SIMD,
WebAssembly, pick your fad).

------
dallamaneni
Looks like someone at Google realized that Servo (Quantum) is coming

~~~
Etzos
Servo uses SpiderMonkey to provide Javascript support, so retiring Octane (a
Javascript engine benchmark) would have nothing to do with Servo (a layout
engine).

~~~
dallamaneni
Got it. I did'nt realize that.

