
MRI Developers Don't Use RubySpec and It's Hurting Ruby - jc00ke
http://rubini.us/2014/12/31/matz-s-ruby-developers-don-t-use-rubyspec/
======
nateberkopec
Rubyspec is a great project for all the reasons Brian outlines.

However, it's also a failure - in part, due to Brian and his attitude towards
contributors. See this twitter conversation:
[https://twitter.com/the_zenspider/status/547527644535726080](https://twitter.com/the_zenspider/status/547527644535726080)
He's been grinding on Rubyspec for years, bless him, but I think there's a
reason why he was unable to rally the community behind his effort, both in
terms of gathering more contributors and in making it "official" in terms of
the language spec.

JRuby, currently the only non-MRI ruby implementation that you can seriously
consider for production use, runs the MRI test suite against JRuby. RubySpec
is far from The Only Solution, though Brian would like you to think it is.

~~~
rmoriz
>However, it's also a failure - in part, due to Brian and his attitude towards
contributors.

Looks like he seems to be burned out - maybe he just wants something in return
for his time investment e.g. a paid job or some other gratification in return
for his work.

Btdt - you put _a lot_ of work into something, people/companies start benefit
from it but they don't return anything important (monetary compensation is not
a bad thing -it allows one to pay the bills/go on holiday/get a life).

We've seen this "pattern" a lot in the OSS world, but saying a different
attitude would "fix everything" is a lie. We (as in the Ruby community) need
to crowd-fund independent developers who are not paid by companies for their
OSS contribution IMHO.

~~~
headius
Brian has been paid to work on Rubinius and RubySpec full time for at least 6
years.

~~~
rmoriz
oh - didn't know that. Okay that makes a hughe difference IMHO.

------
vorg
It's disappointing to hear about such problems with Ruby. I hit the same
problem with the Groovy Language spec when I first came across Groovy. Its
creator, James Strachan, initiated an implementation, test kit, and spec all
within 6 months of each other (impl beta-1 in Dec 2003, and spec JSR-241 in
May 2004). The project managers who took over from him, Graeme Rocher and
Guillaume Laforge, changed direction by stopping work on the spec and
refocusing the Groovy reference implementation to be the scripting language
behind Grails. (Of course, _Groovy 'n' Grails_ was intended to chisel away at
some of the market share of _Ruby on Rails_ but that's another story.)
Strachan often wrote that the spec was to enable anyone to make their own
implementation of Groovy if they want to, and right up to his very last
posting ever [http://groovy.329449.n5.nabble.com/Paris-write-up-
tt395560.h...](http://groovy.329449.n5.nabble.com/Paris-write-up-
tt395560.html#a395571) on the Groovy mailing list on 5 Dec 2005, he maintained
that what they were building was the reference implementation.

If Rocher and Laforge had come clean about how they turned the RI into the
language itself, the backlash might have blown over quickly, but instead they
led developers along for many years afterwards, not changing the spec to
dormant until April 2012. Projects other than Grails who've tried to build
atop Groovy have had to risk the ref impl changing in breaking ways between
versions. The most spectacular incident was when Groovy++, an experimental
static compiler built by Alex Tkachman that hooked via annotations into
Groovy's AST, had to drop back down from Groovy 1.8 to 1.7 in 2011, and my own
side project was also affected by the change. It turned out Rocher and Laforge
had secretly employed a mate to extend Groovy with the exact same static type-
checking and compilation functionality as Groovy++ and were obviously trying
to shake us off.

Unlike Ruby, Groovy only has one other implementation, GrooScript, built by
Jorge Franco, which generates JavaScript from Groovy syntax. When the
developers of the most used implementation of a language want to protect their
control, it certainly does hurt the ecosystem, turning it into an "echo
system".

~~~
marktangotango
Are you sure you're not assigning to malice that which could be due to
ignorance or indifference?

~~~
vorg
When you see the same behavior repeated consistently over 10 years, it doesn't
matter if it's done consciously (what you call "malice") or as an unconscious
habit ("indifference"), the effect is what matters and the intent is the same.
The Groovy 2.4 release candidate was released a few days ago and announced on
the personal blog of the project manager, but not on the Groovy community
mailing list, a consistent behavior which started about year ago. Also a year
ago, he launched a weekly mailout newsletter he controls about Groovy separate
from the Codehaus repository or the backing company Pivotal Inc, soliciting
for subscribers. It all looks like a clear attempt to take over from the other
developers involved in building the Groovy codebase. I listed plenty of
previous examples of such behavior by these project managers going back 10
years that's harmed the ecosystem, probably irrepairably, and I hope Ruby
doesn't go down the same path.

------
lgleason
Both Charles Nutter and Matz are smart guys. They are also both great people.
I say this having spent time with both of them. This whole, my implementation
is better than theirs coming from Brian is crazy.

Matz and Charles have helped to set the tone for the community. While I
appreciate Brian's passion, it sounds like he needs to check his ego. No
matter how smart we are, we can always learn things from other people. Without
the collaboration of others neither JRuby or Ruby would be what they are today
which is why they are successful.

~~~
vidarh
He might need to check his ego, but the state of Ruby specification is
deplorable, and despite being a big fan of Ruby, it's also a big road block to
development.

The small number of mature Ruby runtimes is a bad sign.

~~~
whistlerbrk
See, iirc, Matz years ago specifically said "fork Ruby". He wants lots of
different versions which I feel to some extent means diverging from a formal
spec, and then occasionally maybe coming back to it.

~~~
vidarh
And that's even worse. I'm working on a Ruby compiler. While it is far from
being at a stage where this is a real problem, currently there's no sane way
of knowing whether or not I'm "close enough" to MRI other than extensive
testing of every single piece of Ruby code I want to work.

"Diverging from a formal spec" is one thing. As it is, many versions of MRI
has diverged from how the core team _thinks_ it works. If MRI is "the spec",
then Ruby changes from revision to revision as the test coverage is nowhere
good enough to prevent regressions or changes in behaviour.

Basically: Nobody knows Ruby.

Frankly, I very much hope that another Ruby implementations overtake MRI
sufficiently in quality to out compete it enough to shift the initiative of
definining the language to a more responsible team.

------
sams99
The thing I find depressing and frustrating about this kind of discussion is
the fatalism and non-constructiveness.

I really wish it was:

"I set up a server that runs ruby spec on ruby-head daily and automatically
reports spec failures to ruby-bugs"

So many companies are making big bucks off Ruby, yet so little are willing to
fork out a bit of money and time to make Ruby better.

~~~
zem
it's unreasonable to expect the dev who has already put a ton of work into
developing and maintaining rubyspec and making it work nicely with all the
ruby implementations and versions he could to have to put in the additional
work to do this as well. I don't blame him for getting discouraged at having
to chase a moving target on top of all that - I agree with him that having the
MRI devs contribute to rubyspec was a reasonable thing to expect.

~~~
sams99
Throwing away a project that is clearly working is depressing. can't
Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a
system that runs the suite on latest?

It's just such a better outcome, nobody needs to change workflows its just
that information gets reported upstream earlier in a much more useful time.

~~~
mahmoudimus
The Ruby devs attitudes in the discussions linked FTA makes me question the
longevity of Ruby as a serious Enterprise language.

~~~
manyxcxi
When did it become a serious Enterprise™ language? Not being flippant or
denigrating Ruby, but when I do work for established enterprises, it's never
in Ruby. Java, .NET, PHP, and occasionally Python. Just because startups use
it to get off the ground quickly, or to build the front end of their website
doesn't make it enterprise.

~~~
fat0wl
yes I don't want to get into a flamewar either, but I am an ex-RoR dev and
since switching to enterprise Java I hear almost nothing about it largely due
to enormously widespread acceptance of Spring (JavaEE is even re-gaining some
traction).

Long story short (and hope my facts are right! not swearing by this, was just
some googling...), I had to optimize some old code for an RoR client I
freelance for and wanted to add in some more advanced ORM (still fairly simple
though, mapping entity results from a stored procedure query). All the
solutions I saw for this involved kindof hacking the db connector to execute
some raw queries. I was curious as to why such a basic feature wouldn't be
natively supported, so eventually I found some forum posts asking why it isnt
(it may be by now) and all the responses were in the vein of: "That is not the
Rails way and therefore the core team chooses not to support those types of
features".

I know people think Java is too bulky & sprawling but after 6 months with JPA,
SpringData, Spring JDBC etc. I was kindof left with the feeling "Man, there is
a bit of a learning curve to pick up the overarching concepts of the Java
ecosystem and its design paradigms at first, but once you are over that hump
you can do more, easier with _any_ ORM lib than with ActiveRecord". I know
there is another Ruby ORM gaining traction (DataMapper) but it just seems that
the opinionated nature of Ruby/Rails core dev teams makes it doomed &
dangerous for enterprise use (and indeed maybe its not even their goal).

The Java approach is bulky and design-by-committee for sure but their userbase
seems to demand that the fullest possible spectrum of features be supported,
and then the library developers try to provide their recommended approach for
new projects. Also the specs expand pretty quickly... had to use JPA 2.0 for
some projects and found myself constantly frustrated because in the
advancement to 2.1 there seemed to be a landslide of awesomeness added to
support missing advanced database features.

I guess its one of those things where now it feels like a breath of fresh air
thinking "I have more power in my pinky than....". It's a shame too cuz I
still get calls from companies in a lurch desperate for RoR devs but
philosophically I just can't bring myself to go back in that direction after
seeing how unscary & mind-bogglingly powerful the enterprise langs have
become. In part thanks to RoR, I'm sure! Wouldn't mind using it for front-end
but all that stuff is so interchangeable, the syntax diffs are barely noticed
in the development process. I think really the only front-end techs with
noticeable differences in feel are the ones with advanced data-
binding/component libraries.

Maybe there are some mind-blowing gems out there now though? Who knows....

~~~
dankohn1
You are ignoring a decade of history in declaring Rails to be "dangerous for
enterprise use". The reality is that Rails apps are used in thousands of
enterprises around the world, as well as in "web-scale" businesses like
Github, Airbnb, and Groupon.

However, Rails (and specifically ActiveRecord) has a very specific design
philosophy (see [http://david.heinemeierhansson.com/2012/rails-is-
omakase.htm...](http://david.heinemeierhansson.com/2012/rails-is-omakase.html)
) which does not include support for stored procedures (because business logic
belongs in the app, not the database).

Of course, there are ways to make stored procedures work with Rails (see
[https://github.com/leopoldodonnell/uses-stored-
procedures](https://github.com/leopoldodonnell/uses-stored-procedures) ) but
they are likely to end in tears, since you are working against the grain of
the framework.

The basic argument for Rails is not about support for or against any specific
underlying technology (if you can reach Facebook-scale on PHP, then you can
make anything work), it is that using a mature and well-crafted framework
maximizes developer productivity.

For most startups, as well as most enterprises, that is the critical resource.

~~~
foobarian
The "enterprise" I work at uses a dozen little RoR apps to put a quick and
dirty UI on some DB tables so analysts/sales/support folks can interact with
the data.

The main problem with Ruby in this kind of setting is that these internal
tools are extremely hard to maintain because of backwards compatibility issues
--which there are plenty due to lack of spec, organic language development
etc. The effort to upgrade the Ruby version is much greater than doing
incremental hacks to support some half-day feature, so the codebase stays
pinned to the 2007 Ruby release, we can't use new gems, have to live with old
bugs or missing features, etc. And with every new incremental change the
project gets harder to upgrade.

The Java ecosystem has done a lot better in this regard. Scala is not very
good either, we have had a couple Scala projects with version lock-in effect
as well.

~~~
marktangotango
This is the longterm perspective thats rarely seen here, thank you.

------
bhrgunatha
> Later that year, at RubyConf 2008, I gave a talk titled, What Does My Ruby
> Do about RubySpec. Matz and several other MRI developers attended.
> Immediately after my talk, contributors to Rubinius sat down with Matz and
> other MRI developers to discuss their effort to create an ISO specification
> for Ruby. We asked whether RubySpec could be part of the specification but
> were told that it was not appropriate to include it.

This seems telling.

Does anyone have any concrete information about why Matz and the Ruby team are
opposed to using RubySpec then?

Has there been any progress on creating an ISO spec?

EDIT: It seems Ruby has a published ISO spec since April 2012 -
[http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...](http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579)

------
tessierashpool
I don't want to get in the middle of drama, but if you take anything away from
this, read the actual code.

Just read the actual code of the MRI tests.

------
lnanek2
Matz is the creator of Ruby. If this guy really wanted to play ball, why
didn't he just submit PRs for improving the real Ruby's test suite? Instead he
just made his own thing, well duh it didn't end up a part of the real Ruby.
OK, a hosting provider wanted their own Ruby implementation to fix concurrency
issues, but that doesn't suddenly make the creator of the language have to do
things their way. If you fork or reimplement a project and do whatever you
want, well you took control so you gained something, but the creator has no
obligation to change to your fork/reimplementation.

~~~
krschultz
That glosses over all of the arguments the OP made about making a test suite
that can work with all implementations of Ruby.

------
Terr_
Regardless of other warts with the language, I really love how the Java
Language Specification lays down the law.

~~~
mbell
I find Java's approach to specs to be one of the worst parts of the ecosystem.
I've come across numerous obvious bugs with various Java specced libraries
that have been marked `won't fix` because the bug is actually either a bug in
the spec or just got left out in order to rush the spec out the door (rush
being a loose term here given how often Java actually updates it's various
specs).

~~~
sinistersnare
At least the bugs are documented. Ruby bugs just are, theres nothing
confirming them.

Java bugs need to stay there because if they were fixed then it would be a
backward incompatible change.

~~~
mbell
That attitude is exactly why I have no interest in working with Java.

~~~
mindcrime
Wait, you're saying you'd prefer undocumented bugs and a constant stream of
breaking changes to the runtime that force you to constantly port and report
existing, working code?

I don't get it... the Java way seems pretty right to me. That said, I will
allow that they might have been a little bit _too_ rigid about not allowing
breaking changes. At the very least this policy has resulted in Java evolving
more slowly than it might have otherwise. But I don't necessarily find that to
be unacceptable, all things told.

~~~
mbell
I prefer that bugs get fixed when found.

In my experience the Java way is 'oh, a bug, well, that's a bug defined in the
spec so we can't fix it now, we'll make a note for the next version'...5 years
pass.

The 'I don't want to touch working code' argument is another strange Javaism.
If you expect your code to not rot your going to have to constantly maintain
it anyway.

~~~
gear54rus
And sadly, it seems the same with JS. Year 2015:

    
    
      x = null;
      typeof x; //object
      x.a = 1;  //TypeError: x is null

------
hartator
Has Matz stated the reasons they are not using RubySpec? Knowing him, I can't
really believe he has chosen to ignore RubySpec without good reasons.

~~~
drivingmenuts
[https://bugs.ruby-lang.org/issues/7549#note-2](https://bugs.ruby-
lang.org/issues/7549#note-2)

Apparently, he and some of the key devs don't like design by committee or much
in the way of bureaucracy.

~~~
Sivart13
The problem with reading too much into that comment is that the issue being
commented on involves setting up an entire "design process" for ruby,
including councils and voting and this and that.

It's troublesome to conflate the bulk of that proposal with the reception of
RubySpec itself. It may be that some of the Ruby folks would've been receptive
to a more limited proposal like "all released versions of Ruby should pass
RubySpec", but we don't live in the universe where that happened.

Even then, telling a group of language developers that they should commit
tests into what is effectively your "pet project" is a big ask.

~~~
taf2
Could also have submitted bug reports during the RC pointing to a failing ruby
spec test especially the segfault...

------
jes5199
I almost didn't catch the epilogue - he's shutting down the RubySpec project
entirely, because it hasn't accomplished what he hoped.

~~~
headius
I believe he's shutting it down as a political move. They're still going to
use it to maintain Ruby compatibility and they'll still need to run it against
MRI. It just isn't a standalone project now.

------
senthilnayagam
@senthilnayagam: . @yukihiro_matz can you respond
[http://t.co/qeeVAluOhJ](http://t.co/qeeVAluOhJ)

@yukihiro_matz: @senthilnayagam I am not in charge of testing. But as far as I
understand it has been communication problems. Blaming no use.

@senthilnayagam: . @yukihiro_matz when merb could merge with rails, rubyspec
shpuld merge with MRI & become official reference for all Ruby implementations

------
joevandyk
Why don't MRI developers use RubySpec?

~~~
nateberkopec
As linked in another comment, basically:

[https://bugs.ruby-lang.org/issues/7549#note-2](https://bugs.ruby-
lang.org/issues/7549#note-2)

1\. Matz is, and always will be BDFL. 2\. The core Ruby team wants to talk in
terms of C and MRI being the Ruby spec, not a third party/written RubySpec

~~~
vidarh
That's an absolutely terrifying attitude from a language developer....

~~~
nateberkopec
Is it? Python is _exactly the same_. It has no language spec (defined by its
implementatation), and it has a BDFL who hasn't pushed for one.

~~~
nedbat
It would be good for Python to have more rigor in its definition. But the
situation here doesn't have an analog in the Python world: there is no
"PythonSpec" third-party test suite that is a) more complete than the official
suite, b) exposing segfaults in previously working code, and c) going unused
by the core developers.

------
msie
What of the ISO standard that mRuby is based on?

[http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...](http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579)

Aw crap, I have to buy this doc to see it?

~~~
cremno
The final draft is available:
[https://www.ipa.go.jp/osc/english/ruby/](https://www.ipa.go.jp/osc/english/ruby/)

But the ISO document is mostly irrelevant, if we talk about Ruby
implementations like CRuby, JRuby, or Rubinius.

------
dyeje
Some of the discussions he links to in the article are disturbing. Many posts
refusing to implement process because they don't want any process at all.
Doesn't seem like a healthy approach.

~~~
headius
Be wary of this cultivated list of links. There are others out there that
would make it very clear why the proposed design process was a misfit given
the projects and people involved.

------
issaria
THIS Brian again, nobody uses rubinius and it doesn't hurt anyone. This guy
has a long history of promoting rubinius by attacking MRI, first time I saw
him was on Baruco 2013, if you watch the video, his attitude is so annoying.
Evan Phoenix's Ruby implementation was an interesting project, but now totally
ruined by this childish guy. Brian, my advise is to stop bitching about MRI
and seriously improve the documentation of rubinius, the are many interesting
topics on rubini.us, but most of them are WIP since forever.

------
serve_yay
Jeez, what a mess.

------
danielweber
Why _should_ they have been using RubySpec?

~~~
joevandyk
Did you read the article? It contains 10 reasons why the MRI tests aren't
sufficient.

~~~
danielweber
The Ruby devs also doesn't use any libraries I wrote, and that's okay.

Why is _RubySpec_ the special sauce that is so special that the devs not using
it is horrible?

~~~
stormbrew
Honestly, just the fact that rubyspec exposes a segfault in a new release is
more than reason enough that they should be using it. It is de facto a more
complete test suite than the ad hoc one they've been making.

Given that, the onus is on the MRI developers to demonstrate why they are
actively avoiding the use of a tool that could improve the stability of their
language and environment.

~~~
headius
They have used and contributed to RubySpec in the past, but many (like me)
were turned off by the maintainers' attitudes toward contributors and lack of
respect. For example, see the zenspider link elsewhere in this thread.

Nobody questions RubySpec as a project. But there are many nontechnical
reasons why it was never adopted wholeheartedly by ruby-core.

~~~
joevandyk
Now that the main maintainer quit, can those nontechnical reasons be resolved?

~~~
jpgvm
Hopefully. I would be willing to take a crack at it.

~~~
vidarh
If you (or anyone else) do, I'd be ready to help.

------
crb002
MRI unit tests are the spec.

[https://github.com/ruby/ruby/tree/trunk/test/ruby](https://github.com/ruby/ruby/tree/trunk/test/ruby)

------
jc00ke
I mistakenly cut off the full title of the post: "Matz's Ruby Developers Don't
Use RubySpec and It's Hurting Ruby"

------
grover_hartmann
Fuck, this idiot is now being pendatic in #ruby as well.

2015-01-02 14:45:30 brixen raise your hand if you have implemented Ruby and
used MRI's tests

Glad he didn't get things to go his way. He cries too much. If you don't like
some projects go write your fucking own.

------
jagawhowho
Ruby is about to die permanently. A language that copied emacs-lisp but with
syntax? Lol that gives it features to impress the ignoramus majority but real
Haskell or Lisp programmers know better.

