Hacker News new | comments | show | ask | jobs | submit login
MRI Developers Don't Use RubySpec and It's Hurting Ruby (rubini.us)
291 points by jc00ke 1119 days ago | hide | past | web | favorite | 112 comments



Rubyspec is a great project for all the reasons Brian outlines.

However, it's also a failure - in part, due to Brian and his attitude towards contributors. See this twitter conversation: https://twitter.com/the_zenspider/status/547527644535726080 He's been grinding on Rubyspec for years, bless him, but I think there's a reason why he was unable to rally the community behind his effort, both in terms of gathering more contributors and in making it "official" in terms of the language spec.

JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use, runs the MRI test suite against JRuby. RubySpec is far from The Only Solution, though Brian would like you to think it is.


"RubySpec is far from The Only Solution"

you're right, but I remember learning quit a bit of ruby by working through RubySpec back in the day. and I remember liking how well structured, logical and organized the suite was, very exemplary. so I'm sorry to see it go. I have no perspective on Brian or the mri devs so I can't say anything about the situation around it, I just liked the code and spec and found it very well done.

It would certainly be nice to have such a ruby specification and test suite that was endorsed by ruby community at large (unlike what I saw in those example MRI tests).


I came to the thread to make sure someone was spreading truth rather than FUD, and this was the first comment I saw. Bless you.


>However, it's also a failure - in part, due to Brian and his attitude towards contributors.

Looks like he seems to be burned out - maybe he just wants something in return for his time investment e.g. a paid job or some other gratification in return for his work.

Btdt - you put a lot of work into something, people/companies start benefit from it but they don't return anything important (monetary compensation is not a bad thing -it allows one to pay the bills/go on holiday/get a life).

We've seen this "pattern" a lot in the OSS world, but saying a different attitude would "fix everything" is a lie. We (as in the Ruby community) need to crowd-fund independent developers who are not paid by companies for their OSS contribution IMHO.


Brian has been paid to work on Rubinius and RubySpec full time for at least 6 years.


oh - didn't know that. Okay that makes a hughe difference IMHO.


> JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use

Are there concrete shortcomings when using Rubinius in production, or is it just not battle-tested yet?


Every time I've tried to use it, it's either been unstable (segfaults are fun) or orders of magnitude slower than MRI.

It's a promising project, but as far as "no GIL, true concurrency, mature GC" implementations go, JRuby delivers the goods.

Edit: For kicks, I'm trying to boot my primary app with RBX right now. RTLK threw some "id for nil" exceptions due to some initializer stuff, so I disabled that, and now, trying to run my test suite, after 2 minutes of sitting there doing who-knows-what, it finally attempts to boot my app and dies with a "Missing constant" error despite the fact that I put in debug code to ensure that the file containing said module is a) loaded, b) executing properly, and c) that said constant actually exists in the module tree.

This code boots and runs just fine under MRI 1.9, 2.0, 2.1, and JRuby 1.7. I might be able to get it running with some more work and explicit load path management, but I'm off to a party for now, so I'll poke at it a bit when I get back.


It's disappointing to hear about such problems with Ruby. I hit the same problem with the Groovy Language spec when I first came across Groovy. Its creator, James Strachan, initiated an implementation, test kit, and spec all within 6 months of each other (impl beta-1 in Dec 2003, and spec JSR-241 in May 2004). The project managers who took over from him, Graeme Rocher and Guillaume Laforge, changed direction by stopping work on the spec and refocusing the Groovy reference implementation to be the scripting language behind Grails. (Of course, Groovy 'n' Grails was intended to chisel away at some of the market share of Ruby on Rails but that's another story.) Strachan often wrote that the spec was to enable anyone to make their own implementation of Groovy if they want to, and right up to his very last posting ever http://groovy.329449.n5.nabble.com/Paris-write-up-tt395560.h... on the Groovy mailing list on 5 Dec 2005, he maintained that what they were building was the reference implementation.

If Rocher and Laforge had come clean about how they turned the RI into the language itself, the backlash might have blown over quickly, but instead they led developers along for many years afterwards, not changing the spec to dormant until April 2012. Projects other than Grails who've tried to build atop Groovy have had to risk the ref impl changing in breaking ways between versions. The most spectacular incident was when Groovy++, an experimental static compiler built by Alex Tkachman that hooked via annotations into Groovy's AST, had to drop back down from Groovy 1.8 to 1.7 in 2011, and my own side project was also affected by the change. It turned out Rocher and Laforge had secretly employed a mate to extend Groovy with the exact same static type-checking and compilation functionality as Groovy++ and were obviously trying to shake us off.

Unlike Ruby, Groovy only has one other implementation, GrooScript, built by Jorge Franco, which generates JavaScript from Groovy syntax. When the developers of the most used implementation of a language want to protect their control, it certainly does hurt the ecosystem, turning it into an "echo system".


Wow. Fascinating bit of context. Thanks for sharing it here; it's the kind of narrative that is incredibly hard to appreciate if you're not directly affected/involved.


Are you sure you're not assigning to malice that which could be due to ignorance or indifference?


When you see the same behavior repeated consistently over 10 years, it doesn't matter if it's done consciously (what you call "malice") or as an unconscious habit ("indifference"), the effect is what matters and the intent is the same. The Groovy 2.4 release candidate was released a few days ago and announced on the personal blog of the project manager, but not on the Groovy community mailing list, a consistent behavior which started about year ago. Also a year ago, he launched a weekly mailout newsletter he controls about Groovy separate from the Codehaus repository or the backing company Pivotal Inc, soliciting for subscribers. It all looks like a clear attempt to take over from the other developers involved in building the Groovy codebase. I listed plenty of previous examples of such behavior by these project managers going back 10 years that's harmed the ecosystem, probably irrepairably, and I hope Ruby doesn't go down the same path.


Both Charles Nutter and Matz are smart guys. They are also both great people. I say this having spent time with both of them. This whole, my implementation is better than theirs coming from Brian is crazy.

Matz and Charles have helped to set the tone for the community. While I appreciate Brian's passion, it sounds like he needs to check his ego. No matter how smart we are, we can always learn things from other people. Without the collaboration of others neither JRuby or Ruby would be what they are today which is why they are successful.


He might need to check his ego, but the state of Ruby specification is deplorable, and despite being a big fan of Ruby, it's also a big road block to development.

The small number of mature Ruby runtimes is a bad sign.


See, iirc, Matz years ago specifically said "fork Ruby". He wants lots of different versions which I feel to some extent means diverging from a formal spec, and then occasionally maybe coming back to it.


And that's even worse. I'm working on a Ruby compiler. While it is far from being at a stage where this is a real problem, currently there's no sane way of knowing whether or not I'm "close enough" to MRI other than extensive testing of every single piece of Ruby code I want to work.

"Diverging from a formal spec" is one thing. As it is, many versions of MRI has diverged from how the core team thinks it works. If MRI is "the spec", then Ruby changes from revision to revision as the test coverage is nowhere good enough to prevent regressions or changes in behaviour.

Basically: Nobody knows Ruby.

Frankly, I very much hope that another Ruby implementations overtake MRI sufficiently in quality to out compete it enough to shift the initiative of definining the language to a more responsible team.


[deleted]


If MRI fails RubySpec specs, and the MRI developers didn't write those specs, is that a bug in RubySpec or MRI?

It's egotistical because he's saying they should make their project conform to his third party specification.


> If MRI fails RubySpec specs, and the MRI developers didn't write those specs, is that a bug in RubySpec or MRI?

If the specs were written based on (and working in) a previous MRI version, how can it be anything but an MRI bug? Especially when previously working code segfaults?

> It's egotistical because he's saying they should make their project conform to his third party specification.

How could he do anything but a third-party specification when the first-party refuses to produce a specification?


A segfault from running interpreted code is definitely an MRI bug.


Unless there are third party C extensions involved.


The thing I find depressing and frustrating about this kind of discussion is the fatalism and non-constructiveness.

I really wish it was:

"I set up a server that runs ruby spec on ruby-head daily and automatically reports spec failures to ruby-bugs"

So many companies are making big bucks off Ruby, yet so little are willing to fork out a bit of money and time to make Ruby better.


it's unreasonable to expect the dev who has already put a ton of work into developing and maintaining rubyspec and making it work nicely with all the ruby implementations and versions he could to have to put in the additional work to do this as well. I don't blame him for getting discouraged at having to chase a moving target on top of all that - I agree with him that having the MRI devs contribute to rubyspec was a reasonable thing to expect.


Why is it unreasonable to expect the RubySpec dev to do additional work, but reasonable to expect additional work from the MRI devs to comply with RubySpec?

I get that "the implementation is the standard" is frustrating for people who want to make an alternate implementation, but why should that obligate the original developers to accept some third party's definition of what their project should be?


They don't have to accept it. I don't have an opinion one way or another about the rubyspec being some kind of a standard. But if it exists and shows a sigsegv scenario and nobody bothered to run it before a release... that's a failure of developers. It's not even additional work.


It's not a third-party definition, it's their definition, should they want to contribute to it. By writing Ruby specs you're defining Ruby.


> By writing Ruby specs you're defining Ruby.

And presumably the MRI devs would say that by coding MRI you're defining Ruby. What gives RubySpecs the authority to say they are the canonical definition of Ruby and everyone should follow it, when the people who created Ruby disagree?


If that's their attitude, then frankly there's every reason to complain about that. Defining a language by implementation is absolutely horrible practice.


You can't call code a spec. Saying you're defining Ruby by coding it just means you don't have a spec.


In my experience, executable specs are much more useful than their non-executable brethren. They prove their own correctness, and can't gloss over important details.


> They prove their own correctness, and can't gloss over important details.

They most certainly don't, do either of those things. You can still have incorrect, incomplete specs even with an executable test suite. You only get those properties through constant work.

The idea behind TDD is to do that as you develop the application, but even then you're not automatically saved from incorrectness and incompleteness.

The only way to really do it is the way RubySpecs intended, create a third place for the specification of the language to live, that any implementation can use to measure its compliance against. If anyone running any implementation on any platform finds a problem and traces it to the implementation, then a new test case can get written and all implementations can then fix the bug assuming it's present. Code quality goes up for everyone.

Most other applications don't need to do it this way, they're intended for an install base of one. But since Ruby has multiple implementations intended to be able to run under multiple platforms on multiple architectures, a living collection of specs that ensures that each combination still has the same behavior would seem to be absolutely essential for one's sanity if one cares about the quality of the language. Each implementation then only has to maintain platform-specific test suites. It's ultimately less work.

But just like TDD/BDD, it requires organizational buy-in to really get the benefits. Everyone working on Ruby needs to be involved in maintaining the test suite or there's no way you can consider the suite canonical. If you can't consider the suite canonical, there's no point in having it.

I develop on MRI at present, but at some point I'm going to switch to either JRuby or Rubinius. If Brian actually follows through on his goals to implement even better ways of specifying Ruby than a test suite, I'll probably switch to Rubinius.


It's interesting reading your comment that purports to disagree with mine, while feeling that it instead agrees... I think perhaps the confusion is that I misread your comment that I replied to as suggesting that you can't have an executable spec, rather than (as I now think you meant) as merely claiming that an implementation can't be its own spec, which I definitely agree with.

I think something like RubySpec is fantastic, and that it's really too bad that, perhaps only because of personality conflicts with the maintainer, it hasn't been successful at standardizing behavior across ruby implementations.


I suspect from this reply that you're confused about what rubyspec is. Rubyspec is an executable spec. It is thousands of unit tests defined against the official ruby implementation (MRI).


They dont define anything, they are tautological.


A program can't prove its own correctness in any useful way. Especially for a system that is under continual development.


Throwing away a project that is clearly working is depressing. can't Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a system that runs the suite on latest?

It's just such a better outcome, nobody needs to change workflows its just that information gets reported upstream earlier in a much more useful time.


> can't Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a system

Heroku does sponsor Matz, Nobu, and Koichi. We hired them and give them a full time salary to work on Ruby.We don't tell them what to work on (i.e. we don't dictate what features or projects get shipped in what versions) it's more like corporate sponsorship. GitHub hired tmm1. Beyond that CRuby has a host of other non-sponsored contributors that contribute code and doc patches as well as set up tooling. You don't need corporate sponsorship for that, you need a passion and some time. You, in fact, could be the very person that sets up this automated system.

I find the attitude of '<company x> should sponsor <thing y>' a bit misguided. While I agree that companies who profit from a OSS should give forwards to OSS, individual contributors are ultimately they only way progress gets made. If you or someone reading this works for a company making money off of Ruby do that thing! Tell your boss you can't deploy on friday afternoon and have to fix a bug in the Ruby codebase. Ship the project on company time, and then boom...your company just sponsored that thing. It's like magic!

Anywhoo, yes finding out what things are blocking the MRI team from adopting a tool/technology and working around them can be extremely valuable. For example Matz has agreed to move the codebase development to github. This would make contributions a bit easier, however there is a blocker. There are a ton of SVN bots and tooling written around the current workflow. Right now the Ruby core team wants to spend time focusing on pumping out C code to fix bugs, improve performance, and progress the language forwards. They don't want to go off into the woods on tooling. These types of projects could be hugely impactful to the team and don't require C knowledge so most Ruby devs may be able to help. I encourage you and others to reach out to the core team to ask them what tooling projects they need and how we can help them. Understanding why someone isn't using a tool/technology is a really good start.

tldr; Good idea, let's talk to the core team about ways to help with tooling.


The point is that it's not sufficient to run the current 20k tests against head daily; there are so many unplanned variations and 'emergent' design that there's a full time job just in keeping up with the changes to MRI ruby.

And even if there wasn't, @brixen has repeatedly pointed out how the MRI authors have chosen to ignore the RubySpec project entirely. That's really sad.


The Ruby devs attitudes in the discussions linked FTA makes me question the longevity of Ruby as a serious Enterprise language.


I suspect this isn't a goal of those Ruby devs. It almost seems that "Ruby" is more like an ideology rather than a language, at least from comments like "Continuous changes is Ruby."

Note that I'm not trying to criticize this position, rather, I'm pointing out that the core Ruby team seems to have different goals than, say, the community around Java or C#.


When did it become a serious Enterprise™ language? Not being flippant or denigrating Ruby, but when I do work for established enterprises, it's never in Ruby. Java, .NET, PHP, and occasionally Python. Just because startups use it to get off the ground quickly, or to build the front end of their website doesn't make it enterprise.


Ruby (MRI/JRuby) are gaining traction in automated testing at large enterprises. Ruby is easy to learn and integrates well with java.

Cucumber provides better reporting than JUnit/TestNG. For web testing you have watir. For SOAP services you have savon which IMO is much easier than SOAPUI. For RESTful web services you just make direct calls. Then there is sikuli for ad hoc GUIs.


How do you define enterprise? I've been using ruby in projects I wouldn't call 'startup' for the last several years... including with teams up to 80 people (i.e. real enterprise)


yes I don't want to get into a flamewar either, but I am an ex-RoR dev and since switching to enterprise Java I hear almost nothing about it largely due to enormously widespread acceptance of Spring (JavaEE is even re-gaining some traction).

Long story short (and hope my facts are right! not swearing by this, was just some googling...), I had to optimize some old code for an RoR client I freelance for and wanted to add in some more advanced ORM (still fairly simple though, mapping entity results from a stored procedure query). All the solutions I saw for this involved kindof hacking the db connector to execute some raw queries. I was curious as to why such a basic feature wouldn't be natively supported, so eventually I found some forum posts asking why it isnt (it may be by now) and all the responses were in the vein of: "That is not the Rails way and therefore the core team chooses not to support those types of features".

I know people think Java is too bulky & sprawling but after 6 months with JPA, SpringData, Spring JDBC etc. I was kindof left with the feeling "Man, there is a bit of a learning curve to pick up the overarching concepts of the Java ecosystem and its design paradigms at first, but once you are over that hump you can do more, easier with any ORM lib than with ActiveRecord". I know there is another Ruby ORM gaining traction (DataMapper) but it just seems that the opinionated nature of Ruby/Rails core dev teams makes it doomed & dangerous for enterprise use (and indeed maybe its not even their goal).

The Java approach is bulky and design-by-committee for sure but their userbase seems to demand that the fullest possible spectrum of features be supported, and then the library developers try to provide their recommended approach for new projects. Also the specs expand pretty quickly... had to use JPA 2.0 for some projects and found myself constantly frustrated because in the advancement to 2.1 there seemed to be a landslide of awesomeness added to support missing advanced database features.

I guess its one of those things where now it feels like a breath of fresh air thinking "I have more power in my pinky than....". It's a shame too cuz I still get calls from companies in a lurch desperate for RoR devs but philosophically I just can't bring myself to go back in that direction after seeing how unscary & mind-bogglingly powerful the enterprise langs have become. In part thanks to RoR, I'm sure! Wouldn't mind using it for front-end but all that stuff is so interchangeable, the syntax diffs are barely noticed in the development process. I think really the only front-end techs with noticeable differences in feel are the ones with advanced data-binding/component libraries.

Maybe there are some mind-blowing gems out there now though? Who knows....


The main benefit of Ruby seems to be the ability to catch method names and treat them as an argument. I bought "Metaprogramming in Ruby", and the author seems delighted that you can do "mycvsrow.columnname". Several other people have confirmed to me that it is really just being able to drop off quotes around what would otherwise be strings in other languages. I don't get it, at all. Maybe it's a knee-jerk reaction to verbose languages like Java. But I fail to see what Ruby offers over a Haskell or F#-like.

I got excited by seeing how passionate and excited people like patio11 were about Rails. But then seeing how Rails prefers insecure defaults and had exploits because they were essentially eval'ing user posted data really hampered my enthusiasm.

Seeing articles like this reinforces the idea that Ruby is cowboy programming, just making stuff up as they go along. Like PHP but with class.


Although I agree with what you are saying, this is hardly new. (And still disappointing IMO).

Developers in the RoR community have a tendency to say that "Rails is very good for one set of problems (Basecamp), and using it for anything else is problematic". This has been the answer for every attempt that tried to introduce some worthwhile idea to the Ruby community, be it OO design, TDD or something more "enterprisey" like hexagonal architecture. It's not the Rails Way™, move on.

I've heard developers whom I respect (ie. Avdi Grimm, Sandi Metz) shrug this inherent limitation of Rails off, and I find it really weird. It's akin to Stockholm syndrome really.


You are ignoring a decade of history in declaring Rails to be "dangerous for enterprise use". The reality is that Rails apps are used in thousands of enterprises around the world, as well as in "web-scale" businesses like Github, Airbnb, and Groupon.

However, Rails (and specifically ActiveRecord) has a very specific design philosophy (see http://david.heinemeierhansson.com/2012/rails-is-omakase.htm... ) which does not include support for stored procedures (because business logic belongs in the app, not the database).

Of course, there are ways to make stored procedures work with Rails (see https://github.com/leopoldodonnell/uses-stored-procedures ) but they are likely to end in tears, since you are working against the grain of the framework.

The basic argument for Rails is not about support for or against any specific underlying technology (if you can reach Facebook-scale on PHP, then you can make anything work), it is that using a mature and well-crafted framework maximizes developer productivity.

For most startups, as well as most enterprises, that is the critical resource.


The "enterprise" I work at uses a dozen little RoR apps to put a quick and dirty UI on some DB tables so analysts/sales/support folks can interact with the data.

The main problem with Ruby in this kind of setting is that these internal tools are extremely hard to maintain because of backwards compatibility issues--which there are plenty due to lack of spec, organic language development etc. The effort to upgrade the Ruby version is much greater than doing incremental hacks to support some half-day feature, so the codebase stays pinned to the 2007 Ruby release, we can't use new gems, have to live with old bugs or missing features, etc. And with every new incremental change the project gets harder to upgrade.

The Java ecosystem has done a lot better in this regard. Scala is not very good either, we have had a couple Scala projects with version lock-in effect as well.


This is the longterm perspective thats rarely seen here, thank you.


hi dan,

i apologize, i must have been unclear because it seems that you (& some of the commenters below) are dissecting some of my statements quite literally whereas i put a lot of disclaimery type of statements to try to indicate that this is a combo of personal opinion / hazy memory (if the commenter below ever sees this he can also now receive my apology about mentioning datamapper as an ORM gaining traction, that must be a memory from years ago when i still did RoR). :D

I am aware that RoR has been successfully adopted many places, but i guess part of what i failed to explain here (that thankfully foobarian touches upon a bit) is that i believe that part of the power of "a spec" in a programming context is to maintain a solid core of functionality, that then also paves the way for expanded functionality in the future based on extensions of a similar style of syntax.

I know Ruby & Rails teams are not one and the same, I just have many memories of experiences in that community where library evolution is not incremental, does not expand functionality, deprecates as a rival lib rises in popularity.

In Java you see 2 things -- slow spec evolution & incremental expansion of features. It takes time (I know the fast-moving lang communities hate this) but I find it makes for much more robust codebases that survive upgrade processes with wayyyyyy less re-factoring/re-writing, and in type even some libraries that had their own syntax start to embrace/support the spec (Hibernate implements JPA now...).

So.... I suppose, yes, it is a personal matter perhaps. Some companies have used Rails successfully. But in freelance it has kindof bitten me (the overhead of dealing with security vulnerabilities / gem upgrades alone is astronomically more than I've dealt with in upgrading Java projects). I understand that a competent team can handle this & enterprises can pour money into making it work, but as a programmer who often works alone & hates forced unplanned maintenance, it is a tough pill to swallow.

Also, maybe it is because I am dealing with a lot of enterprise/legacy code at times. Sure, maybe business logic shouldn't go in the database. But if someone supplies me a library of highly complex Oracle functions & tells me i need to wrap these into services ASAP, am I really going to begin an immediate re-write? I would prefer that my framework is just able to wrap these functions until a re-write can be scheduled (if the codebase isn't retired before then...). Also in the enterprise community I don't find as much of a desire to have ALL logic in the app layer... often devs feel that as long as a stored procedure / view / whatever returns a meaningful coherent entity that is useful in the business logic layer, they don't really care where/how the query was aggregated.

I also understand anything can be made to scale & work for an enterprise... but it takes enterprise-level support!! I am more a champion of the rogue programmer trying to build an enterprise with a few lines of code a day. ;)

Best, john


DataMapper isn't gaining traction, sadly. v1 is abandonware, v2 forked off to become a different project called ROM (which explicitly "isn't an ORM"). I still use DataMapper v1 because I find it a thousand times nicer than ActiveRecord, but it's a minority taste.


That sounds like a philosophy of RoR, not Ruby. How would adopting RubySpec help with that?


The author claims that he is a professional developer, while bashing at a programming language (Ruby) because he wanted more abilities from an ORM (AR) of a web framework (Rails).

He thinks that DattaMapper is trending, when it's clearly not(!!) and doesn't know Sequel (another popular actively developed ruby ORM). Hence it's easy to deduce that he doesn't have a clue about the landscape he is talking about.

Yet he is bashing Ruby as if it's hard to write a module with a couple of complex, custom-made raw SQL queries.

... blah! ...


RS needs work contribution from each Ruby to make it sustainable.

It would also show leadership from MRI to encourage a semi-independent Ruby self-test framework that can run on any Rubyish interpreter. (Granted RS has RBX bias, but it would show leadership to coordinate common infrastructure.)

Since RS exists, seems like a good idea to go with that for full up integration testing, but each Ruby should still unit test it's own, low-level bits. (Is MRI doing that at least?)


> So many companies are making big bucks off Ruby, yet so little are willing to fork out a bit of money and time to make Ruby better. <

And why would/should they when there are quite literally dozens (or possibly hundreds or even thousands) of developers who will happily work on "making Ruby better" for no cost to said companies?

A large percentage of software developers will quite happily hack away at projects for nothing in return simply because they enjoy it (and would, quite incorrectly, claim that as compensation itself), in order to gain notoriety, boost their own ego, or any of a number of related reasons.


> reports spec failures to ruby-bugs

But why should the Ruby developers work to fix 'bugs' against an unsupported 3rd party specification?


It's essentially regression testing. Rubyspec specs out the behaviour of Ruby according to what MRI implements, so any failure is a change in behaviour from the initial implementation. Most of the time this is undesirable/unintentional.


> Later that year, at RubyConf 2008, I gave a talk titled, What Does My Ruby Do about RubySpec. Matz and several other MRI developers attended. Immediately after my talk, contributors to Rubinius sat down with Matz and other MRI developers to discuss their effort to create an ISO specification for Ruby. We asked whether RubySpec could be part of the specification but were told that it was not appropriate to include it.

This seems telling.

Does anyone have any concrete information about why Matz and the Ruby team are opposed to using RubySpec then?

Has there been any progress on creating an ISO spec?

EDIT: It seems Ruby has a published ISO spec since April 2012 - http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...


I don't want to get in the middle of drama, but if you take anything away from this, read the actual code.

Just read the actual code of the MRI tests.


Matz is the creator of Ruby. If this guy really wanted to play ball, why didn't he just submit PRs for improving the real Ruby's test suite? Instead he just made his own thing, well duh it didn't end up a part of the real Ruby. OK, a hosting provider wanted their own Ruby implementation to fix concurrency issues, but that doesn't suddenly make the creator of the language have to do things their way. If you fork or reimplement a project and do whatever you want, well you took control so you gained something, but the creator has no obligation to change to your fork/reimplementation.


That glosses over all of the arguments the OP made about making a test suite that can work with all implementations of Ruby.


Regardless of other warts with the language, I really love how the Java Language Specification lays down the law.


I find Java's approach to specs to be one of the worst parts of the ecosystem. I've come across numerous obvious bugs with various Java specced libraries that have been marked `won't fix` because the bug is actually either a bug in the spec or just got left out in order to rush the spec out the door (rush being a loose term here given how often Java actually updates it's various specs).


At least the bugs are documented. Ruby bugs just are, theres nothing confirming them.

Java bugs need to stay there because if they were fixed then it would be a backward incompatible change.


That attitude is exactly why I have no interest in working with Java.


Wait, you're saying you'd prefer undocumented bugs and a constant stream of breaking changes to the runtime that force you to constantly port and report existing, working code?

I don't get it... the Java way seems pretty right to me. That said, I will allow that they might have been a little bit too rigid about not allowing breaking changes. At the very least this policy has resulted in Java evolving more slowly than it might have otherwise. But I don't necessarily find that to be unacceptable, all things told.


I prefer that bugs get fixed when found.

In my experience the Java way is 'oh, a bug, well, that's a bug defined in the spec so we can't fix it now, we'll make a note for the next version'...5 years pass.

The 'I don't want to touch working code' argument is another strange Javaism. If you expect your code to not rot your going to have to constantly maintain it anyway.


And sadly, it seems the same with JS. Year 2015:

  x = null;
  typeof x; //object
  x.a = 1;  //TypeError: x is null


Maybe this doesn't work for the library ecosystem (standard libraries being where modules go to die), but the language and VM specs are very well designed.


> but the language and VM specs are very well designed

Java's language spec exists, I'll give it that. Well designed? It was when it was created but, it has evolved very poorly. Look at C# as an example of what a similar language designed at a similar time could become while still being 'Enterprise'.


Has Matz stated the reasons they are not using RubySpec? Knowing him, I can't really believe he has chosen to ignore RubySpec without good reasons.


https://bugs.ruby-lang.org/issues/7549#note-2

Apparently, he and some of the key devs don't like design by committee or much in the way of bureaucracy.


The problem with reading too much into that comment is that the issue being commented on involves setting up an entire "design process" for ruby, including councils and voting and this and that.

It's troublesome to conflate the bulk of that proposal with the reception of RubySpec itself. It may be that some of the Ruby folks would've been receptive to a more limited proposal like "all released versions of Ruby should pass RubySpec", but we don't live in the universe where that happened.

Even then, telling a group of language developers that they should commit tests into what is effectively your "pet project" is a big ask.


Could also have submitted bug reports during the RC pointing to a failing ruby spec test especially the segfault...


Matz: community members can deprive of the power of the dictator, when he apparently loose his ability to make rational decision. After deprivation, the community will make up new rules, hopefully flexible ones. I don't want that situation, but the day will come sooner of later

I think that's a very disappointing point of view, and I say that as a someone who used Ruby in the past. In light of this I can say it will not be my go-to language in the future.

I feel like after a certain point of community uptake, a project surpasses any single developer and becomes a part of that community in and of itself. I understand wanting to maintain control of your pet project, but for community projects I feel like there should be community control. That said, you should have the freedom and the right to do whatever you want with your pet project. Maybe the solution is for the community to fork so they can make the changes they see fit.

Given the way the MRI devs and Matz in particular have responded, perhaps it's time for the community to fork Ruby if they don't like the way Matz insists on remaining the dictator. It's his project and he can do what he wants with it, but we don't have to use it.


I am of the opinion that, while he took advice and help from the larger Ruby community, he never considered it a "community" project in the sense of communities surrounding other languages.


I almost didn't catch the epilogue - he's shutting down the RubySpec project entirely, because it hasn't accomplished what he hoped.


I believe he's shutting it down as a political move. They're still going to use it to maintain Ruby compatibility and they'll still need to run it against MRI. It just isn't a standalone project now.


@senthilnayagam: . @yukihiro_matz can you respond http://t.co/qeeVAluOhJ

@yukihiro_matz: @senthilnayagam I am not in charge of testing. But as far as I understand it has been communication problems. Blaming no use.

@senthilnayagam: . @yukihiro_matz when merb could merge with rails, rubyspec shpuld merge with MRI & become official reference for all Ruby implementations


Why don't MRI developers use RubySpec?


As linked in another comment, basically:

https://bugs.ruby-lang.org/issues/7549#note-2

1. Matz is, and always will be BDFL. 2. The core Ruby team wants to talk in terms of C and MRI being the Ruby spec, not a third party/written RubySpec


That's an absolutely terrifying attitude from a language developer....


Is it? Python is exactly the same. It has no language spec (defined by its implementatation), and it has a BDFL who hasn't pushed for one.


It would be good for Python to have more rigor in its definition. But the situation here doesn't have an analog in the Python world: there is no "PythonSpec" third-party test suite that is a) more complete than the official suite, b) exposing segfaults in previously working code, and c) going unused by the core developers.


That other languages are equally badly specified is not a good argument for why it's not terrifying.

Then again, I don't care about Python. I do care about Ruby.


They could always have their own fork/branch for upcoming releases of Ruby. It's not like the spec has to freeze the language.


What of the ISO standard that mRuby is based on?

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...

Aw crap, I have to buy this doc to see it?


The final draft is available: https://www.ipa.go.jp/osc/english/ruby/

But the ISO document is mostly irrelevant, if we talk about Ruby implementations like CRuby, JRuby, or Rubinius.


Some of the discussions he links to in the article are disturbing. Many posts refusing to implement process because they don't want any process at all. Doesn't seem like a healthy approach.


Be wary of this cultivated list of links. There are others out there that would make it very clear why the proposed design process was a misfit given the projects and people involved.


THIS Brian again, nobody uses rubinius and it doesn't hurt anyone. This guy has a long history of promoting rubinius by attacking MRI, first time I saw him was on Baruco 2013, if you watch the video, his attitude is so annoying. Evan Phoenix's Ruby implementation was an interesting project, but now totally ruined by this childish guy. Brian, my advise is to stop bitching about MRI and seriously improve the documentation of rubinius, the are many interesting topics on rubini.us, but most of them are WIP since forever.


Jeez, what a mess.


Why should they have been using RubySpec?


Did you read the article? It contains 10 reasons why the MRI tests aren't sufficient.


The Ruby devs also doesn't use any libraries I wrote, and that's okay.

Why is RubySpec the special sauce that is so special that the devs not using it is horrible?


Honestly, just the fact that rubyspec exposes a segfault in a new release is more than reason enough that they should be using it. It is de facto a more complete test suite than the ad hoc one they've been making.

Given that, the onus is on the MRI developers to demonstrate why they are actively avoiding the use of a tool that could improve the stability of their language and environment.


They have used and contributed to RubySpec in the past, but many (like me) were turned off by the maintainers' attitudes toward contributors and lack of respect. For example, see the zenspider link elsewhere in this thread.

Nobody questions RubySpec as a project. But there are many nontechnical reasons why it was never adopted wholeheartedly by ruby-core.


Now that the main maintainer quit, can those nontechnical reasons be resolved?


Hopefully. I would be willing to take a crack at it.


If you (or anyone else) do, I'd be ready to help.


> It is de facto a more complete test suite than the ad hoc one they've been making.

This is a _huge_ stretch. It's hardly surprising that the suite of tests MRI is running against aren't failing against a _released_ version.


> Why is RubySpec the special sauce that is so special that the devs not using it is horrible?

Assuming TFA isn't actively lying[0]: because it fixes the issues outlined which allow (amongst other things) cross-implementation and cross-version uses, because it tests behaviours not tested by the MRI suite anyway and because it's there as an extensive test suite for the language.

Oh, and because according to TFA it found regressions in just about every release it's existed for. Regressions which could thus have been caught before the release was cut.

[0] and I don't see any reason to assume otherwise, quite the opposite.


I wouldn't say TFA is lying but it is presenting only part of the truth. MRI does have an actively developed test suite, which they run in CI and which we use to test compatibility in JRuby. MRI has contributed to RubySpec in the past and many contributors I know stopped (like I did) for nontechnical reasons. They are not ignoring it, but it is a hard project to collaborate on given the attitudes of the project lead that should be evident in this post.


Because RubySpec is the best specification for Ruby that exists. By far.


I see what you're saying, but this isn't about using any one person's pet project. This discussion would be moot if MRI Devs were using a different competing third-party Ruby-Spec like tool, but they aren't. They wrote their own tools which contain documented bugs and flaws which they are not interested in fixing, all the while a much better tool exists that they refuse to use.


That's basically how I read this. It seems that Ruby is specified de facto by what MRI does.

Reading this blog post it sounds like the developer of RubySpec did this:

1. Decided Ruby needed more formal specification 2. Made their own formal specification 3. Kept telling the core developers to use it 4. Is giving up after years of the core developers not bending to the way he thinks that Ruby should work

The whole thing just read as kind of arrogant to me. Why should the ruby developers have to follow his specification? The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.

Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. If some of the existing tests or two large or don't test only one function, why not submit a patch to fix those issues. That doesn't mean the entire test week needs to be thrown out and the language respecified.

This seemed more like someone trying to force themselves into a position and giving up after a few years.


> 1. Decided Ruby needed more formal specification 2. Made their own formal specification

RubySpec isn't a formal specification in any way, shape or form. RubySpec is an implementation test suite, and for MRI a regression suite. It uses language constructs and expects a behaviour A based on Ruby version B (which is itself assumed to match MRI version B).

RubySpec isn't somebody thinking up his own definition of Ruby (let alone doing so formally), it's somebody encoding MRI behaviour into a test suite because he needed that to code his own Ruby implementation.

> Why should the ruby developers have to follow his specification?

There's nothing to follow, there's a test suite to run. A test suite which encodes existing MRI behaviour. Why should ruby developers run it? Because it catches bugs is a pretty good reason.

> The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.

How is segfaulting on existing code not an indictment of the core process when just running an existing test suite would have caught it?

> Instead of saying "you should be using my test", you could commit mew tests and patches to upstream.

Have you missed the laundry sheet of issues in the existing MRI testing system?


The problem with "MRI as spec" is that nobody really knows how the language is defined.

E.g. large parts of the 1.8.x series behaved differently from what most people thought it did in terms of the bootstrapping of inheritance of core classes. I wrote a blog post about Ruby behaviour based on introspecting the 1.8.6 interpreter that some people insisted was wrong because it worked differently than intended. Yet the behaviour persisted for years until it silently changed at some point (I don't know which revision).

In that case it was no big deal since the reason the real behaviour was pretty much unknown was that nobody depended on the behaviour, but from what I've seen this is fairly common with MRI.

As a "spec" it is one of the most unstable environments I've worked with. Though I love Ruby the language, MRI is a problem as the default implementation and atrocious as a "spec".

I frankly hope one of the other implementations gains enough traction to force the core team to the "negotiating table" over a proper specification.


RubySpec is nothing more than Ruby code doing things that worked in the previous version. Whenever it breaks, that is a regression in MRI.

Arguably it's no different than something breaking in your current Ruby program, and after spending a bunch of time debugging you realize it is a problem in MRI. If that happens, you would file a bug against MRI. RubySpec just makes it more apparent.


Fuck, this idiot is now being pendatic in #ruby as well.

2015-01-02 14:45:30 brixen raise your hand if you have implemented Ruby and used MRI's tests

Glad he didn't get things to go his way. He cries too much. If you don't like some projects go write your fucking own.



I mistakenly cut off the full title of the post: "Matz's Ruby Developers Don't Use RubySpec and It's Hurting Ruby"


Ruby is about to die permanently. A language that copied emacs-lisp but with syntax? Lol that gives it features to impress the ignoramus majority but real Haskell or Lisp programmers know better.




Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: