Hacker News new | comments | show | ask | jobs | submit login
How Ruby Borrowed a Decades Old Idea From Lisp (patshaughnessy.net)
57 points by pat_shaughnessy 1623 days ago | hide | past | web | 62 comments | favorite



Sure, lambdas are not new, but there's still some interesting (maybe not worth calling them novel) ideas in Ruby's blocks:

First of all the observation that most method only care about a single block. By having custom syntax for "method that takes a single block" you get a certain simplicity that makes methods look like built-in keywords:

    loop { puts "Nuts" }
It does make multi-block methods look ugly, but frankly, I think they look ugly in any language.

Lisp has macros (which has both it's advantages and disadvantages), but I find that Ruby's blocks make the lack of macros more tolerable.

Compared to JavaScript/Perl/Python-lambas, Ruby's blocks don't introduce another scope. "self" and "return" still refer to the method's scope (I'm not sure what this type of scope is called; it's not the variable scope I'm talking about here). By introducing another set of keywords that work on the block ("next" and "break") you can easily wrap code in blocks without changing the semantics. See http://yehudakatz.com/2012/01/10/javascript-needs-blocks/ for more information about the difference between blocks and lambdas in object-oriented languages.


> First of all the observation that most method only care about a single block.

Except when they don't, but if you can't build multi-block methods (or methods on blocks) because it looks like shit, you don't think about them. It's a form of blub, nothing more. There are numerous multi-block methods, Ruby's syntactic limitations just makes them unwieldy and thus avoided. Just as Python's or Java's syntactic limitations make higher-order methods unwieldy and thus avoided.

> By having custom syntax for "method that takes a single block" you get a certain simplicity that makes methods look like built-in keywords

Smalltalk stands in stark disproval of this assertion.

> It does make multi-block methods look ugly, but frankly, I think they look ugly in any language.

Eh?

    aCondition ifTrue: [ "true block" ] ifFalse: [ "false block" ]

    [ "some code" ]
        ensure: [ "recovery code" ]

    [ someObject aMessage ]
        on: aSignal
        do: [:sig | "action" ]
I don't think that looks ugly.

> Compared to JavaScript/Perl/Python-lambas, Ruby's blocks don't introduce another scope.

They do, if you define a variable within the block it'll be local to the block. Ruby 1.9 even allowed shadowing variables from the parent method (which isn't possible in 1.8)

> "self" [...] still refer to the method's scope

So does it with the vast majority of languages. Certainly does in Python. Javascript behaves strangely with its somewhat-dynamically-scoped-but-not-really `this` but it's a peculiarity of JS, not Ruby.

> and "return" still refer to the method's scope (I'm not sure what this type of scope is called)

It's a non-local return, not really a scope.


> Smalltalk stands in stark disproval of this assertion.

I stand corrected. Although Smalltalk solves it by introducing a special method call syntax (which is perfectly fine). I was mostly thinking about C-style-method-call-syntax when I wrote this (Perl, Python, JS).

> They do, if you define a variable within the block it'll be local to the block. Ruby 1.9 even allowed shadowing variables from the parent method (which isn't possible in 1.8)

I did mention explicitly that I wasn't talking about variable scope, so yeah, we're both correct :)


> Although Smalltalk solves it by introducing a special method call syntax (which is perfectly fine).

Meh. Smalltalk comes from a time when not all languages had to be ALGOL-style, so it just has its own syntax.

> I was mostly thinking about C-style-method-call-syntax when I wrote this (Perl, Python, JS).

That is no issue, if you have first-class blocks. At most you'll have commas between your blocks because they'll be arguments like every other. Here's a python-ish version using chevrons for blocks:

    error = (request.form['username'] != app.config['USERNAME']).then(⟨
        "Invalid Username"
    ⟩, else=⟨
        None
    ⟩)


"Dynamic extent" might be close to what you're talking about. That's what Common Lisp calls the hint that an object may be stack allocated because it's unused after the call.


This is an excellent point — that Ruby embodies the syntactic insight that most functions that take any functional argument take only one functional argument.

Frankly this turned me off from Ruby when Matz presented it at LL2 in 2002. My reaction was “this is an ugly hack that won't work with more than one funarg, by someone who doesn't really understand functional programming”. [I was coming from years of hardcore programming in Smalltalk and Common Lisp.] It was only years later that I realized this special case was so overwhelmingly the common case that it was reasonable to give it a nice syntax, and that there's a (big) place for programming that isn't in a “functional” style, but still makes liberally use of funargs. (Most Smalltalk and Lisp that I've seen occupies this middle ground too.)

It reminds me of my first reaction to Python: “Meaningful indentation is a terrible idea!” Only a year or two later when I was writing Lisp pseudocode in a notebook [late '90's — no parenthesis highlighting or syntax-directed editor on my analog notepad] did I realize that what I was sketching used whitespace as shorthand, and was practically Python as it stood.

That Ruby blocks have a weird scope, and that the syntax for creating and invoking closures outside of the block syntax is so cluttered, are unfortunate and seem mostly independent of having a compact notation for single-funarg function calls. Except that if, like Lisp and Smalltalk, all use of closures used the same syntax, this syntax would probably have become more polished.


I've always thought the func block list syntax in Perl [1] also had a strong influence on Ruby's blocks. Here's an example of map in both languages:

  # ruby
  double = [1..10].map { |x| x * 2 }

  # perl
  my @double = map { $_ * 2 } 1..10;
NB. For more examples of the func block list syntax in Perl see grep and List::Util / List::MoreUtils modules.

And it's easy create new ones as well:

  sub loop (&) {
    my $block = shift;
    $block->() while 1;
  }

  loop { say "Nuts" };
And you can also chain them together to build more complex constructs:

  sub ifTrue (&@) {
    my ($block, $cond) = @_;
    $block->() if $cond;
    $cond;
  }

  sub ifFalse (&@) {
    my ($block, $cond) = @_;
    $block->() unless $cond;
    $cond;
  }

  ifTrue { say "True" } ifFalse { say "False" } 1 == 1;
I once wrote a lazy functional syntax in Perl using this approach :)

[1] - Not sure what the proper name for this syntax is!


Blocks, and the way they make it very easy to add new control structures "seamlessly", are indeed cool—but aren't ruby's blocks taken pretty much verbatim from smalltalk (which of course is also many decades old, if not quite as old as lisp)?

Indeed, smalltalk seems a better example, as it uses methods and blocks to implement all control structures, even the "built-in" ones.


Probably. In terms of features, Ruby is pretty much an unholy mix of Smalltalk, Perl, Awk with a dash of Lisp thrown in and a parser that's a monstrosity of epic proportions put on top (the MRI 1.8.x series had a Bison parser with 6k-7k lines of C to implement context-sensitive tweaks...) to make the whole thing have a syntax palatable for those of us who'd otherwise complain loudly about square brackets or parentheses...

And I write this as someone who loves Ruby - there's much to love about Ruby, but a clean design is not it.


Lol - very funny. I have to agree you; as a Rubyist studying MRI internals there's a lot of very confusing code in there. And FYI Ruby 1.9/2.0 retains the confusing Bison grammar file.

One of my goals with the eBook is to allow readers to get an understanding of how Ruby works without having to struggle through the confusing implementation themselves.


But do you agree that Ruby is an unholy mix? I would say it is a holy mix, Matz targeting programmer happiness, principle of least surprise etc.

Speaking of internals, do you know enough internals of Smalltalk, python, Perl implementations to judge, Ruby is confusing compared to them? How would you compare them to Ruby implementions MRI and 1.9/2?


No - not the "unholy" part. Like vidarh I'm someone who loves Ruby (why else would I write a book about it!) but I do agree with him about the internal implementation: it's very confusing and hard to understand. At times, this leaks out into the syntax of an otherwise beautiful language. JRuby and especially Rubinius are great alternatives to MRI if you're interested in studying internals - they're much cleaner and easier to understand.

Sorry I don't know anything about the internals of Smalltalk, Python or Perl and can't comment on them.


> aren't ruby's blocks taken pretty much verbatim from smalltalk

Yes, with some crippling to fit them in a different syntax (Ruby's blocks are not first-class, so using more than one of them or methods on the blocks themselves isn't very nice)

And amongst the important similarities are return being nonlocal from within blocks.


I admittedly don't know a lot about Smalltalk - I was aware Ruby's (and every other language's) object oriented ideas were invented originally with Smalltalk, but I didn't know it had a block/closure syntax also.

In this post I wanted to go back as far in time as I could :)


> In this post I wanted to go back as far in time as I could :)

Then you might want to look at Simula as well. As much as the term "object oriented" was invented in conjunction with Smalltalk, a lot of the basic concepts (classes, objects...) came from Simula (but not closures).


> but I didn't know it had a block/closure syntax also

Blocks are one of the most important concepts of Smallalk. Every control structure is done via blocks in Smalltalk, even if/else is a method (message) on a boolean object, that takes two blocks as arguments: one for the 'true' case and one for the 'false' case.


You should check out Smalltalk blocks. They are much closer to what Ruby has than Lisp-style closures are.


Tcl uses blocks for pretty much everything, and they're not that bad looking:

    if { $something } {
        do this
    } else {
        do something else
    }
The first one is actually an expression, but the do this and do something else are simply blocks of code passed to the 'if' command.


Its beauty (and power) comes with the consistency of the syntax ala Smalltalk / Lisp.

PS. Here's an example of creating something similar with Perl:

    sub IF (&@) {
        my ($block, $flow) = @_;
        [ map { $flow->{$_} || sub {} } qw/else then/ ]->[ $block->() ]->();
    }

    sub THEN (&@) {
        my $block = shift;
        my $flow  = shift || {};
        $flow->{then} = $block;
        $flow;
    }

    sub ELSE (&) {
        my $block = shift;
        +{ else => $block };
    }

    IF { 1 == 0 } THEN { say "True" } ELSE { say "False" };
And with these being functions and not statements we can also do:

    my $result = IF { 1 == 1 } THEN { "True" } ELSE { "False" };


Well, but Tcl blocks are nothing more than strings that you pass as arguments that then get eval'ed using uplevel. Ruby blocks are "real" language constructs, and are real closures, in difference to Tcl blocks.


I can assure you that Tcl blocks are quite "real" in that they are code that gets executed. Initially, yes, they are treated as a string, but then they get byte compiled, and remain 'code' internally.


Compared to (...)Python-lambas, Ruby's blocks don't introduce another scope. "self" and "return" still refer to the method's scope

'self' in Python still refers to the method's scope too (if the lambda is inside one).

  class A(object):
    def b(self):
      self.name = "John"
      c = lambda: "Hello, %s" % self.name
      print c()
  A().b() #prints "Hello, John"


Strictly speaking that's merely a convention.

Python simply has explicit method receivers.

There's nothing pretty you from calling it `obj` or `foobaz` or whatever - it's just a normal variable.


> It does make multi-block methods look ugly, but frankly, I think they look ugly in any language

Tcl... builtin 'for'

    for {set i 0) {$i < 10} {incr i} {
        puts $i
    }
user proc 'myfor'

    myfor {set i 0) {$i < 10} {incr i} {
        puts $i
    }
In some languages, they look exactly the same.


For DSL/metaprogramming fun (and occasional hair-pulling, heh), Ruby also gives you instance_eval and its siblings/cousins for running a block with "self" bound to a different object. I've mainly used it for providing the ability to run a block in the scope of a certain object without the author of the block having to reference that object at all (so field names can be referred to without any prefix, which gives a nice clean syntax).


Yup - I'm planning to cover that later in the same chapter in my eBook. Your description is perfect: it's a closure with the "self" redirected to a different object (the receiver of instance_eval).


Thanks for the link to Yehuda's post - it was a great read. I love how he compared Javascript and Ruby, and it's intriguing what he said at the bottom about how JS might include "block lambdas" similar to Ruby in the future. Fascinating!


> older languages such as C or Java.

Ruby and Java saw their first public releases within a month of each other.


Yes you're right! I stand corrected; will update the post when I have a chance.


"It's [2012], and programming languages have almost caught up with 1958." -- paraphrased from, oh, you know, some guy :)


DOn't imagine the reasons languages are different from List is that nobody knew any better.

Programming in Lisp is like programming in assembler. Reading it is worse. The learning curve is tremendous.

Languages serve lots of demographics, not just comp-sci wonks.


When did the shift that makes this okay happen? Once upon a time, people who knew a lot about computer science were respected and praised; people tried to emulate their algorithms, coding style and planning.

Now, they are "wonks" for knowing a slightly different (and demonstrably effective) style of programming.


>When did the shift that makes this okay happen?

Now that there are millions of programmers. Programming shifted from a fine art to a pop art, to crib something from Alan Kay.


Hey! No offense meant. Its just true that a lot of programmers are not CS-trained. So not the target demographic of Lisp.


I would contend that not being CS-trained doesn't eliminate them from the pool of people who could benefit from knowing Lisp and similar languages. As a programmer, unless you just want to be outclassed in a few years, you have to stay on top of developments in the industry and the history of the industry. Knowing the languages that modern languages pull their tricks from is one such way. And it doesn't take a CS degree to do that with many of the resources being put out these days.


Same argument for any hot language. Who keeps up on them all? Excuse my french, "CS wonks"

The rest of the crowd is trying to ship code, in the toolset they have some facility with. That toolset has to be available, easily digestible, hopefully targeted at their problem space.


If someone is a biology researcher who does some programming for their research, does that programmer need to stay on top of developments in the software industry? Or know the history of the programming field ... as well as keep up with the biology field?


Well I guess that depends on what you're doing within you're research.

In bioinformatics knowing about, say, functional programming and parallel algorithms could be really useful where data coming out of sequencers is growing faster than Moore's law can keep up, if you want to scale across many cores as efficiently as possible.

Or graph theory, or... any number of useful techniques waiting for an application.


Certainly. But most aren't so inclined. Most need a way to glue together existing components. For example, "make a web connection to X, call command-line program Y, and use a shared-library extension to do Z. Plot in Excel." The result might be very useful scientifically, and it's definitely programming, but there's no real reason to learn about lambdas, s-expressions, and monads.


You would be surprised how often non-CS researchers doing some programming use languages that are essentially Lisp with weird syntax (R, Mathematica...). Which I think exactly demonstrates that you don't need any knowledge of history or modern developments in computing to benefit from Lisp.


You do not need to be a CS major to appreciate Lisp.

I've had a number of fun conversations discussing and demonstrating Scheme with novice and 'rank-and-file Java/C#' types. The ones who find programming languages fun in themselves are the sort of people who light right up when they're introduced to something like Scheme. That's really all it takes is just a bit of passion and curiosity to make the leap.

In my experience, professional folks I've spoken to have either never heard of Lisp or have associated it with antiquity. You'd be very surprised who'd appreciate being shown new ways of doing things.


I don't think the target demographic of Lisp is "CS-trained" people. Do you? Why? Because functional programming, which is actually a lot more like the math everyone else learns, is somehow for "wonks?"

When you say things like this, part of me hears you saying, "I have not invested time learning this skillset. However, people argue I should. So I have decided said people are 'wonks,' and therefore I can safely dismiss them." Basically anyone with any differentiation from your skillset falls into that definition.


this makes no sense. on a purely personal, self-development level, you're talking yourself out of opportunities for no gain.

there's nothing magical about lisp compared to other languages. to read it you don't need a cs degree and you don't need vast levels of intelligence - you just need practice. after a while it becomes quite clear.

you're free to think whatever you want, of course, but this kind of negative attitude - "it's for the smart people, not me" - just means that you lose out.

lisp itself is not that far from a language like python (or, i guess, in the context of this thread, ruby) once you get used to it. it's really not rocket science. all you're doing here is harming yourself.


No, I don't reject it. I'm simply making the point that, still, some people (the majority of people) are not up to Lisp. Most folks never read the manual, just hop on board and start coding.

So, thus, the (Very) slow acceptance of Lisp.


"Practical Common Lisp" is a good introduction to the power and readability of non-wonk Lisp.


Agreed. It makes CL look way easier than assembly :D

edit: not that I consider it to be like assembly, really.


  >> The learning curve is tremendous.
Try "Land of Lisp" http://www.amazon.com/Land-Lisp-Learn-Program-Game/dp/159327...


I've been thinking about block implementation off and on for a while (hobbyist language designer here) and I'd like to try implementing them so that they are never values in the language semantics.

That way, a class of errors becomes impossible; namely, those errors that involve returning from a block whose method has already returned. I would have closures in the language but they would be just like Scheme closures: they would be independent of their creation context except for lexical bindings. I would compile blocks right into the method in which they appear.

Unfortunately, I don't have experience with any language that uses blocks. I'm quite familiar with closures but not blocks. I'm wondering if any Ruby or Smalltalk (or ...) programmer can give some reasons why blocks ought to be values.

Also, (referring to the post) does anyone know why rb_block_t was not embedded in rb_control_frame_t? That's the usual idiom in C and I can't think of a reason not to use it.


  In fact, the computer science concept behind blocks, called “closures”
As far as I know, the language-free concept of "block" is orthogonal to the concept of scope. That is, a block is not by necessity a closure.


You might be right about the general meaning of the term "block." However, in Ruby blocks happen to be implemented as closures. This was one of the main points I was trying to make... that blocks aren't as simple as they seem, at least in Ruby.


That's a totally legitimate point. I'm just pointing out that the quoted bit would lead the reader to believe that a block is defined as a closure.

The problem is almost everyone I talk to in the startup scene thinks that an anonymous function/block == a closure.


> The problem is almost everyone I talk to in the startup scene thinks that an anonymous function/block == a closure.

This is a big pet peeve of mine - I keep seeing people use 'closure' as a synonym for 'lambda'. It's a simple error, but it bugs the hell out of me.

Then again, it's a dead giveaway that the person has never programmed in a functional paradigm, which may be a good thing to know, depending on the context.


I'd be interested if the author had any thoughts on the ruby_parser project's unresolved issues of S-expression use in ruby 1.9, as at http://blog.zenspider.com/blog/2012/08/what-are-block-args-.... (not my project, but something I'd like to use once they've solved all this - I know the author of the above link was looking for input).


I'll be sure to take a look at that, and thanks for the link. No, no thoughts on it right now.


Excellent article, I really enjoy reading about the nuts and bolts of languages.


Thank you!


i thought old (original) lisp was dynamically scoped?


Yup--I believe it was Scheme that introduced lexical scoping to Lisp, with Common Lisp adopting it later on. Of the commonly used Lisps today, only Emacs Lisp is dynamically scoped.


Of the commonly used Lisps today, only Emacs Lisp is dynamically scoped.

That's not quite accurate. Common Lisp and Clojure are both lexically and dynamically scoped. In CL, things created with def* (defun, defparameter, defvar, etc...) are dynamic, while function arguments and things created with let are lexical[0]. In recent versions of Clojure, it's necessary to explicitly make a symbol dynamic with metadata.

[0] This is probably not entirely correct or complete, but should get across the general idea.


so the article is misleading, isn't it? it says "Lisp programming language, invented in 1958 by John McCarthy. Lisp pioneered many fundamental computer science concepts, including closures", but closures were first created elsewhere - see http://en.wikipedia.org/wiki/Closure_(computer_science)#Hist...


Oops - I missed that in my research. Yes, I agree this sentence is misleading; I'll update the text later on today. I suppose it's only true that Lisp (specifically Scheme) was the first widely used implementation of closures.


yes, i think that would be a better way of phrasing it. it's weird how old scheme is (1975 - 37 years old).




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: