Hacker News new | comments | show | ask | jobs | submit login
IcedCoffeeScript (maxtaco.github.com)
288 points by swannodette on Jan 28, 2012 | hide | past | web | favorite | 58 comments

This fork brings some up deep issues, perhaps intractable, around about the development of syntax-y languages. Useful source transformations require marginalized forks of the compiler and all the development burden and risks that entails. In some sense CoffeeScript inherits the actual problem with JavaScript - a select few determine the introduction of language features. Perhaps these language features are best shipped as libraries?

Maybe it doesn't matter. Perhaps most users don't need it. Or perhaps as we continue to develop software we'll find that we'd rather defer work to the compiler / macros and the high cost of syntax is simply not worth it.

I'm looking forward to see how 3 languages I immensely enjoy (JavaScript, CoffeeScript, ClojureScript) co-evolve :)

Regarding development burden, if ICS is implemented as additions to the language, then it can get all updates to the rest of CoffeeScript from "underneath" - the only proviso is that CoffeeScript syntax doesn't change incompatibly with it. And as ICS can be seen as a simple macro (plus libraries it uses), I would guess it is implemented this way (simple source transform on top of CoffeeScript, not hacking at the Jison grammar). This is one way around the issue you raise.

Regarding the whether it's part of the language or a library as a distribution issue, it depends on what benefits users. For syntax, there's some benefit in uniformity (other developers can read it). I'm seeing ICS as a suggestion of a new feature in CoffeeScript. If it's really good (meets a need; bugs and design mistakes are fixable and fixed; it doesn't break other things), it could be back-integrated into CoffeeScript proper. Whether to do so can be better decided once data exists. :-)

It's a great forking model for exploring new syntax, IMHO.

EDIT but it breaks the CoffeeScript philosophy of readable JavaScript, and (eg) addes debug line numbers in to the source code (see "load" on the sample code). Also, other examples and more details on ICS here https://github.com/maxtaco/coffee-script/blob/iced/iced.md

> the actual problem with JavaScript - a select few determine the introduction of language features.

What language exists in which this is not the case? Language by committee sounds terrible.

> What language exists in which this is not the case?

Languages which have very little "own shape" (syntax) and let you transform it easily (via macros) or provide high flexibility.

Mostly concatenative languages and Lisps, but e.g. Smalltalk or IO can probably be fit in that category as well, they have a minuscule syntactic core and that core is used to build abstractions and structures putting both the language's designers and the language's users on a level ground.

Of course that creates other issues, where every developer or organization has its own lingo and structures, making even going from one codebase to the next more expensive. It's a tradeoff.

Still, realizing that you have pretty much the same power as the language builders themselves when it comes to creating abstractions and datastructures and control flows... is an enjoyable feeling.

> Language by committee sounds terrible.

Can end horribly, and can end pretty well. Haskell was designed by a committee over ~10 years (FPCA '87 to the release of "The Haskell 98 Report" in 1997).

It does generally end baldy.

ClojureScript (which swannodette works on). You can invent your own language features in a Lisp.

Haskell was designed by committee.

I'm not talking about language design by committee. For example, ClojureScript is far more restrictive in its design than either JavaScript or CoffeeScript. I'm simply questioning where the line is drawn concerning what constitutes language extension and what is perhaps best served as a library.

IcedCoffeeScript works through a CPS transformation, not sure how you could do that as a library in CoffeeScript...?

I was going to complain that I could find no mention of TameJS (from which the await and defer keywords/features clearly originated). Then I noticed that the IcedCoffeeScript author is also the author of TameJS.

It actually used to be called Tame, not Iced:


me too. but you'ld think that it would help to at least say somewhere (like a subtitle in the banner) "tame + coffeescript"

I like the generality of the defer/await concepts of TameJs (and by extension, iced coffee). But I was curious if these concepts could be used without code generation, as in Async.js. So I came up with Queue.js:


I haven't tried it in production code yet, but it looks like a convenient abstraction with tunable parallelism.

Another notable feature is the generation of readable stacktraces in iced, as explained here:

https://github.com/maxtaco/coffee-script/blob/iced/iced.md (search for: "Debugging and Stack Traces")

Why didn't the "iced" feature just get added to CoffeeScript? Why have two separate packages?

This is one of the great things about doing language work on GitHub -- folks can start experimenting in their own directions, and some of it will make it back into the original project (ES5 strict mode early warnings: https://github.com/jashkenas/coffee-script/issues/1547) ... and some of it doesn't: https://github.com/satyr/coco

There's currently an open pull request for Iced to be merged with CoffeeScript, and the merge will happen if the fork manages to:

* Provide a seamless veneer over sync patterns in async code: defer in loops, with try/catch, within the middle of an expression...

* Handle both synchronous and parallel style "maps"...

* Prove to be more pleasant / powerful to use than callbacks and promises, in practice.

* And then the hardest bit -- do it all without breaking CoffeeScript's golden rule: it has to compile into straightforward JavaScript -- i.e., the callbacks you would have written in the first place.

It's a tall order, but if Max manages to pull all that off, it'll be pretty great.

Hi Jeremy,

streamline.js already fits the bill, including your golden rule: the core idea was precisely to generate the callbacks that the developer would write otherwise.

Today it works with both CoffeeScript and Javascript, but in a decoupled way: the streamline transformation is applied to the JS generated by the CS compiler. It could easily be repackaged as a CS language extension.

It has been around for more than a year and the CS+streamline combination is powering at least one live site: http://www.thethingdom.com/


This is a fork of the language, not an official feature. It’s as if you wrote your own version of JavaScript that trampolines function calls and performs tail-call optimization. You still need to go through the social process to get your ideas into the main “trunk.” Selling people on adopting your fork might be one way to get everyone excited about the idea.

That being said, right up front he explains the two reasons why this is a departure from CoffeeScript’s core philosophy: `await` and `defer` are changes in the semantics of JavaScript, and not surprisingly, code written in IcedCoffeeScript no longer compiles into more-or-less recognizable JavaScript.

Modulo syntax and sugar, CoffeeScript is JavaScript with some minor changes to the AST (such as soaking up nils with the Elvis operator). IcedCoffeeScript is another language that evolved from JavaScript and transpiles to JavaScript.

Before adding such features willy-nilly to CoffeeScript, folks need to be comfortable walking away from a core value. I’m not saying that’s a bad thing, but it’s a problem related to the Innovator’s Dilemma: To embrace this new value, CoffeeScript will have to alienate some portion of its user base that value compiled JavaScript that is easily recognizable to anyone who has read the CoffeeScript source.

As a user, you don't really need to walk away from anything. There isn't any difference between Iced CoffeeScript output that doesn't use await and vanilla CoffeeScript output. You could literally switch out somebody's name-brand CoffeeScript for Iced and he'd never know the difference unless he had a function called await. It's purely an ideological concern, not a practical one.

You’re preaching to the choir!

But one objection we will both encounter repeatedly is that if a language supports feature X, and you work on a team/inherit code/use libraries, there will always be people who don’t want to use feature X but wind up dealing with it any ways because someone else used it.

So the naysayers will claim that if it’s added to the core language, it’s only a matter of time before they wind up dealing with it whether they like it or not. Given the way that most Ruby users have to live with monkey-patching in Ruby whether they do it themselves or not, I fully expect that if await and defer are added to core CS, there will be libraries or CommonJS modules that use it and presto, you may find yourself looking at some of its output in the debugger one day.

I’d rather use the feature myself and benefit from it, but I can accept what the luddites are saying even if I don’t end up coming to the same conclusion about whether the feature should be added or withheld :-)

UPDATE: I recall being told that tail-call optimization should not be added to certain languages because it encourages programmers to write recursive functions that are “hard to read.” You might argue that if you never write recursive functions, why should you care? But people do care.

Maybe. Tail recursion optimizes itero-recursive algorithms, turning them into iterative algorithms by eliminating the overhead.

Confused? Here are the iterative, recursive, and itero-recursive ways of writing the factorial function:

    function fact_r(N) {
        return N <= 0 ? 1 : N * fact_r(N - 1);
    function fact_i(N) {
        var acc = 1, i;
        for (i = 1; i <= N; i += 1) {
            acc *= i;
        return acc;
    function fact_ir(N, acc) {
        acc = acc || 1; // default value of acc
        return N <= 0 ? acc : fact_ir(N - 1, N * acc);
So the accumulator variable becomes an argument variable which may have to be separately initialized. The key feature which makes something optimized for "tail calls" is that when f(something) wants to recurse, it returns f(something else). In other words, certain conditional logics can happen, but don't leave an operation like the "n " in "n recurse(n - 1)" on the stack or else you can't get rid of the function call. It's not even a special optimization -- basically you can move one line in the compiler a couple steps ahead of where you might have automatically written it, and it suddenly treats iteration and tail-recursion the same.

So that's the difference. If you can understand fact_ir, which does the iterative idea in a recursive syntax, then you can understand tail recursion.

> code written in IcedCoffeeScript no longer compiles into more-or-less recognizable JavaScript

So this is a more fundamental fork than first appears... And it explains why it includes debugging information (loading up the code examples reveals artifacts like lineno: 6 in the output JavaScript).

I recall a debate about debug info, turning on the values you note. Can't find it right now. (though, apparently extra-linguistic Source Maps are being introduced into browsers http://news.ycombinator.com/item?id=2862629)

Personally, I think "clean JavaScript" is the right adoption strategy, for now, in a JavaScript-dominated world. When CoffeeScript has native browser support, that value can be dropped (it might not happen: MS supports VB, Google supports Dart... even if Mozilla supports CS, the others mightn't).

Finally, I don't think the resulting JavaScript is that hard to understand - you can easily see the correspondences with the source. However, it's the thin-edge of the wedge. What changes will be added next?

I haven't looked at this implementation specifically, but the issue with many similar proposals/implementations is that the generated JavaScript is ugly. One of the nice things about CoffeeScript is that there's no magic--it's very easy to figure out what the generated JavaScript is doing, and there's a straightforward CS <-> JS mapping. jashkenas (CoffeeScript's author) has been reluctant to change that.

IIRC, Jeremy Ashkenas actually brought this up in the pull request discussion, but he sounded pretty impressed by the minimalism of the code ICS generates. There seems to be more ideology-based opposition among the community.

I did some simple tests, and frankly the generated code is ugly, certainly by comparison to vanilla CS. Gone is the "oh I see how it did that, it's just a Javascript version of my Coffee!" It's now more like what C++ used to look like when compiled to C (remember cfront? probably not. Be glad you don't).

As someone else said, if you avoid await/defer entirely, it seems to back off to exactly what Coffeescript does, so there is a nice progression path.

code: https://github.com/danx0r/testiced#readme

Upon skimming the pull request[1], it looks like this will eventually make it into CoffeeScript.

[1]: https://github.com/jashkenas/coffee-script/pull/1942

Being able to syntactically assemble several parallel requests together has an obvious advantage in making code more readable. Having a hard time wrapping my mind around the keywords. From the examples I think more wait/collect than await/defer. For instance:

Await these items that I want to defer.

Wait on this code block and collect these items.

It's fantastic to see so much enthusiasm and innovation happening around JavaScript.

While I'd love to see some of these features find their way back into the core language (or native support for coffeescript in node or browsers) it's fair to say that's a long time off if it's ever going to happen.

In the meantime, I'm getting to massively clean up the logic and syntax of my apps. Huge thanks to everyone working on this!

Harmony will include generators, which are basically coroutines.

The way that this manages to transform those examples makes me think that it would make all kinds of other things more palatable than the vanilla coffee script. I've written chains of callbacks in plain JS that this would make so much nicer to work with.

Can't agree more. There is no other language in the world that needs this as badly as JS does.

- I am hoping that this would be merged back if it could be somehow concluded that there isn't a much cleaner way to achieve the same result. Even if the resulting JS isn't as clean as what CS produces now. The ICS approach looks safe; they could be assured by similar approaches in more mainstream languages, most prominently C# (and F#) which have much bigger teams attacking the same problems.

If the prospect of this getting merged back looks bleak, I might consider switching ICS once I am convinced there aren't any other breaking issues.

So was there more discussion about await and defer in the many github issues on the subject? This output seems more debuggable than previous CScript defer attempts.

Here's the main one for Iced: https://github.com/jashkenas/coffee-script/pull/1942

I remember there was another discussion before where they first started talking about Max's implementation, but I can't seem to find it at the moment. That did seem to be the general consensus, though.

Why is defer needed? Why can't I follow C# and do:

   result = await search("test")
Is this just so that it plays nicely with non-standard callback APIs? I'd prefer to introduce Promises (i.e., q) and have await interact with that. I guess this is not the NodeJS way though.

search("test") is not an asynchronous call, in your example, because there's no callback function. (In JavaScript)

defer() is needed because it becomes the placeholder for the callback function, and lets Iced know how many functions it needs to wait to call back.

In other words, JavaScript doesn't return a Promise/Future/Task<T>, so the defer syntax is necessary. You could get rid of it by simply capturing the callback params and returning them as a value, though your code would end up having a lot of `[0]`'s in them, since most callbacks are of the form (thing, err)

When I made this comment the other day, I wasn't actually serious that someone should build a JavaScript compiler that does CPS:

"CPS is also generally used as an IR in compilers, not something you write by hand"


This reminds me of my Celluloid::IO system in Ruby (which uses coroutines to provide a synchronous API), except Celluloid::IO lets you have as many event loops as you want:


When a language has dialects, you kno you are doing something right... go coffee-script!

For another cool CoffeeScript "dialect", see Contracts:


It's a great way to write defensive code for those of us who find writing regular unit tests a little onerous even (if they are often necessary). I'm planning on using it in all my CoffeeScript, up to -- but not including -- generated production Javascript.

If you're only using CoffeeScript on the server, then you should check out node-fibers to achieve the same effect.

Here's an example of using my Common Node library (that implements a number of sync APIs using fibers - http://olegp.github.com/common-node/) with CoffeeScript: https://gist.github.com/1447709

This is certainly a development to applaud, but for me, the most important feature about async JS programming is to be able to catch errors in the asynchronous code, and do it without code clutter. So I hope that if this gets accepted to mainline CoffeScript, it will provide for a working try/catch around the await/defer, along with caller+callee stacktraces capturing (at least in debug mode). At that point, it will be hard for me to not justify the use of one compilation layer above JavaScript, which currently certainly is, and even so with await/defer.

Asynchronous error handling without clutter is why I created LAEH, have a look if you want: https://github.com/ypocat/laeh2

EDIT: To clarify - an uncaught error in async handler means your Node.js is going down (unless you use the catch-all handler, which is an even poorer practice). In browser it means your callback never returns, leaving you in void (with a short an ambiguous stacktrace) to search where the problem is.

Nice work, I love the syntax. Was a bit disappointed at the verbosity of the compiled js though. Wondering if the same thing could be achieved without the secret functions, classes & variables? (Deferrals, findDeferral, __iced*...)

I think I can imagine a more direct translation of await & defer, though of course I haven't actually tried to implement it (yet).

> The function defer returns a callback, and a callee in an await block can fulfill a deferral by simply calling the callback it was given.

I'm wanting in CoffeeScript knowledge and/or intelligence, so could someone explain the above please? (In the examples, defer seems to specify the variable that gets the result of await.)

I tried to formulate very differently so you can see that as another point of view. Sorry for the lack of formality.

defer is a function which returns another function. When that returned function is called, we get out of the await block.

So, to take the same example:

    for k,i in keywords
      search k, defer out[i]
Here, defer(out[i]) returns a function. We pass this function to search.

Search is like this:

  search = (keyword, callback) ->
    # do some search
And here, by calling callback, which is the function returned by defer, it will end the await block.

Here's a trivial example:

    some_function = (fn) -> fn(1234)

defer(x) returns a callback, it's given to some_function. And some_function calls it, which ends the await block.

Thank you for your thoughtful answer, it's clarified some issues. I'm afraid it also raises more questions for me (I probably need to go study CoffeeScript some more). BTW, the text I added is actually from the first example - the simplest:

  await $.getJSON url, defer json
Is the following right? Syntactically, "defer json" is an argument to the function "$.getJSON". So you could write it like this:

  await $.getJSON( url, defer(json) )
The code we are waiting for is the "$.getJSON" call. That code indicates that it is finished by calling the function returned by defer.

But what does the function returned by defer do? It looks like it (somehow) sets its argument to the result... so in the above example, "json" (somehow) gets the result of the call; in your example, "x" (somehow) gets a value. I'm guessing it's some convenient compiler magic that ICS does, and not an issue of CoffeeScript syntax.

An upshot of this is you can't give defer any old argument (e.g. a numerical value) - it has to be a variable that is capable of being set.

Also, who is the "callee"? A "callee" is a function that is called... Oh, does it mean (eg) "$.getJSON" is a callee, and it calls the callback function given to it (in the jQuery docs, it's notated as "success(data, textStatus, jqXHR)"; here, that success function is the function returned by defer). I guess, given the purpose of ICS, there will always be a callee in the block - but it seems to me that, syntactically, you could just directly call the function returned by defer. Ah... but (I bet) part of the compiler magic is that you are only allowed to call the function returned by defer from within another function - this is because the result of that other function is needed in order to set the argument of defer to it. Whew.

One last thing: the description talks about "deferrals" begin "fulfilled", but it doesn't say where these defarrals come from. Now, I think they are created by calling defer.

Hmmm.... it's probably better for me to think about this in terms of lisp macros, instead of C calls, as macros allow you to do weird source code transformations, like setting the value of a variable given as an "argument". i.e. ICS is a macro.

Note that I haven't tested ICS so it's only based on my understanding of the examples/docs.

"Is the following right? Syntactically, "defer json" is an argument to the function "$.getJSON". So you could write it like this: await $.getJSON( url, defer(json) )"

Yes, exactly.

"'m guessing it's some convenient compiler magic that ICS does"

Yes, exactly. When you say "defer(x)", it basically returns a callback like this:

  function(result) {
    *x = result;
    Code to terminate the await block*
"One last thing: the description talks about "deferrals" begin "fulfilled", but it doesn't say how they are created. Now, I think they are created by calling defer."

Yeah, defer returns a callback and when you call it, it "fulfills" the await, i.e. you tell await that you're done with that async block.

Thanks a second time, I'm getting there! I realized one can look at the transformed javascript output, to see just what it does. The setting of the variable in the defer becomes:

  $.getJSON(url, __iced_deferrals.defer({
      assign_fn: (function() {
        return function() {
          return json = arguments[0];
      lineno: 6
Presumably, when the function that __iced_deferrals.defer returns is called by the code in .getJSON that delivers the result, it will set arguments[0] to that result, and then call the assign_fn above. This will set json to that result.

Also, I realize that it doesn't set json to the returned value of $.getJSON, as $.getJSON doesn't return anything - it uses a callback to deliver its result. It seems that ICS always works this way - which I guess implies that the use-case it's for always uses callbacks...

BTW: I'm not sure why the above has a function which is then called immediately. Maybe a coffeescript thing, for separating namespaces? I assume the lineno: 6 is for debugging - I recall some debate against this. Good to see it got in.

About the function which is called automatically, it's a common pattern in javascript. It's used to create a new namespace/modules as block are defined per function instead of {}.

And, you are right that we're not assigning the return value of $.getJSON. Basically, once $.getJSON has finished with the request, it will call a function with the result. This result is set to the variable passed to defer. It's a bit of a brain teaser but that's the whole point of the await/defer thingy. Instead of using a callback to retrieve the result as an argument, you just write a variable after defer and you receive it.

If I understand correctly, everything after "await ..., defer" block will be executed only after the asynchronous operations from the await block are completed.

Does it mean that it's impossible to return any value from a function that contains await block?

  module = 
    dataPath: "http://search.twitter.com/search.json?q=blah&callback=?"

    init: () ->
      # Fetch data and do some stuff with it
        $.getJSON @dataPath, defer @data
      # Return yourself (this won't work)
      return @

There is a way around it. Here, everything after the await is transformed into a callback for the await magic:

    waity = (forFn) ->
      await forFn defer r1, r2
      console.log "values of defer variables", r1, r2
      return "this only returns from the result callback, not waity"

    syncReturn = waity (cb) ->
      setTimeout ->
        cb "r1 val", "r2 val"

    console.log "waity's synchronous return value:", syncReturn
So our output is:

    waity's synchronous return value: undefined
    values of defer variables r1 val r2 val
To return, you can simply write:

    waity = (forFn) ->
       setTimeout -> 
         await ...
       return "works as normal"

The aim of your docs should be to show how simple and readable IcedCoffeeScript is: using multi character variable names in your examples would help to achieve that.

When you see a two-liner of

  for k, i in keywords
    search k, defer out[i]
Do you really get lost or confused? Are you actually complaining about a problem, or just a violation of your favorite style guide? Personally, I'm looking for what makes IcedCoffeeScript unique; all else is just cruft, so keeping that terse is just fine with me.

Yes, I lose a few milliseconds. You'll notice your example had to include the loop for 'search k' to have any meaning, so I suspect you did too.

I'm still confused. Are those key-value pairs? What is "i" actually doing?

k is the value, i is the index.

Just on the website itself: it's very impressive presentation and usability for playing with examples. Couple of issues:

1. The floating title looks cool esp with greying out above it, but it page-down/space moves down more than a page (in effect), so it skips some text. I didn't realize I was missing some text for a while.

2. In the Animal example, the "Run" button doesn't work (though "run" works from the "load" overlay for it). I'm uising Firefox 9.0.1 on Ubuntu 10.04

Awesome! I'm wondering whether using jQuery's deferreds can give close to same advantages.

Looking at first parallelSearch example, it could be like that (in regular CoffeeScript):

  parallelSearch = (keywords, cb) ->
    searches = [search k for k in keywords]
    $.when.apply(searches, null).then cb
Although seems like it can't really cover all Iced's use cases. Hopefully the fork will continue to be maintained!

What this really needs is some simple example code for Use Cases :)

This is awesome

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact