
Kal – a clean JavaScript alternative without callbacks - rzimmerman
http://rzimmerman.github.io/kal/
======
rybosome
It's remarkable how similar this code looks to the monadic way of doing it.

For this Kal code...

    
    
        task getUserFriends(userName)
          wait for user from db.users.findOne {name:userName}
          wait for friends from db.friends.find {userId:user.id}
          if user.type is 'power user'
            for parallel friend in friends
              wait for friendsOfFriend from db.friends.find friend
              for newFriend in friendsOfFriend
                friends.push newFriend unless newFriend in friends
          return friends
    

Here's a pseudo-ish implementation in pseudo-ish Haskell.

    
    
      getUserFriends userName = do
        user <- findUser userName
        friends <- findFriends user
        if (User.type user) == "power user"
          then friends ++ parMap rdeepseq $ (getSecondNodes friends) friends
          else friends
    		
      getSecondNodes firstNodes friend = do
        secondNodes <- findFriends friend
        diff firstNodes secondNodes
    
    
    

I like the concept of Kal, and I look forward to seeing what people do with
it. Regarding syntax, I have to admit that I share the opinions of a few
others in terms of preferring symbols over so many keywords, but that's a
minor nitpick. Great job!

~~~
GeZe
If you are looking for a syntax closer to Haskell's, you can take a look at
LiveScript ([http://livescript.net/](http://livescript.net/))

eg.

    
    
        getUserFriends = (userName) ->
          user <- findUser userName
          friends <- findFriends user
          ...
    

Specifically, the section on "backcalls":
[http://livescript.net/#backcalls](http://livescript.net/#backcalls)

~~~
damncabbage
If you are looking for Haskell, you can take a look at Fay
([https://github.com/faylang/fay/wiki](https://github.com/faylang/fay/wiki)).

~~~
chx
What about Haste? [https://github.com/valderman/haste-
compiler](https://github.com/valderman/haste-compiler)

------
rzimmerman
This is a project I've been working on in my spare time for a while now and
I've finally decided to throw it out there and get some feedback. Do you think
the 'wait for' callback syntax is useful? What big features do you think are
missing?

~~~
ricardobeat
The async handlers looks great, specially parallel and series iterators. I'm
not sold on

    
    
        wait for friends from db.friends.find()
        //vs
        friends = await db.friends.find()
    

it's a bit hard to follow despite the natural language.

Is this a fork of the CoffeeScript compiler? Will it benefit from upstream
changes/features?

~~~
graue
I also vote for 'await' here. The 'wait for friends from' construct looks too
much like BASIC for my liking. It's not clear which terms are built into the
language ('wait', 'for', 'from') and which are not ('friends'). Also, since
the eventual result of that line is an assignment to 'friends', using the
existing = operator makes the language more consistent.

~~~
rzimmerman
Thanks for the feedback. It seems like a lot of people would prefer a more
normal looking assignment.

If you get a chance (and use TextMate or Sublime Text), please give syntax
highlighting a try
([https://github.com/rzimmerman/kal.tmbundle](https://github.com/rzimmerman/kal.tmbundle))!

~~~
graue
Unfortunately I don't use those editors — maybe you can write a Vim
highlighter next? Might also be worth adding syntax highlighting to the
website (even if you have to do it manually for now), if you're set on the
current syntax and think highlighting will make that much of a difference.

~~~
jasonlotito
Have you tried converting it in:
[http://coloration.ku1ik.com](http://coloration.ku1ik.com)

------
aufreak3
(Don't mean to divert this thread from Kal, but this is on the topic of
"callbacks are bad" etc.)

For some time now, I've been accumulating async patterns I've needed when
writing JS code in my monadic IO.js library[1]. The aim of IO.js is to provide
higher levels of thinking about sequences of actions without worrying about
whether they are async, and with a rich set of error management tools (for ex,
exceptions are turned into recoverable conditions in IO.js).

The crux of the "callback hell" problem is, contrary to what many have
claimed, is not the nesting that results when the callbacks need to be called
in temporal order. That much is straightforward to deal with. The "hell" rears
its head when you need to coordinate multiple such sequences that are running
concurrently. The composable actions you get from a monadic treatment combined
with CSP-style channels, are an expressive framework to build abstractions on
(tldr - Haskell's implementation is awesome!).

For illustration, the framework in IO.js is flexible enough to implement a
node.js web server that can express PG's "Arc challenge" concisely (though
that challenge is practically obsolete). Take a look at [2].

For a simpler example, `IO.trace` is a straight forward way to generate a
trace dump to console of a sequence of asynchronous actions. You don't need to
"enable trace" for an entire app. You can choose to trace only a particular
sequence. This is pretty neat when debugging. Stack traces are useless when
dealing with async processes anyway.

[1]:
[https://github.com/srikumarks/IO.js](https://github.com/srikumarks/IO.js)
[2]:
[https://github.com/srikumarks/IO.js/blob/master/examples/arc...](https://github.com/srikumarks/IO.js/blob/master/examples/arc.js)

~~~
mistercow
>The "hell" rears its head when you need to coordinate multiple such sequences
that are running concurrently.

This situation is why I wrote DelayedOp:
[https://github.com/osuushi/DelayedOp](https://github.com/osuushi/DelayedOp) .
Of all the little reusable code bits I've written, this one has ended up in
more of my projects than any other.

Basically, a DelayedOp is a lightweight manager that runs a callback after its
"wait" and "ok" calls have balanced. It ends up being equivalent to _.after(),
but it's more natural to use than manually counting your concurrent
operations, and it has some conveniences for debugging, so that if a callback
never fires, you can find out why.

------
molf
Excellent work. Especially useful since it appears to throw exceptions. That
means it's a step up from Iced Coffeescript.

The syntax is a little verbose and might be shortened to something like C#:
`user = await db.users.findOne {name:userName}`. Would be a lot more clear to
me.

~~~
crazygringo
As far as I can tell from reading the readme, it does _NOT_ support throwing
exceptions from within callbacks.

Because, of course, there's no way to technically do that in anything that is
JavaScript at its core -- at least as far as I'm aware of. Correct me if I'm
wrong?

It says "This includes error handling via callbacks." \-- not error handling
via exceptions. Elsewhere it says "Any errors reported by fs.readFile
(returned via callback) will be thrown automatically.", so maybe it converts
certain callback error functions to exceptions? But in any case, that's not
the same as throwing an exception inside of a callback, and having it "bubble
up". Unless I'm misunderstanding (which would be great!).

~~~
TheZenPsycho
You can do it using monadic augmentation. Or aspect oriented programming.

In plain english, make a function that takes two functions
makeADoThingFunctionThatHandlesErrors(doThing, handleError)

and returns a new version of the function that is internally wrapped in a try
catch block with the handleError function getting called from the "catch"
block. This is relatively straightforward to accomplish using closures/high
order functions. This is one utility function you write once.

Finally you can do this with a whole "monadic" api such as jquery- create a
function that loops through all the methods on an object prototype and wraps
them in an error handler, returns a new version of the prototype you can
inherit from.

Or instead of doing all that, you just use some library with promises (such as
Q, or jQuery) that implements all the above, and all you need to do is write a
single error handler function, and any error that gets thrown by any function
in a promise chain gets caught and sent to that function.

------
kev6168
I suggest to use ..= (or ...=) in place of the keywords 'wait for', and
'from'. Less words, and '...' traditionally means 'to be continued', or 'wait
for a while'.

For this Kal code:

    
    
      task getUserFriends (userName)
        wait for user from db.users.findOne {name:userName}
        wait for friends from db.friends.find {userId:user.id}
        return friends
    

would become:

    
    
      task getUserFriends (userName)
        user ..= db.users.findOne {name:userName}
        friends ..= db.friends.find {userId:user.id}
        return friends

------
10098
The title is kinda funny to the russian ear because kal means feces in Russian
:)

~~~
rzimmerman
That is really unfortunate...

I chose it because it in Hebrew it roughly means something like
easy/simple/BASIC.

edit: More importantly, does that make you more or less likely to use it?

~~~
raquo
I'm Russian and I didn't blink reading the title. I never stopped to
contemplate the fece-ness of iCal either, until today (thank you, 10098).
Anyway, keep the name. It's a fun easter egg as far as I'm concerned.

Also, note that kal is mere `faeces` in Russian, not `shit`. I don't remember
when I've last used the word outside of a hospital... or inside, for that
matter.

------
mjackson
For the curious, the equivalent example from the OP using promises[1]:

    
    
      function getUserFriends(userName) {
        return db.users.findOne({ name: userName }).then(function (user) {
          return db.friends.find({ userId: user.id });
        });
      }
    

[1]: [http://promises-aplus.github.io/promises-spec/](http://promises-
aplus.github.io/promises-spec/)

~~~
prezjordan
It's more obvious if you have multiple levels.

    
    
      function getUserFriends(userName) {
        return db.users.findOne({ name: userName })
          .then(function (user) {
            return db.friends.find({ userId: user.id });
          })
          .then(function (friends) {
            return somethingAsync(friends);
          });
      }

~~~
mjackson
I don't ever write code like this personally. The multiple levels make it more
difficult to reason about, IMO.

------
tuxracer
[https://github.com/kriskowal/q](https://github.com/kriskowal/q) solves this
today without needing to cross compile. [http://koush.com/post/yield-
await-v8](http://koush.com/post/yield-await-v8) will address this natively
however unlike Q or Kal you can't start using it today.

~~~
ricardobeat
There is no need for hacks, just run node with the `--harmony` flag to get
yield support today.

Q is a huge, monster library. It's 2x the size of caolan/async, which is
already bloated. I personally think it's better to use
[https://github.com/mbostock/queue](https://github.com/mbostock/queue) or
another small, understandable library.

~~~
ChiperSoft
If you're taking about node, what difference does it make how large the
library is?

~~~
ricardobeat
These libraries work in the browser. It's not just size: debugging,
understanding code, and performance matter too.

------
ufo
Does anyone know if there is a JS dialect out there that adds support for
callback-less async code without also adding a bunch of other coffeescripty
syntax changes at the same time?

~~~
neilk
Yes, it's called "JavaScript". Specifically, combining generators with
promises.

[http://blog.alexmaccaw.com/how-yield-will-transform-
node](http://blog.alexmaccaw.com/how-yield-will-transform-node)

Definitely not as clean as a whole new language, but you can program like this
today, at least in bleeding edge node and some recent browsers.

------
jay_kyburz
rzimmerman I like it! and would consider using it for my next big project.

If I were you I would make a slight course correction follow in the footsteps
of Python and Go and have one and only one correct way to do something.

When you say use two spaces, but you can use more if you want, make it an
error to use more.

Pick the correct way to call a function. Pick the correct way to declare a
function.

Keep up the good work.

~~~
rzimmerman
Thanks! I wouldn't call it stable or production ready by any means, just as a
disclaimer.

~~~
jay_kyburz
Oh hey, another idea, if you keep the project really tight and small, I wont
be too intimidated to get into it and help fix issues if I start using it.

------
dogles
I am a big fan of languages trying to solve callback hell. I use a library in
Haxe that allows for similar syntax:
[https://github.com/proletariatgames/haxe-
continuation](https://github.com/proletariatgames/haxe-continuation). Haxe
compiles to Javascript and is great for node programming...but it has the
added benefit of also compiling to c++, php, java, c#, etc., etc. ;)

------
LordHumungous
JS without callbacks... so JS without the best part of JS?

~~~
rzimmerman
It's admittedly a bit link-baity. The async structures in Kal do compile to
callbacks in JS.

------
anuragramdasan
Of the tonnes of Javascript libraries that are unloaded on HN regularly, this
is one of the more fascinating ones. Absolutely love the syntax. Great job!

------
TheZenPsycho
So instead of encouraging you to actually fix the bad nested callback code,
you can dress it over with fancy syntax.

Well done on making a whole language though. It's pretty challenging and I
hope you do well.

Aside: This seems very similar to narrativeJS in concept
[http://www.neilmix.com/narrativejs/doc/](http://www.neilmix.com/narrativejs/doc/)

~~~
rzimmerman
Thanks. I think the main issue with JS callback syntax is that it quickly
becomes unmaintainable. Hopefully this fixes a lot of that by abstracting away
the mechanics. I don't think there's something fundamentally wrong with the
"relinquish control and wait for I/O" model, so I tried to make that explicit
here.

------
harlanji
I came across Galaxy yesterday, linked to from the callbacks/goto discussion.
It's an NPM module that gives C# async/await semantics to JS/es6 code (of
course it requires the harmony VM). It simply toggles function*/function on
things, such as required modules, and provides some extra helpers... the
author gives a really nice walkthrough:

[https://github.com/bjouhier/galaxy](https://github.com/bjouhier/galaxy)

Galaxy does not claim to get rid of callbacks, though... only to help make
writing JS/es6 with async/await semantics easier. I've yet to try it out, but
the instructions seem clear.

------
pyalot2
How many times do I have to say this. Generator/Yields and equivalent
constructs don't actually solve the callback problem, they exacerbate it. That
is, because you only get to direct one call-level down, but, that's now how
you'd code. You'd have some perfectly synchronous code until it hits some
infrastructure I/O part some N call layers down, and then now what? You're
gonna convert everything to yields? Got any idea how messy that is?

Javascript needs proper co-routines. Nothing short of co-routines will solve
the issue. Please lookup microthreads, greenlets, fibres, etc.

~~~
jeswin
I think you have it wrong, generators would solve the problem. If you have
asynchronous code N call layers down, then everything above it would be
asynchronous as well. So you would only need to yield the first call.

Check how Q.nfcall works.

~~~
pyalot2
That's a faulty assertion. A generators working principle is that it can
resume into the closure if the "caller" is looping over the generator.

Let's assume a simple call stack, A -> B -> C. Now C would be a generator, so
B has to loop over C instead of just calling C.

But what about A? Well if B is now looping, and if yielding out of a loop is
how you cooperatively multitask, then A now has to loop over B. So you have to
convert the entire call stack from A -> B -> C to A(for ... in B(for ... in
C)).

You can argue that a compiler can make this invisible, however there are some
serious drawbacks with that in that:

1) One solution would be to convert any call into a loop. How do you know
which ones are actual calls (stuff like Math.pow etc?) 2) You only convert
things that have generators in their call stack to loops, but how do you know
that?

Both the above scenarios would depend on being able to infer the
"generatoricity" of the called symbol. Something which is very, very hard to
do for dynamic languages like Javascript.

Co-routines work differently. Co-routines save the entire call stack and
restore another one on switching to somewhere different. Hence if you have A
-> B -> C and C is some I/O that would have to go to some scheduler, then the
A -> B -> C call stack is saved, the co-routine would switch to say, Main ->
Scheduler, the scheduler would do its work, and when it's ready, it switches
back to A -> B -> C where that co-routine left of.

~~~
jeswin
I am not sure if I followed entirely.

But let me give you an example, again with Q.nfcall.

    
    
        MultiplyAndSquare = (a, b, cb) ->
            Multiply a, b, (err, product) ->
                Square product, (err, square) ->
                    cb null, square #ignore errors for now.
        
        nonCallbackVersion = (a, b) -> Q.nfcall(MultiplyAndSquare, a, b)
        
        result = yield nonCallbackVersion(10, 20)

~~~
pyalot2
You see how the caller has to be aware of how what he's calling is a
generator. You're dragging generators all the way trough the call chain as
soon as anything deep down in the call chain is a generator, and you'd want to
use generators to acomplish asynchronicity.

Let's say you write some code A, which uses some other code B, which uses some
other code C. C is also used by various other pieces of code like X, Y, Z, D,
E, F, G, H, J, K etc. Now at some point you decide, well, you don't like
callbacks in C, so you'll convert it to something asynchronous. You'll now
just have to refactor A, B, D, E, F, G, H, J, K, X, Y and Z to make all their
calls to C a generator iteration, and you'll have to convert any code that
depends on A, B, D, E, F, G, H, J, K, X, Y or Z to be aware of that, any any
code which uses those, to be aware of that.

~~~
jeswin
Sure, call sites of C will need to change, since C's signature changed. That's
expected, since C doesn't even take a callback anymore. But callers of A,B,D,E
etc don't need to change. A,B,D,E etc can still take a callback parameter and
return C's result via the callback.

    
    
        MultiplyAndSquare = (a, b, cb) ->
            Multiply a, b, (err, product) ->
                Square product, (err, square) ->
                    cb(null, square) #ignore errors for now.
    
        
        C = (a, b, cb) -> 
            MultiplyAndSquare(a, b, cb)
            
        
        ABDEXY = (a, b, cb) -> 
            #Calls C
            C(a, b, cb)
            
        
        callersOfABDEXY = (a, b) ->
            #Calls ABDEXY
            ABDEXY a, b, (err, result) ->
                Console.WriteLine(result)
            
        
        #Refactored C. Returns a promise instead.
        refactoredC = (a, b) -> 
            (Q.async -> 
                result = yield Q.nfcall(MultiplyAndSquare, a, b)
                return result)()
        
        #Refactored ABDEXY.
        refactoredABDEXY = (a, b, cb) -> 
            #Change needed here, since C now returns a promise
            refactoredC(a, b).then (result) ->
                cb(null, result)
    
        #No need to refactor callersOfABDEXY

------
ronreiter
What's up with everyone trying to re-invent coroutines? I personally think
people should upgrade JavaScript instead of inventing new languages all the
time, since it creates fragmentation.

------
tlb
I'm puzzled by the bootstrapping process. It's written in Kal itself, and you
distributed compiled code as JS. Do you have a way of re-boostrapping from
only the source in github?

~~~
rzimmerman
The only way to re-bootstrap is to download a compiled (JS) release from npm,
then use that to compile the .kal source in the git repository. Then you can
use your newly compiled version of kal to recompile itself. This is how I
create the npm releases.

It's not turtles all the way down, though. If you look way back, the 0.1ish
releases of the compiler were written in CoffeeScript. I ported the compiler
source over from CoffeeScript file by file as it matured and eventually got
rid of the dependency entirely. Technically, you could start with the last
release that was written in CoffeeScript and compile up to the current
release.

~~~
tlb
Interesting. It must get tricky when you make an incompatible change to the
language. I guess you'd make the changes to the compiler still using the old
language, update the test cases, make it stable, then change the compiler
sources to reflect the new language, recompile the compiler, then check in the
new compiler JS. With plenty of git checkpoints along the way.

~~~
arnarbi
One of the gems in CS literature illustrates this nicely: [http://cm.bell-
labs.com/who/ken/trust.html](http://cm.bell-labs.com/who/ken/trust.html)

Essentially self-compiling compilers can "remember" things that are nowhere to
be found in the source code.

------
arghbleargh
It appears to be very similar to icedcoffeescript:
[http://maxtaco.github.io/coffee-script/](http://maxtaco.github.io/coffee-
script/)

~~~
smilekzs
Agreed. Why reinvent wheels?

~~~
ics
Because spokes and rims work better for different things.

~~~
smilekzs
Perhaps name a serious deal-maker of this over iced-coffee-script?

------
jashkenas
Beautiful work. The `for parallel` notation is a really interesting way of
writing it.

It seems like a number of folks in this thread have expressed interest in your
process -- both of designing the feature set, and of gradually bootstrapping
it away from CoffeeScript to become self-hosting. I'd love to hear more, if
not here, then in a blog post, perhaps...

~~~
rzimmerman
Thanks! I'm a big admirer of your work, so it means a lot to me to hear that.
I wrote up a little bit about this if you are still interested:
[http://blog.rzimmerman.me/how-kal-compiles-
itself/](http://blog.rzimmerman.me/how-kal-compiles-itself/)

------
egeozcan
What do I do when I want to limit the number of tasks that are run in
parallel? I'm talking about something like the queues in the famous async.js.
I've seen tame.js trying to solve this by a construct named rendezvous but to
me it looked even messier than inline callbacks.

------
reustle
Using 4 spaces to indent the JS code and 2 to indent the Kal code seems a bit
disingenuous

------
huu
Is it possible to cut out some of the explanatory comments in the examples or
to consolidate them somehow? It's difficult to see at a glance how the code
looks, especially without any syntax highlighting at this stage

~~~
rzimmerman
I added syntax highlighting to the github landing page manually for now. Try
the .tmbundle if your editor supports it. It's not full featured but it gets
the colors going. I'm working on getting it into pygments.

------
calvintennant
In the `Asynchronous Wait` example did you mean: `function
getUserFriends(userName, next) {` when you wrote: `var
getUserFriends(userName, next) {`?

~~~
rzimmerman
Woops good catch!

------
progx
But where is the error handling ?

You have some examples with bloated err-returns, but where is that in your
kal-code ?

------
a__remie
cbs - cbSucces cbe - cbError

replaces all the "if (e) return cb(e)" lines with cbe when you call the async
function.

A whole abstraction over js just because of this, running away from the
problem...instead of solving it on the spot, keep running.

------
__xtrimsky
I Love callbacks, wtf is wrong with you guys :'(

~~~
progx
Not callbacks are a problem, but many nested callbacks are unreadable und hard
to maintain.

------
ilaksh
ToffeeScript is very similar but better.[https://github.com/jiangmiao/toffee-
script](https://github.com/jiangmiao/toffee-script)

~~~
ghotli
This comment would be much more insightful if you put forth your argument as
to why it is better.

~~~
ilaksh
Sorry I was in a hurry. It has many more features for a sync that looks like
sync and for other things. It has a cleaner syntax. It supports easy access to
multiple callback return values. It has all of the other CoffeeScript
features. It seems very obvious if you look at ToffeeScript that it is
objectively better. I find it difficult to believe that it isn't getting more
attention. Probably just because it doesn't have a web site.

------
pencilcode
i really like the syntax - really close to english. is there a side-by-side
comparison of kal and the generated javascript?

~~~
rzimmerman
Thanks for the suggestion. The closest to a side by side is here
([http://rzimmerman.github.io/kal/#asynchronous-
wait](http://rzimmerman.github.io/kal/#asynchronous-wait)), and also the table
of operators in the readme
([https://github.com/rzimmerman/kal/blob/master/README.md#oper...](https://github.com/rzimmerman/kal/blob/master/README.md#operators-
and-constants)). Also the examples directory on the github repo.

Apologies if you saw these already. I'll add something like what you
suggested, though.

------
halacsy
very cool & pragmatic project: javascript / java / c++ coders get one step
closer to functional programming.

------
u2328
Yay, another JavaScript super-set language that solves _one_ problem.

~~~
graue
On the one hand, this comment has the kernel of a legitimate criticism in it,
and gets across why I probably won't use this language in a real project.

On the other hand, that's so rude! Not every project has to be the next
CoffeeScript to be interesting and worthy of appreciation. If you don't like
it, just click on a different story instead of cutting down someone who chose
to make something cool and share it with us.

~~~
u2328
You're right. I pretty much just blurted out the first thing that came to my
mind when I read about it, and what I said was equivalent to garbage. It was
completely uncalled for and I apologize to everybody in this forum for
lowering the level of discourse, much less crapping over someone's hard work
and pride.

