Some really impressive optimization techniques used in this library :)
What's also really impressive is that ultra-fast means callback-level performance. And that performance is achieved even though promises are exception-safe (and don't crash the process like exceptions in callbacks ;)
In fact, this library may become the turning point that causes node.js userland to start switching to promises...
Like function templating using the Function constructor, try-catch wrapping, closure avoidance and many more. The source is a goldmine of JS optimization techniques (some V8-specific, some not so much)
This sounds very interesting even to those of us who don't need a promises library. If you or anyone else would like to go into detail, I bet a lot of people here would care (including me).
esailija is the author, and I believe he mentioned that he is planning to write an optimization guide (specific to the library, though it will probably contain generally useful info).
As a co-maintainer of Q, and author of the ES6 promise spec, this is extremely impressive work. I have had the privilege of watching it develop over the last few weeks, and am nothing short of amazed with what the author has done. The speed of this library opens up many doors for putting promises in places where previously some might have considered them not-low-level enough.
I'm finally starting to wrap my head around the concepts involved in promises, and was able to find a couple places to use them in my current rewrite of a JS codebase. However, my initial search doesn't show any articles describing why I should pick one of these over the other, particularly if they all implement the Promises/A+ spec.
Would someone be kind enough to give me some idea what the meaningful differences are between Q, when, and bluebird?
What you want is a library that is fast (like the one mentioned), and is easy to debug. Most promises libraries don't implement a good stack trace. The one submitted for this article has what looks like a pretty amazing stack tracer, and is quite fast. If it truly follows the Promises A+ spec (which I'm sure it does), it's probably your best bet.
I will speak mostly to Q, since I am a co-maintainer.
Q has historically stemmed from the work of Mark Miller, one of the authors of the E programming language, from which JavaScript promises are largely inspired. As such, it aligns with his larger goals for a promise ecosystem, which include object-capability security and using promises as proxies for remote objects. For an example of promises applied in these contexts, see his paper at [1].
In practice, all of this is somewhat theoretical, as Q also serves a large constituency of "normal" promise users in Node and the browser. There, it differentiates itself via a focus on features and integrity over speed (mainly as a matter of prioritization; we like speed too!); excellent Node.js integration [2]; and an internal system which allows promises for remote objects via Q-Connection [3] (but also gives some nice methods for promises for objects etc. for the same-machine case [4]).
I would say that Q is my favorite library to work with, as it has thought out its design choices in a depth I don't see in the other libraries---or perhaps more accurately, the choices they have made don't align with my sense of craftsmanship as well :). That said, you should probably not use it if your concern is speed. We have always taken the position that, while speed is nice, most of the time you are doing an asynchronous operation anyway, so any overhead from the promise library will matter very little.
"ref_send", the ability to create a deliverable value, would be my core linkage on Q and more-so promises (as not an author)
http://waterken.sourceforge.net/ .
The background is interesting, but I'm not sure what value an association with historical context brings. I've tried other promise libraries, but stub my toe on missing capabilities I'm used to, or end up distracted with mechanics. I don't think I've ever found myself saying, gee, would that this be closer to ref_send, but then again almost all implementations happen to tack close to. So, uh, Domenic does have a more useful account of Q. Mostly here to drop some ref_send. ;)
I'm not really seeing the speed on a fairly realistic perf [1] involving communicating with a web worker. Wrote a big blog post looking into this a couple weeks back [2].
that one uses window.addEventListner which gives an advantage to libraries that prefer that to ones that use that one first over ones that use messageChannel first. here is a fixes one[1], looks like everything using mutation observer is about the same throughput.
My background is that I've been trying to create a promise library that just creates without also being underscore for promises [1]
What sort of performance are we talking about that we are able to test without really measuring other things. For instance when was able to dominate many of the perfs by actually doing many of them synchronously in cases where other libraries would force it async.
That's not throughput. You are running exactly 1 promise in parallel at a time and waiting for it to be resolved.
However, on node.js you can have more than 1 client served by your server. For instance, your server could have e.g. 10000 promises/callbacks/generators active at a time. Take a look at how different solutions handle that https://github.com/spion/async-compare/blob/master/latest-re....
The results don't change at all if we for example increase latency of I/O by an order of magnitude - that's why latency is not an interesting metric for this use case. It's pretty intuitive, all the 10000 users will just get their results 9ms later but the fact that we could even handle them in the first place without crashing our server matters.
----------
You are right that for your use-case, pretty much the only thing that matters is what implementation is used for queuing async calls since it's such a tight bottleneck. Bluebird attempts to use mutation observer if available.
This isn't related to the library, but promises in Javascript are much different than what I learned about promises in Scheme. In Javascript, promises seem to be about asynchronous operations. In Scheme (and other languages?), promises are used for lazy evaluation. Scheme provides the delay and force forms for this. Now I need to make sure that I don't get confused about this in the future.
For a functional programming analogy, JS promises are more like monads than about lazy evaluations. I don't know if there is a good Scheme analogy because in Scheme you can use call/cc and friends while in JS you are forced to write your async code in explicit continuation passing style (promises are just a way to make this a bit more manageable).
Looks awesome, I've been using JS Promises recently and it's made control flow a lot easier to handle. I've been using Vow, because I like having an .always() that gets called after the promise was handled (rejected or fulfilled), but sadly that library eats tracebacks (at least, I can't figure out a way to do it). Bluebird looks promising, but IE<9 isn't supported at the moment (https://github.com/petkaantonov/bluebird/issues/3). Rules it out for most of my projects.
1. Its a promise library. It gives you a Promise class, a PromiseResolver and some tools to convert node functions to promises or to support dual promise/callback API.
2. Its super-fast and has full stack traces that go back until the beginning of the async event chain
5. Yes, promises are confusing. I'm going to write a straight-forward introduction (that uses bluebird) this weekend. But once you get past the new words and definitions, promises are actually not complicated at all. Infact, here is a short intro:
How are promises different? Instead of writing your functions to take callbacks, you write functions that return promises. Imagine that node's fs.readFile returned a promise instead of taking a callback. Then this code:
fs.readFile(file, function(err, res) {
if (err) handleError();
doStuffWith(res);
});
Still not good enough? What if I told you... that if you return something from inside the .then() callback, then you'll get a promise for that thing?
Say you want to get a line from a file:
var linePromise = fs.readFile(file).then(function(data) {
return data.toString.split('\n')[line];
});
var beginsWithHelloPromise = linePromise.then(function(line) {
return /^hello/.test(line);
});
Thats pretty cool. But the coolest thing is probably this: if you return a promise from within `.then`, you will get that promise outside of `.then`:
function readUploadAndSave(file, url, otherPath) {
// read the file
var filePromise = fs.readFile(file);
// upload it when done reading
var uploaded = filePromise.then(function(content) {
return uploadData(url, content);
});
// also save to another place locally
var savedLocal = filePromise.then(function(res) {
return fs.saveFile(res, otherPath)
});
// return a promise that "succeeds" when both saving and uploading succeed:
return Bluebird.join(uploaded, savedLocal);
}
readUploadAndSave(file, url, otherPath).then(function() {
console.log("Success!");
}, function(err) {
// This function will catch *ALL* errors, from the above
// operations including any exceptions thrown inside .then
console.log("Oops, it failed.", err);
});
Now its easier to understand the chaining. At the end of every function passed to a `.then()`, return a promise. Lets make our code even shorter:
function readUploadAndSave(file, url, otherPath) {
// read the file
return fs.readFile(file).then(
// upload and save when done reading
return Bluebird.join(uploadData(url, content),
fs.saveFile(res, otherPath));
});
}
What if we want to make sure the data is uploaded first before saving?
function readUploadAndSave(file, url, otherPath) {
// read the file
var uploadedPromise = fs.readFile(file).then(
// return promise that its uploaded
return uploadData(url, content);
});
return uploadedPromise.then(function() {
// return promise that its saved.
return fs.saveFile(res, otherPath);
});
}
Or even shorter, skip the temporary promise variables and chain .then the way we chain `stream.pipe` in node:
function readUploadAndSave(file, url, otherPath) {
// read the file
return fs.readFile(file).then(
// then upload it
return uploadData(url, content);
}).then(function() { // after its uploaded
// save it
return fs.saveFile(res, otherPath);
});
}
And similarly to how in a stream.pipe chain the last stream is returned, in promise pipes the last returned promise is returned.
Thats all you need, really. The rest is just converting callback-taking functions to promise-returning functions and using the stuff above to do your control flow.
Ah, and also, you can also return values in case of an error. So for example, to write readFileOrDefault (which returns a default value if for example the file doesn't exist) you would do this:
I like your examples. As one who is casually familiar with promises, I would find it helpful to see more promise patterns/recipes. In particular, how about a blog post on the promise equivalents to the https://github.com/caolan/async Collections and Control Flow methods (since that library seems to be heavily used in the node community)? Would a higher level promises utility library be appropriate or overkill?
Yes, I could include that in my guide. Since promises are actual values, most of the tools in async.js become unnecessary and you can just use whatever you're using for regular values, like your regular array.map / array.reduce functions combined with some promise tools like .all
You already have async.waterfall and async.auto with .then and .spread chaining.
files.getLastTwoVersions(filename)
.then(function(items) {
// fetch versions in parallel
var v1 = versions.get(items.last),
v2 = versions.get(items.previous);
return [v1, v2];
})
.spread(function(v1, v2) {
// both of these are now complete.
return diffService.compare(v1.blob, v2.blob)
})
.then(function(diff) {
// voila, diff is ready.
});
async.map is straightforward.
// download all items, then get their names
var pNames = ids.map(function(id) {
return getItem(id).then(function(result) {
return result.name;
});
});
// wait for things to complete:
Promise.all(pNames).then(function(names) {
// we now have all the names.
});
If you want to wait for the previous item to download first (ala .mapSeries) thats also pretty straightforward: you just need the previous download to complete, then run the next download, then extract the item name, and thats exactly what you express in the code:
// start with current being an "empty" already-fulfilled promise
var current = Promise.fulfilled();
var namePromises = ids.map(function(id) {
// wait for the previous download to complete, then get the next
// item, then extract its name.
current = current
.then(function() { return getItem(id); })
.then(function(item) { return item.name; });
return current;
});
Promise.all(namePromises).then(function(names) {
// use all names here.
});
The only thing that remains is mapLimit - which is a bit harder to write - but still not that hard:
var queued = [], parallel = 3;
var namePromises = ids.map(function(id) {
// How many items must be complete before fetching the next?
// The queued, minus those running in parallel, plus one of the parallel slots.
var minComplete = Math.max(0, queued.length - parallel + 1);
// when enough items are complete, queue another request
// for an item, then get the item's name.
return Promise.some(queued, minComplete)
.then(function() {
var download = getItem(id);
queued.push(download);
return download;
}).then(function(item) {
return item.name;
});
});
Promise.all(namePromises).then(function(names) {
// use all names here.
});
Is a library needed? Probably, but only for mapLimit.
What's also really impressive is that ultra-fast means callback-level performance. And that performance is achieved even though promises are exception-safe (and don't crash the process like exceptions in callbacks ;)
In fact, this library may become the turning point that causes node.js userland to start switching to promises...