
Node-fib: Fast non-blocking fibonacci server - dchest
https://github.com/glenjamin/node-fib
======
raganwald
Note to self: Starting immediately, all raganwald projects will have a “Is it
any good?” section in the readme, and the answer shall be “yes."

------
glenjamin
Author here, didn't really expect this to get picked up anywhere but since it
has I'd like to point out the idea was to demonstrate that computationally
expensive algorithms can be split across multiple iterations of the event loop
to avoid blocking it.

In this case concurrent requests take advantage of each others' memoisation,
which would be somewhat trickier to do with threads as you'd probably need to
worry about locking.

~~~
jerf
Yes, you too can by required by the compiler to implement cooperative
multitasking by hand. In 2011.

Yes. It is an answer to the criticism made, and I acknowledge that. But it is
not a very _good_ answer to the objection. The better answer is "don't do that
in Node.js", which is still not all that great (it's _really_ easy to
accidentally write something that blocks badly), but is better.

~~~
glenjamin
Well the other option for computationally expensive code is to use some sort
of worker that runs a sufficiently fast language.

JavaScript on v8 is actually one of the fastest interpreted languages
available, so unless you _really_ need to drop down into C or similar,
splitting across the event loop or using another node child process is not an
unreasonable way to approach CPU heavy calculations.

~~~
masklinn
> Well the other option for computationally expensive code is to use some sort
> of worker that runs a sufficiently fast language.

Which only helps if you _know_ your code is going to be slow. If you somehow
implemented an algorithm with a quadratic complexity and did not test for
sufficiently large input, you might not realize what's going to happen before
it hits production.

> JavaScript on v8 is actually one of the fastest interpreted languages
> available

1\. Nobody is denying that.

2\. The issue is with the behavior of evented systems in general and node in
particular in case of in-request CPU-bound code paths, namely that the whole
server blocks killing concurrency and basically DOSing the instance.

~~~
glenjamin
I can't disagree with any of these points.

NodeJS is no magic bullet, those who treat it as such should be extremely
wary. It is, however, rather nice to work with.

------
justin_vanw
I think this proves Ted's point. Look at that code. It is ugly as sin. Since
I'm not an expert at node.js, I can't follow what is going on at all. Plus,
this is memoizing. That is just cheating, it doesn't matter how you implement
the algorithm if it memoizes.

If the creator of this github repo is to be believed, implementing a 3 line
algorithm in node, you must use 20 lines of intricate, ugly code. If that is
the case, node.js IS cancer.

~~~
robinduckett
Node.js can be done in different ways to this, the fact is, that this code
works asynchronously, and this is how Node does async.

------
carllerche
Instead of memoizing in the server you could memoize in the browser and use
socket.io to ask connected browsers if they have a memoized value for a given
fib number. That would get you around the limitations that bascule mentioned.

~~~
glenjamin
Now there's an idea! That could be rather fun...

------
rkalla
As much as this whole Ted-gate thing has turned into a amped up argument of
sorts, there has been a significant amount of knowledge for everyone shaken
out of the trees about node.js as a result.

Take for example this implementation, how many people getting started with
node knew about async or memoization?

Could be just me, but I learned a lot about what not to do and how to do
certain things more efficiently with node as a result of the whole fiasco.

Thanks Ted :)

~~~
knieveltech
Aahz's law

    
    
       The best way to get information on Usenet is not to ask a
       question, but to post the wrong information.

~~~
corysama
There's an xkcd for everything... <http://xkcd.com/386/>

~~~
rhizome
Unfortunately, it's usually that one.

------
ovi256
I'd like to point out that this uses important improvements over the naive
Ziuba's strawman version: memoization and callbacks using the event loop. The
callbacks improve concurrency, allowing a single process/thread to multitask
requests, and memoization reduces total time spent per request.

Actually, if the async lib memoization facility shares values between
requests, which seems quite sure to me, all requests but the first one are
served from its cache in the test the author uses as example. This still
doesn't highlight any strength of node.js IMO, asides from the easy shared
memoization. It could be translated to Tornado-web, an async Python framework,
almost literally.

~~~
masklinn
> I'd like to point out that this uses important improvements over the naive
> Ziuba's strawman version

The Ziuba version was not "a strawman", it was an example used to trivially
generate high in-request CPU load and demonstrate the misbehavior of evented
system under this condition.

If you want to compute fibonacci fast, just use Binet's Fibonacci formula, it
runs in O(1), then your only issue is that you're going to overflow _very
fast_ on doubles (not sure you'll even reach fib 1500). Using decimal, you can
probably go higher before reaching your limits (though you'll run out of
display space long before that if you print all digits): Python's
decimal.Decimal reaches ~fib 4000000000 before an overflow error.

~~~
SamReidHughes
Binet's formula doesn't run in O(1) if you want an exact answer. Using a + b *
sqrt(5) representations or floating point with sufficient precision, you can
get O(n log n 2^O(log* n)) time (quoting Wikipedia's Furer's algorithm page
here). Which beats O(n log n log log n) time, if you were wondering. If
something beats that for computing Fibonacci numbers, that'd be pretty cool.

------
fendrak
Is it just me, or have all these implementations of fast Fibonacci servers
missed the original intent of the 'Cancer' article? As I read it, it was more
of a commentary on giving novice concurrent programmers enough rope to hang
themselves than on the capabilities of Node itself.

~~~
lamby
Perhaps if enough alternative implementations are submitted to HN it will
drown out the silence on the actual issue that was raised.

------
wpeterson
The only difference in this implementation is that it uses memory-based
caching (memoization) to compute given value once and then serve the cached
copy.

Of course this is fast for 1000 iterations, since the effective cost is zero
from the second request.

People really are misunderstanding the critique of fibbonacci as representing
any CPU intensive task.

TLDR;

All the author did here was remove the CPU intensity by caching the
calculation

~~~
DrHankPym
I'm actually very confused by his criticism. You can write a CPU intensive
task in any language, and you'll have the same problem.

Or is that the point? Some people believe Node.js will magically make all
processing computations = 0? I'm all for discouraging the rumor that Node.js
will solve every problem, but don't call it Cancer.

~~~
burgerbrain
The issue is _what happens_ when you write that CPU intensive task. If you do
it in idiomatic Go, that same server will keep on responding to other requests
in the meantime. With Node.js, that is not the case. You have to do things
like, well, what this article does, to get it to work.

~~~
DrHankPym
I'm still not getting it. Go might be able to handle an extra request or two
for high CPU intensive tasks, but it will inevitably suffer the same
consequence of locking all it's processes given enough requests.

Still doing anything that is CPU intense on that layer is stupid in the first
place.

~~~
burgerbrain
A so called 'goroutine' locking up will not lock up the rest of that server.
Other pages will still be served by that server. This has nothing to do with
the efficiency of the underlying language, it has to do with Go doing
cooperative multitasking _transparently_. It might as well be a separate
process (in fact, for all you really know it could be).

The code seen here does cooperative multitasking _explicitly_. It is
essentially explicitly specifying places where rescheduling can take place.
The result is the same but with this Node code you are getting your hands far
more dirty.

The point of the Cancer article is that since it (allegedly) is not made
explicitly clear what is going on, programmers can deal far more damage to
themselves than they would with other schemes.

I really don't know how to explain it any better than that.

PS: "cpu intensive" is relative. The fib example is a deliberate exaggeration
of what you'd see in reality, to make the point trivial to observe.

~~~
willvarfar
Right. If Ted-gate was advocating using go or erlang I'd be so cool with
everything except his choice of language.

But he wanted us to go back to CGI folks...

~~~
burgerbrain
To be clear, CGI doesn't exhibit the issues seen with Node either, it's issues
are elsewhere (and separately debatable).

------
callahad
I wonder why I don't often see folks directly computing Fib(n) using the
equation given in SICP exercise 1.13:

Fib(n) = round(φ^n / sqrt(5)), where φ = (1 + sqrt(5)) / 2.

Cites:

[http://mitpress.mit.edu/sicp/full-text/book/book-
Z-H-11.html...](http://mitpress.mit.edu/sicp/full-text/book/book-
Z-H-11.html#%_thm_1.13)

[https://secure.wikimedia.org/wikipedia/en/wiki/Fibonacci_num...](https://secure.wikimedia.org/wikipedia/en/wiki/Fibonacci_number#Relation_to_the_golden_ratio)

~~~
raganwald
To answer your (possible rhetorical) question literally, the requirement here
isn’t that it generate Fibonacci numbers efficiently, it’s that it implement
the recursive algorithm efficiently.

It’s true that another algorithm will be much faster, but that deprives us of
the opportunity to discuss how this implementation of the recursive algorithm
delivers much higher performance than a naïve implementation.

The back story is that an abrasive blog post asserted that Node.js was a
terrible platform for certain types of problems, and this algorithm was given
as an example. The author here is simply showing that with care, Node.js will
not be the problem.

For shits and giggles, here’s another Fib algorithm, chosen strictly for the
pleasure of implementing it:

[https://github.com/raganwald/homoiconic/blob/master/2008-12-...](https://github.com/raganwald/homoiconic/blob/master/2008-12-12/fibonacci.md)

~~~
foca
The specific point is that the evented model is not designed as a "one size
fits all" solution. It's great for certain scenarios, not very good for
certain others.

Basically, the original blog post tries to get to this, remarking on _node.js
is sold as if it was the silver bullet of web servers, that will scale to
anything for anything you_.

Of course, you can try to adapt most problems into something that would work
in an evented model, but you have to know VERY WELL what you're doing.

IMO the solution is simple: polyglotism. Have parts of your app in node. Other
parts in python. Others in ruby. Whatever works best for each specific small
part :)

Node is not useless, but it's definitely not my choice to program everything
and anything. Same thing with, for example, Rails :)

------
jrockway
This is the wrong approach. The right approach is to write your
computationally-intensive code so that it runs in another process, and then to
talk to that process over the network. Network servers must only talk to the
network.

~~~
prodigal_erik
If I have unused capacity, I can afford simpler threaded code. If I'm
overloaded, I need some other platform for the real work. It sounds
interesting but unless I need to write my own software load balancer, I'm not
sure what node (as implemented today) is appropriate for doing.

------
mark242
Note to Ted: next time, just use sleep().

~~~
glenjamin
To which the response is the same: simply don't block the event loop and use
setTimeout.

The whole cancer thing is based on the premise that blocking the event loop is
unavoidable. I'm attempting to say that it isn't. Jerf's points about how it
can happen accidentally are perfectly valid, it's up to the developer not to
do something stupid and to actually have decent tests and benchmarks in place
- as it would be with any approach.

Its certainly true that significantly poor performance in code which doesn't
release the loop will degrade a single-threaded event loop far more than a
threaded approach.

~~~
mark242
CPU speed and bus speed is finite on the machine you're running on. Using
setTimeout to poll the results of some thread you've spawned to run a
calculation is _worse_ performance than just blocking in the first place,
because now when you try to profile your application (how does one do that in
node.js, anyways?) you're going to be looking at a million setTimeout closed
loops.

Here's Ted's entire point: "The point is that the HTTP server isn't the same
entity doing the application work."

..which has basically been true ever since mod_jk was created.

~~~
SaltwaterC
The point is moot. HTTP is just the transport protocol in this case. Doesn't
have to say that it has to be the front end service that talks to the end user
web browser. The OP of the "cancer" article made up that shit. I can bind the
listener of a node service to an UNIX Domain Socket and talk HTTP over that
socket with a front end web server, instead of plain TCP (reduces the IPC
latency). It is IPC between distinct components. It delegates the dynamic page
generation to another service. It is a text interface, although sometimes,
well, most of the times, the binary protocols outperform the text based
protocols therefore the theory is crap. Isn't this "UNIX way" enough for some
people? Forget the part that says: any IPC adds extra latency to the request -
response paradigm aka the opposite of the stuff a web stack should do.

Can you tell me at least one example from the myriad xGI protocols for
interfacing a web server with an application server that actually brings any
benefit over having plain HTTP/1.1 and a reverse proxy? Does it help to have
yet another protocol for doing basically the same task: IPC between a couple
of servers? Any xGI client (aka web server) is basically a reverse proxy for
an IPC protocol. Does it say somewhere, carved in stone, that this protocol
must be other than HTTP? I think that most people that spend the time
engineering yet another xGI solution simply don't see the forest for the
trees.

PS: nobody stops anybody to drop a scgi.js or fastcgi.js module for node.js if
that keeps people warm at night. But maybe I'm one of those tired of this
shit: engineering yet another xGI protocol for every programming language that
floats around the web. NIH syndrome, I'd say. At least HTTP is ubiquitous,
while most servers already HAVE a reverse proxy handler.

------
InclinedPlane

      function fib(n)
      {
    	var phi = (Math.sqrt(5) + 1)/2
    	return Math.floor( Math.pow(phi, n)/Math.sqrt(5)+0.5)
      }

~~~
adgar
Anyone know offhand what the minimum value of n is for which this fails due to
floating point inaccuracies?

~~~
InclinedPlane
Javascript doesn't have more than one number data type. This algorithm may
fail earlier due to loss of precision from the way JS does exponentiation or
sqrt or some such but even a purely additive method will fail due to floating
point inaccuracies as well.

Edit: On an in-browser test with Chrome I get accurate values up to 75 with
the closed form function and accurate values up to n=78 for the recursive /
additive function.

------
bascule
Hmmm...

    
    
        $ time curl localhost:3000/1000000
        curl: (52) Empty reply from server
    
        real	1m0.392s
        user	0m0.006s
        sys 	0m0.004s
    

This actually crashes the server because it runs out of memory:

    
    
        $ node app.js 
        FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory

~~~
flexd
32bit or 62bit node? If 64bit did you specify it can use more ram than the
1.7GB that's the default setting?

You can up the limit by starting node with --max-old-space-size=<size in MB>

~~~
bascule
Fibonacci isn't a kind of problem that should be bounded on RAM though.
Something is fundamentally broken with the way this is implemented.

~~~
jcdavis
Sure it is. Stack space isn't free

~~~
jamwt
It is with TCO.

~~~
jcdavis
How exactly is the particular implementation in question tail-call optimized?
Honest question, not trying to be snarky. I'm well aware that fib can be done
to use TCO, but this doesn't appear to do so

------
amit_heroic
Any tool in the wrong hands, is dangerous. So anyone trying to prove that fib
makes node slow, is stupid. Node.js was made to make memory requests
unblocking; not to generate the fib series for you.

Trying to nail a hammer with a drilling machine, is not to take yo anywhere.

Peace.

------
ww520
I'm actually quite impression with the 5000 requests/second ab result. I can't
get more than 1000 requests/second on my pony laptop. What hardware are you
using?

------
DrHankPym
Wow, this really made me laugh.

~~~
AlexC04
Me too. I was rolling my eyes this morning at all the he said she said stuff.
It's been a pretty standard example of "hacker news eating its own tail"
today.

\- Node sucks;

\- No it doesn't * 2;

\- Use Haskell;

\- (others) ?

This one however has made me laugh as well :)

------
pabloPXL
too much boilerplate :)

<https://gist.github.com/1258982>

------
nateberkopec
BREAKING: glenjamin cures cancer.

------
Detrus
Too bad the code is so nasty

~~~
mattlong
How so? IMHO, it's an elegant demonstration of the async module.

~~~
wmf
Check out the Cilk version, which is not only concurrent but parallel as well:
<http://myxman.org/dp/node/182>

I agree with jerf that either you shouldn't have to worry about splitting your
computation at all or at least you should have syntactic sugar for it. The
Node solution has much more noise than code.

~~~
scooty-puff
<http://hpaste.org/52108>

------
Nemmie
I lolled (at work).

------
pabloPXL
lol ted is really mad :D <https://github.com/teddziuba/node-fib>

------
wavephorm
Now I just need an async ajax library, so I can do:

    
    
      fibonacci(40,function(f) {
        console.log(f);
      });

------
tedjdziuba
I've just submitted a pull request that I think really enhances this app.
<https://github.com/glenjamin/node-fib/pull/5/files>

~~~
wippler
wow.. this is incredibly juvenile. you should need to appreciate other
people's opinions too apart from yours

------
jrussbowman
hilarious

