
On calculating Fibonacci numbers in C - avibryant
http://blog.noblemail.ca/2013/01/on-calculating-fibonacci-numbers-in-c.html
======
xyzzyz
It's interesting to note that one can get O(lg (n)) complexity of analytical
solution without using any floating arithmetic. There are at least two easy
ways to do it: one is to recognize the analytical formula as a result of
diagonalizing certain linear map, and calculate nth power of that map using
repeated squaring instead of diagonalization. In more concrete terms: consider
a square matrix A = {{1,1},{1,0}}. To obtain nth Fibonacci number, raise A to
the nth power and take lower left entry. Clever multiplication by repeated
squaring lets one perform only a logarithmic number of matrix multiplications
instead of linear number using naive method.

Another way is to interpret analytical formula in Q(sqrt(5)) ring, and
implement Q(sqrt(5)) arithmetic using quadruples of integers. More concretely:
suppose we have two numbers of the form a+b sqrt(5), c+d sqrt(5), where
a,b,c,d are rational. then their sum is (a+c) + (b+d)sqrt(5), and their
product is (ac+5bd)+(ad+bc)sqrt(5). Thus, just like we usually represent
complex numbers as pairs of real numbers with special multiplication formula,
we can represent elements of Q(sqrt(5)) as pairs of rationals with special
multiplication formula. then we just use the analytical formula for our
special number system. I've implemented it a while ago in Haskell:
<https://github.com/xyzzyz/FibBinet>

~~~
stephencanon
While I agree with the gist of your comment, it's important to note that
_none_ of these give an O(lg(n)) algorithm for computing fibonacci numbers
(except on a hypothetical computer that can do arbitrary size integer
operations in O(1) time).

The nth fibonacci number has O(n) digits, so you can't even write it out in
time less than O(n). The repeated-squaring approach will involve multiplying
numbers with O(n/2) = O(n) digits in its final step, so a very coarse analysis
says the complexity is at least O(n lg n).

~~~
blackhole
This is completely missing the point of Big-Oh notation. If you start trying
to analyze how long integer operations take, _ALL_ your computational
complexity analysis falls apart for any sufficiently large integer arithmetic
for any algorithm. This is precisely why such operations are ignored, because
it becomes completely impossible to do any meaningful platform-independent
analysis if you try to take that into account.

That's not to say you _shouldn't_ take it into account, but the algorithm he
described is, in fact, O(log(n)) time. It just is. That's how the math works.
Now, you should note that the O(log(n)) time constraint does not take into
account arbitrary integer size which will cause progressively larger constant
factors to influence the efficiency of the algorithm... but it is still a
O(log(n)) algorithm.

~~~
stephencanon
My analysis holds for any implementation for which you can perform O(1)
addition and multiplication of integers of some fixed-size. That's just about
as platform-independent as you can get, but it still captures the _real_
complexity of the operation in a way that handwavy "pretend you can do
arbitrary arithmetic in O(1)" analyses do not.

This isn't simply my viewpoint on the subject; even the very theoretical
textbook by Papadimitriou and Vazirani takes this approach to the question
(see exercise 0.4 and note that everything is carried out in terms of the
complexity of arbitrary-size integer multiplication M(n)).

------
jdreaver
I still don't understand how the author jumped from a cute comparison of
Fibonacci algorithms to saying programmers shouldn't learn more math. Math
isn't just arithmetic and numbers. He acts as if the recursive solution isn't
also math.

Developing mathematical maturity is, in my opinion, an excellent way to think
more clearly, precisely, and rationally about any problem, even if it doesn't
necessarily involve numbers. The author could make the case that _number
theory_ in particular isn't directly applicable to many problems (I do have a
lot of fun with Project Euler though), but again, math isn't just about
numbers. Math teaches you how to reason about the _relationships and patterns_
between _objects_ (which can be numbers, vectors, spaces, groups, rings, etc.
In a certain sense, you become more comfortable with higher levels
abstraction, and it requires you to be precise and explicit about assumptions.

That's jut my two cents. Math education is pretty dismal in the US, but math
can be very enjoyable if you find the right textbook, teacher, or even free
online class nowadays. Never fear math.

EDIT: The author has pointed out that he is making a play on words between
"math" and "Math" (the mathematical library). I didn't understand that when I
wrote this post.

~~~
snoble
I do distinguish between math and Math. Decide for yourself what that
distinction means.

~~~
jdreaver
I reread the article with that in mind, and it makes much more sense, to the
point that I am inclined to agree with much of what you said now. I think you
should highlight that difference closer to the beginning of the article. That
is a nice, subtle play on words.

~~~
snoble
how much emphasis to give to which messages is always the hardest part for me
to writing these posts. I'll keep in mind that at times I can be too subtle.

------
just2n
Re-commenting on OP @ author's request:

The article really isn't about speed, it's more that engineers shouldn't
necessarily take mathematical results as the end-all to their considerations
when developing an algorithm. In particular, what is considered "best" in
theory is not necessarily the best choice in a given program.

Math is important to formulating algorithms, since it can provide, in theory,
a closed form solution for something typically done iteratively (the example
here: fibonacci numbers). But you still need to verify when an algorithm is
going to be correct, and more importantly, appropriate. In this case, it's
pretty well known how inaccurate floats and doubles can be within X
significant figures, and this is why I've always said the analytical solution
doesn't really work on a computer unless you have a CAS library that can
approximate an exact answer to an arbitrary number of significant figures (and
it still won't be accurate if shoved into a float/double afterwards). So
there's a trade-off, and choosing the theoretically best solution (a closed
form solution) isn't necessarily the best possible choice for the task. In
this case, yes, as others have pointed out, if we only need the first 96
fibonacci numbers, ever, it makes more sense to pre-compute them. However, if
we only do the computation once in the entire program, even that doesn't
necessarily make sense. It entirely depends on the application, but that's the
entire point: a closed form for fibonacci numbers isn't the end of
considerations.

I am curious though, why is it so many people are oblivious that there's an
O(logN) time algorithm to compute fibonacci numbers iteratively on integers?
It's a beautiful piece of math, and very simple so everyone should understand
it. Just notice that:

    
    
        ((1, 1), (1, 0))^N -> ((f_{N+1}, f_N), (f_N, f_{N-1}))
    

given N > 0, where f_N denotes the Nth fibonacci number. For SW engineers who
are actively interviewing: does anyone still ask you to write a function to
compute the Nth fibonacci number in technical interviews?

~~~
dbaupp
(The author probably meant on the actual blog post, rather than at the top
level on HN. :) )

~~~
just2n
If that's the case, I apologize, but I feel like a victim of ambiguous
terminology!

~~~
snoble
I'm the king of using ambiguous terminology (I wish I were clever enough to be
ambiguous right now).

------
protomyth
"....since the 96th Fibonacci number is the largest to fit in an unsigned long
int"

Have to agree with the first person to comment on the article's site. If we
are going for speed, just pre-calc the 96 numbers and do an array lookup in
the function guarded by and if.

~~~
just2n
The article really isn't about speed, it's more that engineers shouldn't
necessarily take mathematical results as the end-all to their considerations
when developing an algorithm. Math is important to formulating algorithms,
since it can provide, in theory, a closed form solution for something
typically done iteratively (the example here: fibonacci numbers). But you
still need to verify when an algorithm is going to be correct. In this case,
it's pretty well known how inaccurate floats and doubles can be within X
significant figures, and this is why I've always said the analytical solution
doesn't really work on a computer unless you have a CAS library that can
approximate an exact answer to an arbitrary number of significant figures (and
it still won't be accurate if shoved into a float/double afterwards). So
there's a trade-off, and choosing the theoretically best solution (a closed
form solution) isn't necessarily the best possible choice for the task. In
this case, yes, if we only need the first 96 fibonacci numbers, ever, in our
program, it makes more sense to pre-compute them. However, if we only do the
computation once in the entire program, even that doesn't necessarily make
sense. It entirely depends on the application, but that's the entire point: a
closed form for fibonacci numbers isn't the end of considerations.

I am curious though, why is it so many people are oblivious that there's an
O(logN) time algorithm to compute fibonacci numbers iteratively on integers?
It's a beautiful piece of math, and very simple so everyone should understand
it. Just notice that:

    
    
        ((1, 1), (1, 0))^N -> ((f_{N+1}, f_N), (f_N, f_{N-1}))
    

given N > 0, where f_N denotes the Nth fibonacci number. For SW engineers who
are actively interviewing: does anyone still ask you to write a function to
compute the Nth fibonacci number in technical interviews?

~~~
protomyth
I get the point of the article, but it seems like defaulting to a compiler
optimization that might not be in other compilers is a poorer solution than
examining the problem for another Math solution. You yourself have shown one.

Computer precision is a problem and programmers should learn a lot about how
ints and fp work, but throwing out all Mathematical solutions is really not a
good idea.

------
Strilanc
The analytic solution to the Fibonacci sequence is nice and all, but I
generally dislike methods that require you to very carefully check how much
precision you need. I much prefer the matrix exponentiation by repeated
squaring approach [1].

1: [http://ronzii.wordpress.com/2011/07/09/using-matrix-
exponent...](http://ronzii.wordpress.com/2011/07/09/using-matrix-
exponentiation-to-calculated-nth-fibonacci-number/)

------
alexkus
GMP (arbitrary precision library) implements it as a lookup table for "low" n
but uses a different set of identities for larger n.

<http://gmplib.org/manual/Fibonacci-Numbers-Algorithm.html>

------
aeon10
If im not mistaken the iterative solution of using a simple for loop would
definitely be insanely faster than a recursive algorithm which is
exponential.. What am i missing here?

~~~
dmlorenzetti
The recursive approach isn't necessarily worse, provided the compiler can do a
so-called "tail call optimization" (in the original posting, note the line
that says of the compiler optimization level, "We need -O2 so gcc will
recognize the tail-call.").

Tail-call optimization says that if a function directly returns the result of
calling another function, then you don't need to add to the call stack in
order to effect the function call-- you just modify the current stack frame as
needed. For a recursive function, apparently a good compiler can essentially
transform this into code very similar to what you'd get with an explicit loop.

None of this means that a simple for-loop wouldn't be appropriate, or even
faster. Personally I use for-loops a lot, and, like you, I would naturally
incline to writing this using a loop rather than recursion.

I see yours is a new account, so maybe this is just a cultural thing that I've
got used to. To generalize grossly, recursion is more fashionable on HN than
iteration. Every once in a while you'll see a post here that touches on
iteration-- like maybe somebody will mention the tradeoffs between counting
down and counting up a loop variable. But guaranteed at least once a week
you'll see something on recursion and functional programming.

My impression is that most people here understand that a simple for-loop will
get the job done, but they don't see for-loops as very interesting, and don't
figure it adds anything to the discussion to bring it up.

Meanwhile, there are people here who will actively advocate against using for-
loops. There's a whole theory of functional languages that says that if your
program doesn't directly modify variables, then it's easier to reason about. A
for-loop modifies the trip variable, so it's not a functional construct.

~~~
aeon10
Yes, I understand that recursion is much more appealing and leads to much
cleaner and elegant code than iterative solutions. and thank you for the
introduction to the community I suppose.

Anyway I forked the original code, added an iterative version based on the
comment by (Nick Craig Wood)
(<https://github.com/AeonAxan/benchmarking_fibonacci>)

Here are the benchmarks [recursive fib: 7,765 ms] | [iterative fib: 8,047 ms]
| [analytic opt fib: 54,875] | [analytic fib: 103,937 ms] |

And it appears that the recursive version is faster than the naive iterative
version by a few hundred milliseconds.

~~~
aeon10
Just a note, In the iterative solution, by changing the i (loop variable) from
an int to an long int the times by both iterative and recursive are almost
equal. Here are the new results

[Recursive - 7921 ms], [Iteratiive - 8078 ms]

So the difference is negligible.

------
dxbydt
The author very casually takes a potshot at issue #41, calling it a "really
silly time wasting debate". Was #41 honestly such a silly time waster? Pray,
what would you have done with your precious time instead? I for one took away
a lot from #41. It was a very valuable learning experience. #41 was definitely
way more valuable than computing fibonaccis, if I may say so.

------
orionblastar
I used to use the two bucket method to generate a Fibonacci number, thanks for
showing me another way to do it.

I am working on Ackermann Numbers now, but it takes a very long time to
generate them and I have to either write my own classes to handle big numbers
or using a different library to use large numbers. I moved from C to C++ to do
that. Your unsigned long int variable would run out of room after the 96th
Fibonacci number, and you might have to move to C++ and use a custom class as
well to be able to handle those big numbers.

~~~
dalke
Note that the author did write "since the 96th Fibonacci number is the largest
to fit in an unsigned long int".

------
mathattack
I came to the exact opposite conclusion as the author. Programmers should know
a lot of things, including both recursion and math since tere are many ways to
approach a problem.

------
tyler-dodge
Wait Im confused. Wouldnt the point of using the analytic approach be that i
could start at any n besides 0? Because of course if im just taking the sums
from 0 to n it'd be faster to do it the tail recursive way because thats just
simple addition.

~~~
dubya
The post didn't go into this, but the sum has a lovely closed form as well.
The sum of the (phi^n-psi^n) terms is the difference of two geometric series,
so you can apply sum(r^i, i=0..n)=(r^(n+1)-1)/(r-1) to get a formula that
evaluates the sum exactly.

------
pmb
Only the first 90ish fibonacci numbers fit in a 4 byte int. Use a lookup
table, make it O(1).

