

Plain english explanation of Big O - alrex021
http://stackoverflow.com/questions/487258/plain-english-explanation-of-big-o/487278#answer-487278

======
jacquesm
That's a very clear and accessible explanation of what big O complexity is all
about.

 _But_ (and that's a pretty big but), 'Big O' is not all there is, and once
you've picked your algorithm based on the one that has the best expected
runtime based on 'Big O', you _really_ have to try to make sure that:

    
    
      - your algorithm is executed with the lowest possible frequency
    
      - you concentrate on those pesky n's that you eliminated during analysis to 
        make sure that you don't end up wasting all your CPU time on some 
        little detail somewhere
    
      - you take in to account the effects of your code on caches and virtual memory
     
      - you profile your code afterwards to make sure that all your assumptions regarding 
        the above are correct
    

It is very easy to pick the 'right' algorithm and still get crappy runtime if
you skip those steps.

~~~
silvestrov
Big O only says something about n = Infinite. But n is never infinite in
computer programs.

Big O is a tool for making estimates, and like all other (such) tools, you
have to know its limitations to use it properly.

~~~
edanm
"Big O only says something about n = Infinite."

Err, no it doesn't. Big O is a tool for saying how quickly an algorithm runs
based on the size of n, and it's usually used to measure real things, like
what will happen when the filesystem has to handle 100,000 files, then
1,000,000, etc.

~~~
silvestrov
Nope. You ignore that the constant factor is removed (i.e. all factors but the
one that is dominant for n going to infinite).

More precisly, it tells how the relative speed difference between 2 algorithms
change when n goes to toward infinity. An O(n) will win over O(n^2), but the
latter case might have a huge constant factor that makes it slower than the
former for e.g. n <= million.

And very often, n is smaller than a million.

~~~
whimsy
This is true for O(n) but not for o(n). f(n) is the class of O(g(n)) of g(n)
if g(n) dominates f(n) for sufficiently large n. o(g(n)) strictly dominates
f(n), though, no matter how small n is.

Big O notation, or Landau notation, is not just O(g(n)). It's also o(), Ω(),
ω() and Θ(). It's understood, though, that n is considered to usually be
rather large - another name for this notation is, after all, "asymptotic
notation." In any case that you're really worried about speed, you should
probably be calculating the speed of your algorithm directly rather than using
mere asymptotic generalizations.

Aside: I can't think of any algorithms with huge constants like you described;
in theory, you're correct, but in practice, the asymptotic generalizations
apply for n <= 1000 or even often 100.

------
steamboiler
Interestingly the top rated answer was given by the gentleman who got turned
down by Google

<http://news.ycombinator.com/item?id=1520323>

~~~
joubert
I read his blog post (the entire one, which is unusual for me) and he writes,
early on, that Google discovered him via his prolific presence on Stack
Overflow.

------
gz
Knuth's take: [http://micromath.wordpress.com/2008/04/14/donald-knuth-
calcu...](http://micromath.wordpress.com/2008/04/14/donald-knuth-calculus-via-
o-notation/)

------
aj
Really good and simple explanation. Elegant examples to illustrate his points
as well.

------
beza1e1
Understand the effect of the constant factor hidden within the Big O. Often a
O(n) algorithm is faster than a O(1) algorithm, because n is too small. E.g.
arrays vs hashmaps.

Make sure what n is in each case. For example a graph algorithm with O(n^2)
and n being the number of nodes may actually be O(n) for n being the graph
size (number of edges).

------
erikpukinskis
Apparently computational complexity is no longer the first thing that comes to
mind when I read "Big O".

------
d0m
It starts with: "The simplest explanation" - and it goes on and on for 5
pages.

~~~
jrp
Simple is not short. A very short explanation could be given in formal
mathematics but it would not be simple.

------
zandorg
A friend of mine at University said the only true 'Big O' is Roy Orbison.

------
autarch
That anime was _really_ confusing. I'm not sure I could come up with a simple
plain English explanation.

Was it a dream? A computer simulation? An alternate reality? I mean, really,
wtf?

