
Plain English explanation of Big O - gs7
http://stackoverflow.com/a/487278
======
pmiller2
IMO, most of these explanations of Big O that I've ever seen have missed the
key feature of Big O (the "killer application" if you will): that it's
relatively independent of the exact computational model you use. Sure, you
have to decide on the broad strokes of your computational model, but you don't
need exact, instruction-level details.

This is a pretty huge benefit because it means every CS paper doesn't have to
start with a section introducing an abstract machine. The downfall is, of
course, that you lose some ability to estimate the potential real-world
performance of the algorithm, and the notation itself hides differences
between algorithms of the same Big O complexity.

~~~
sesqu
That's not a feature of O; that's simply a feature of leaving stuff out.

O is a simplification of a function, but says nothing about how to construct
the function. You need to have your (implicit) computational model before you
can start counting critical operations, whether you describe it or not.

This is actually somewhat important, since almost everyone has agreed on a
model that, among other things, considers random access, multiplication,
hashing, and even explicitly O(log log n) operations to take constant time.
It's one of the many reasons why people make incorrect proofs about P=?NP.

 _edit_ : If you meant that the simplification abstracts away implementation
details like 3n^2+42 vs 2n^2+n, you're right, that's valuable. But I still
argue that you ascribe it too much value when it comes to underspecifying your
computational model.

------
sillysaurus2
I'll try my hand at a "plain english" explanation of an O(n^2) algorithm.

It's a function which takes a list of numbers as inputs. You give it a list of
two numbers. It executes four operations before terminating.

You give it a list of three numbers. It executes nine operations before
terminating.

You give it a list of four numbers. It executes sixteen operations before
terminating.

You see how the time required grows as n^2 with respect to the input? That's
an O(n^2) algorithm, and that's why such an algorithm can be bad. A list of
1,000,000 numbers would require a trillion operations.

The way this can happen is if you write a double-for loop:

    
    
      for x in input:
        for y in input:
          ...
    

So, there you go. You probably won't get any plainer of an explanation without
becoming oversimplified and losing meaning.

~~~
adsche
It's plain, I'll give you that. And it gives you an idea. But it's not an
explanation as I think it already lost meaning. The explanation is closer to
o(n^2) (small o) or maybe Theta(n^2) but still lost the important aspect of
only being bound asymptotically.

------
gamegoblin
I always found the mathematical definitions of asymptotic notation the most
clear:

An algorithm with runtime r(n) is "big O" of f(n) if the limit as n ->
infinity of r(n) / f(n) <= some constant.

EDIT: See replies below.

~~~
teraflop
Almost; r(n) doesn't have to have that property, but it must be bounded above
by a function that does have that property. For instance, consider the
function r(n) = x*(sin(x)+1).

~~~
krcz
Or one can use lim sup.

------
ColinWright
There have not only been discussions of this before, some previous HN
contributions have provided significant criticism and enhancements. If you
value the advice and expertise of the HN community at all, you should consider
reading some of these earlier comments:

Here are the previous submissions with the most discussion:

[https://news.ycombinator.com/item?id=1520552](https://news.ycombinator.com/item?id=1520552)
(24 comments, 1242 days ago)

I found them by using this search:

[https://www.hnsearch.com/search#request/all&q=title%3A%28pla...](https://www.hnsearch.com/search#request/all&q=title%3A%28plain+english+explanation+big%29&sortby=create_ts+desc&start=0)

Here are some other submissions:

[https://news.ycombinator.com/item?id=695988](https://news.ycombinator.com/item?id=695988)
(4 comments)

[https://news.ycombinator.com/item?id=2344181](https://news.ycombinator.com/item?id=2344181)

[https://news.ycombinator.com/item?id=3807175](https://news.ycombinator.com/item?id=3807175)

[https://news.ycombinator.com/item?id=3846993](https://news.ycombinator.com/item?id=3846993)

[https://news.ycombinator.com/item?id=5164236](https://news.ycombinator.com/item?id=5164236)

[https://news.ycombinator.com/item?id=5636683](https://news.ycombinator.com/item?id=5636683)

[https://news.ycombinator.com/item?id=5778469](https://news.ycombinator.com/item?id=5778469)

This submission of the same item has some discussion, but mostly of the fact
that this item is submitted so often, whether it's a problem, and whether we
should do something about it, possibly to make HN a better source of curated
discussion:

[https://news.ycombinator.com/item?id=5785523](https://news.ycombinator.com/item?id=5785523)
(24 comments)

The answer seems to be "No".

Additionally, you might be interested in reading these:

[https://news.ycombinator.com/item?id=4655061](https://news.ycombinator.com/item?id=4655061)
: Big-O Misconceptions

[https://news.ycombinator.com/item?id=5770232](https://news.ycombinator.com/item?id=5770232)
: What does O(log n) mean, exactly?

~~~
chris_wot
Thank you!

------
podperson
This is a fine example of Stack* giving huge rankings to very bad answers. Big
O is about execution time not complexity. (And describing execution time as
"time complexity" as some comments do is ridiculous. A lot of something is not
more complicated than a little of something. 4 isn't more complex than π --
and no, I'm not saying the reverse is true either.)

A plain English explanation of Big O notation shouldn't take more than a few
sentences. It's not that complicated, especially if you don't get into nitty
gritty.

~~~
oskarkv
Big O is not necessarily about execution time. Also, why is it ridiculous to
describe execution time as "time complexity"? From
[http://en.wikipedia.org/wiki/Analysis_of_algorithms](http://en.wikipedia.org/wiki/Analysis_of_algorithms)
"Usually, the efficiency or running time of an algorithm is stated as a
function relating the input length to the number of steps (time complexity) or
storage locations (space complexity)."

~~~
haberman
> Also, why is it ridiculous to describe execution time as "time complexity"?

It's unnecessarily different than the normal definition of "complexity." When
you say "complex" in normal conversation, you mean "complicated" or "hard to
understand." You would never say "the time complexity of getting to the
airport is 30 minutes."

It also begs the question. Wrapping your head around the idea of asymptotic
complexity is the hard part of understanding big O. Defining big O in terms of
complexity doesn't help if you don't understand the concept of asymptotic
complexity yet.

I think the best way to explain this, by far, is with a graph.

------
pyk
Interesting... although his explanation of TSP complexity is unfortunately
incorrect (or misstated). The complexity he is describing is specifically for
the naive brute force search [1] algorithm for finding the shortest path for
TSP. And a 200 city TSP is quite solvable, and often times even by a mobile
device like an iPhone 5 [2]! Indeed, no need for all the time in the universe
-- lucky for companies like UPS who utilize TSP variants regularly.

[1]
[http://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common...](http://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common_functions)

[2]
[http://www.math.uwaterloo.ca/tsp/iphone/](http://www.math.uwaterloo.ca/tsp/iphone/)

------
fragsworth
Suppose we have a human-written program that's N-bits long. It takes an input,
and generates an infinite output by performing some hashing of the input. We
run it on an M-bit input to get some output, then stop it manually.

Now suppose we have an input and the output made by this program. But we don't
have the program. Suppose we want to discover by brute force what the program
was.

The brute-force algorithm will create every possible program up to N bits
long, and run each one until either a difference in outputs is found, or it
found the correct output, or too much time has passed.

How do you come up with the order-notation for this brute-force algorithm?

~~~
umanwizard
Since you're bounding (with a constant) the time each iteration can take, this
is just O(2^n).

EDIT: I suppose if the time bound per program is a parameter "m" that you can
vary, then it's actually O(m*2^n).

Without the bound, this algorithm is unlikely to work, since programs can run
indefinitely.

If you modify your algorithm to only generate programs without any infinite
loops, you've solved the halting problem, which is impossible.

~~~
reitzensteinm
I'm sure you know this, but I'd just like to clarify something:

You absolutely can generate only programs that terminate; for instance, throw
out any that have recursion or unbounded loops, or use a non Turing complete
language that is guaranteed to terminate.

That you can't solve the halting problem in general does not mean that static
analysis is fruitless. The answer to "does this terminate" will be yes, no or
maybe, almost always in the latter category.

The correct thing to say would be:

If you modify your algorithm to only generate programs without infinite loops,
you've most likely thrown out the correct solution.

------
spiritplumber
I loved that cartoon!

Oh, wait...

------
bitops
The author of this answer, cletus, is also a member of this community and has
quite a high karma.

See his content here:
[https://news.ycombinator.com/user?id=cletus](https://news.ycombinator.com/user?id=cletus)

------
lelf
Aaaand it's wrong like 99.9% other explanations:

f(n)=O(n^2) is actually NOT f(n) asymptotically grows as fast as n^2, it is f
grows “no faster” than n^2.

“As fast as” is f(n)=Θ(n^2)

Edit: oh, and Big-O is not “a relative representation of the complexity of an
algorithm”

------
twelvechairs
a lot of words for "the larges number of times a loop/operation might have to
run, for a list of size 'n'"

honestly the language of maths too often obfuscates rather than simplifies

~~~
10098
> honestly the language of maths too often obfuscates rather than simplifies

This is wrong on so many levels, I don't even know where to begin.

Mathematical formulas and equations are the ultimate tool for simplification
and abstraction, the best that humankind could come up with. Explaining
mathematical concepts _precisely_ in natural language can be extremely
difficult and time-consuming. The only problem is, you have to _learn_ that
language, just like any other language. Otherwise, it's like complaining that
Japanese "obfuscates" things because you can't read a sentence in Japanese.

~~~
kitsune_
This is true, however, mathematical notation is often intrinsically linked to
its historical origin and might not be the optimal solution from a usability
perspective.

------
ChristianMarks
Speaking of plain English, Knuth's use of Big-Oh notation for asymptotics
brings to mind another domain of human activity altogether.

------
vdhus
[http://qr.ae/GwtSY](http://qr.ae/GwtSY)

------
robobro
The SICP does some fun puzzles in thinking like this.

------
10098
This is at least a 3rd degree repost.

------
ananth99
Thanks for sharing!

