Hacker News new | past | comments | ask | show | jobs | submit login

Its interesting that even in this thread the 2 answers differ by an order of magnitude lol



Just apply it recursively. Let’s get 100 samples of comments suggesting the number of samples to use. Then average those.


FWIW, the threshold I learned was 20 in each bucket, so now you have 3 answers.


Eh it's really the same rule, just applying a different threshold.


The problem is that the sensitivity to the number growth is supposed to be exponential. So if you need 100 samples for "within 10% of the value", then 10 samples should give you almost completely random behavior.

In reality, it depends on your actual distribution, but the OP from this thread here is unreasonably conservative for something described as a "rule of thumb". Almost always, if you have at least 10 of every category, you can already discover every interesting thing that a rule of thumb will allow. And you probably could go with less. But if you want precision, you can't get it with rules of thumb.


The dependence on sample size is not exponential, it's sublinear. The heuristic rate of convergence to keep in mind is the square root of the sample size, i.e. getting 10x more samples shrinks the margin of error (in a multiplicative sense) by sqrt(10) ≈ 3ish.

The exponential bit applies to the probability densities as a function of the bounds themselves, i.e. how likely you are to fall x units away from the mean typically decreases exponentially with (some polynomial in) x.

Of course, this is all assuming a whole bunch of standard conditions on the data you're looking at (independence, identically distributed, bounded variance, etc.) and may not hold if these are violated.


I think both answers are referencing the Central Limit Theorem, that states [simplified] that once you get over 30 samples for each independent variable, you will get a normal distribution.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: