
The Spacing of Binary Floating-Point Numbers - wglb
http://www.exploringbinary.com/the-spacing-of-binary-floating-point-numbers/
======
userbinator
There's a related article on the site about the spacing of _decimal_ floating-
point, which most people would probably find more intuitive:

[http://www.exploringbinary.com/the-spacing-of-decimal-
floati...](http://www.exploringbinary.com/the-spacing-of-decimal-floating-
point-numbers/)

This spacing is also why generating a random floating-point number the "naive"
way will result in a non-uniform distribution.

~~~
onnoonno
What is the naive way? Interpreting a random bit string as a float? Or rather
randint(1000000000L)/1.0e9?

~~~
emilga
The naive way is presumably to just throw all the floats in a hat, mix them
up, and pull one out. This is the method you described.

The problem here is that the floats clump together around zero, and they
spread out further away from zero. So the hat-method is biased towards smaller
numbers.

A possible better method -- this is just an educated guess -- might be to
divide your interval in half, pick the left or right half randomly, and then
divide the resulting interval in half, etc. until your interval contains only
one float. I'm not sure how efficient this would be to implement, or if there
are any edge-cases to consider, but it would at least avoid the inherently
biased distribution of the floats.

~~~
onnoonno
Hmm, but the randint(...)/... method should come pretty close to a uniform
distribution on the 0..1 interval, shouldn't it?

It wouldn't have maximum entropy, though.

------
amelius
I just wish I could replace my floating point numbers by an arbitrarily more
precise version of them (no matter what the cost in terms of performance), in
order to simply test the hypothesis that rounding errors are causing, for
example, convergence problems.

~~~
simonbyrne
I actually do this quite regularly in Julia: it provides a BigFloat type which
wraps the MPFR library. Since it has generic methods, you can simply do
something like foo(big(x)), and all the internal computations will be done in
extended precision, and the result can be compared to foo(x).

~~~
amelius
Does this also work with matrix type of operations? For example solving a
linear system?

~~~
simonbyrne
Yes, you can define an Array of BigFloats, and most of the linear algebra
operations have general purpose fallbacks that work as expected.

------
chrisBob
I really didn't understand floating point numbers until I wrote an app that
displays the pieces and the equation[1]. It even shows the min increment to
the next possible value. Sorry it's still iPad only. If more than 10 people
ever download it I will take the time to make it universal.

[1] [https://itunes.apple.com/us/app/float-
explorer/id928900898?m...](https://itunes.apple.com/us/app/float-
explorer/id928900898?mt=8)

~~~
jstanley
Out of interest, why is this an app and not a web page?

~~~
chrisBob
I am most comfortable with Objective-C. I agree that this would be better
suited to a web-app, but I don't have the time to invest in learning a new
language to do it at the moment.

