

NaNs Just Don't Get No Respect - stianan
http://www.drdobbs.com/article/print?articleId=240005723&siteSectionName=cpp

======
tikhonj
NaNs are annoying because, thanks to them, equality on floating point numbers
is not an equivalence relation. In particular, NaN /= NaN.

This means that in Haskell, for example, you cannot really rely on the Eq
class representing an equivalence relation. Code relying on the fact that x ==
x should be true for all x could not work as expected for floating point
numbers.

I don't know if this has any practical ramifications in real code, but it
certainly makes things less elegant and more complex than they have to be.

~~~
joshAg
Attmepting to just use simple equivalence with any floating point type a
horrible idea to begin with, even without NaNs. You should instead declare
equivalence if the absolute difference between the two numbers is less than
some bound based on what you're doing and not look for bit equivalency.

However, you should be able to examine two NaNs and declare them "equivalent"
(for certain definitions of equivalence) by intelligently examining the bits
based on the hardware that you're running the program on. In the case of a
binary Nan [1] that would entail checking that the exponential fields are both
entirely high (eg 0x8 == (a.exponent & b.exponent), assuming a standard 8 bit
exponent) and that the mantissas are nonzero (eg a.mantissa && b.mantissa).

[1]: "Binary format NaNs are represented with the exponential field filled
with ones (like infinity values), and some non-zero number in the significand
(to make them distinct from infinity values)."
--<http://en.wikipedia.org/wiki/NaN>

~~~
shrughes
That's not true, there are plenty of cases where using equivalence is just
fine. Integer arithmetic, and algorithms that are more reliably written not to
contain any empty intervals are two examples.

~~~
noblethrasher
He was specifically talking about equivalence on floating point types.
Integers don't have or need NaN.

~~~
shrughes
I was talking about integer values (with floating point representation) being
multiplied and added (and divided and floored, I suppose).

~~~
joshAg
even just addition and multiplication with floats make simple equivalence a
horrible idea, due to uncertainty.

For example:

    
    
        float a = 1.0;
        float b = 1000.0;
        for (int i = 0; i < 1000000; ++i)
            a+=1.0;
        b *= b;
    

There is no guarantee that a == b. Floats make everything more complicated,
even simple addition: <http://en.wikipedia.org/wiki/Kahan_summation_algorithm>

~~~
shrughes
There is no uncertainty. There is a guarantee that a == b (if we ignore the
off-by-one error in your post), because IEEE operations are guaranteed to be
accurate within half an ulp. You can safely perform addition, subtraction, and
multiplication, and truncated or floored division, within the 24-bit integer
range for single-precision floats and the 53-bit integer range for doubles.
This is why people can safely use integers in Javascript.

~~~
joshAg
i guess that's what a i get for not double checking my math. here's a revised
version that (as long as i haven't made any other math mistakes) still fits
within a 32 bit signed int but doesn't guarantee simple equality:

    
    
        float a = 0.0;
        float b = 10000.0;
        for (int i = 0; i < 100000000; ++i)
            a+=1.0;
        b *= b;
    

why in the world would you use a float instead of an int for addition,
subtraction, and multiplication, and truncated or floored division, within the
24-bit integer range? it seems like there's no benefit to offset the facts
that floating point operations are slower than integer operations and that
ints can store integers 7 or 8 bits larger.

and what happens when you go beyond 24 bits? since it's a float no error or
warning will be thrown, but now equivalence won't work for numbers that are
easily stored by an int.

~~~
shrughes
Why aren't you capitalizing your sentences? Are you too lazy to write
properly?

Where did I say I'd use floating point numbers for integer math? Yes, let's
move the conversation to a direction it never existed so that you can pretend
you were right.

(The place I'd use it would be in a Javascript implementation, or a Lua
implementation, and other situations where I'm designing a programming
language where I want the simplicity of having only one numerical type. And
that would be a 53-bit range, not 24-bit.)

~~~
joshAg
You said using equivalence for floats that store integers is fine. here is a
link: [1]. The point of my example was to show that that is not the case for
numbers that are easily stored by an int that's the same size as a float.

I've not used lua or javascript, but wouldn't it be better to choose an
implmentation that silently switches between ints, floats, doubles and big-
nums as needed? That way you don't limit the speed float and int operations by
autoconverting up to doubles, when double precision isn't needed.

[1]: <http://news.ycombinator.com/item?id=4400424>

~~~
shrughes
I do not recommend using floating point numbers for integer math. I am saying
that _if_ you have integers stored in floating point representation, equality
comparisons are fine.

> I've not used lua or javascript, but wouldn't it be better to choose an
> implmentation that silently switches between ints, floats, doubles and big-
> nums as needed?

If you want to go through the engineering effort, sure, it might make sense to
have separate int and double encodings. You have to check for the possibility
of integer overflow and convert to double in that case. In particular, this is
useful if you want to use Javascript's bitwise operators. See
[https://developer.mozilla.org/en-
US/docs/SpiderMonkey/Intern...](https://developer.mozilla.org/en-
US/docs/SpiderMonkey/Internals#JavaScript_values) for a description of how
SpiderMonkey does it. Lua just has one representation for numbers, double
normally but it could be float or long depending on how you compile it. There
would be no reason to have single-precision floating point or big-num
representations.

------
malkia
There are much worse thing than NaN's.

They are called denormals. These appear when dealing at the same time with
lots of big numbers (very far away from 0) in operations with lots small
numbers (close to 0).

In such cases the FPU (or whatever deals with fp numbers), switches to a
format that could be very inefficient producing an order of magnitude slower
operations.

For example when dealing with IIR filters in audio, your audio buffer might
contain them. One of the solution is to have a white noise buffer somewhere
(or couple of numbers) that are not denormalized and add with them - it would
magically normalize again.

I'm not a guy dealing with "numerical stability" (usually these are physics,
audio or any simualation engine programmers), but know this from simple
experience.

~~~
zurn
Denormals are part of IEEE fp. If your implementation is too slow, you can
often trade correctness for speed by turning them off in the C/C++ runtimes.

They're also a sign you're skirting on the limits of FP precision (or worse)
so a bit of numerical analysis might still be a good idea...

~~~
malkia
You cannot simply turned them off everywhere. On certain platforms they are
produced always, and nothing can be done, but openly deal with them (by
expecting them to happen).

------
saurik
The things this author likes about NaN are also properties of NULL in many
environments (that NULL cannot be compared to NULL, that operating on NULL
returns NULL, etc.); so while you might not see many languages default
initializing things to NaN, you do see them default initializing things to
NULL with similar effect.

~~~
duaneb
Except this is actually worse, since there are many possible values which
evaluate to NaN.

EDIT: I do not know how D implements NaNs; they may have magic to make them
more sane to work with.

~~~
WalterBright
D does not implement NaNs, it just relies on the IEEE FP hardware to do it.

What D does do is expose NaNs so the programmer can rely on their existence
and use them in a straightforward manner.

------
roryokane
An alternative workaround to writing `float f = 0` in languages without NaN:

    
    
        float f;
        bool thingIsFoo = condition1; // store the result…
        if (thingIsFoo)
            f = 7;
        // ... code ...
        if (thingIsFoo && condition2) // and explicitly depend on it later
            ++f;
    

But this causes an extra `&&` to be computed at runtime, so it seems NaNs are
still better for this case.

------
mieubrisse
You've written quite the interesting and informative article, and your logic
as for why you initialize to NaN was perfectly clear.

------
klodolph
I've gotten a bit pissed at the Microsoft C compiler for (1) having no
standard way to generate NaN or Infinity and (2) having a good enough static
analyzer that if you generate one by casting, it emits a warning saying that
your arithmetic overflows.

Gee, thanks MSC. I didn't expect "x = INFINITY;" to overflow.

~~~
malkia
0/0 should be NaN, 1/0 should be +Infinity, -1/0 should be -Infinity. (I
haven't tried that in a while).

Also check the flags, like /fp:precise for MSVC

------
tzs
URL for people without bionic eyes: [http://www.drdobbs.com/cpp/nans-just-
dont-get-no-respect/240...](http://www.drdobbs.com/cpp/nans-just-dont-get-no-
respect/240005723)

------
voyou
Stop trying to make D happen. It's not going to happen.

------
malkia
Be afraid of QNaN the Barbarian!

