

How Java’s Floating-Point Hurts Everyone Everywhere (1998) [pdf] - llambda
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf

======
pcwalton
This is a rant we came across in the early design of Rust and eventually
decided was pretty misguided. Very few languages actually address these
complaints.

The biggest complaint here is that, in Java, when you perform arithmetic on
32-bit floats you perform 32-bit arithmetic with 32-bit intermediate results.
Java's behavior is, in other words, what you'd expect. In C, though, if you
perform arithmetic on 32-bit floats you're actually performing 64-bit
arithmetic on 64-bit intermediate results (at least most of the time; it's
been a while since I consulted the rules). Java's behavior (which basically
amounts to doing what you asked to do) infuriated the author, so he gave a
bunch of examples of scientific calculations that need 64 bits of precision.
But that's totally unconvincing to me: if you want 64-bit floating point
precision, use doubles. C's implicit conversions in general have caused more
harm than good and what the author is complaining about is having to tell the
compiler to do 64-bit arithmetic on 32-bit values instead of having it follow
complex promotion rules that few programmers even know about.

~~~
ubasu
Note that this "rant" is by Velvel Kahan [1], who won the Turing Award for
designing the IEEE 754 floating point standard [2] (among other things), which
is used for specifying floating point computations on almost all computers
today. So perhaps he has a little more context on this than yourself.

[1]
[http://en.wikipedia.org/wiki/William_Kahan](http://en.wikipedia.org/wiki/William_Kahan)
[2]
[http://en.wikipedia.org/wiki/IEEE_754-2008](http://en.wikipedia.org/wiki/IEEE_754-2008)

~~~
pcwalton
I'm not questioning the author's credentials, nor do I even think he's wrong
about what's most useful in his field (scientific computing). I just think
that programming language design considerations (not wanting to admit implicit
conversions) outweigh his objections.

~~~
acqq
If you claim that Java was right to implement that so and the next language
who ignores his complaints is also, then just accept that both Java and the
next language have weaknesses expressing the computational tasks, from the
point of view of the people who do have to write that kind of code.

Note that for a very long time Windows didn't use FPU at all. They also
"didn't need it." And I think they started to think differently once they
looked enough what Apple does. A lot can be programmed without FPU, but the
edge in the industry is gained by using the hardware fully. Not doing what's
easiest.

And the FPU hardware doesn't introduce penalties when reading 32-bit values
and doing 64-bit computations.

~~~
pcwalton
That example is not comparable. The issue is not whether something is
_expressible_. All 32-bit and 64-bit arithmetic operations are expressible in
both C and Java (although I'd argue that Java makes it easier to know which
you're getting). The issue is just what the default should be, and whether
introducing temporaries should change the value of an expression. This is
impossible to measure objectively, because we're talking formally about the
exact same set of operations, just about which default is more
intuitive/convenient/leading to fewer bugs/etc.

~~~
acqq
I don't agree that it's "impossible to measure objectively." The practice to
store floating values with less bits but calculate with more is very old,
specifically, older than C. It's not accidental, it survived because it's
_needed._ As long as we don't have infinite memory and want to have huge
inputs and outputs we'll need 32-bit floats.

And as long we want to calculate with less error, and we need it, we'll need
doubles as intermediate calculations.

~~~
pcwalton
None of this is relevant to my point. You can use doubles for intermediate
calculations in C, Java, and Rust. The only difference is what the defaults
are.

~~~
bluecalm
Is the group using your choice as default sizeable? it's hard to imagine it's
comparable to all the people who do numerical computations assuming standard
behavior. Now all the algorithms will produce different results and the onea
in Rust will be less accurate. I don't think you can reasonably expect to
avoid a lot of pain with this choice. it would be nice to at least have
compiler flag or something instead requirement to write explicit casts
everywhere.

------
unwind
This was just tweeted by John Carmack
([https://twitter.com/ID_AA_Carmack/status/392298557631254528](https://twitter.com/ID_AA_Carmack/status/392298557631254528))
which is probably why it was submitted.

It's pretty old, it should have a [1998] tag in the title in my opinion.

It's also pretty funny. :)

~~~
daurnimator
*2004

~~~
milliams

      Originally presented 1 March 1998
          at the invitation of the
        ACM 1998 Workshop on Java for
      High–Performance Network Computing
         held at Stanford University
    

Those exact slides might be from 2004 but the content is from 1998.

------
betterunix
If 95% of programmers have misconceptions about floating point arithmetic, _we
should stop using it as the default rational / real number type_. Floating
point is a great optimization, but in a lot of situations it would be a lot
better to have arbitrary-precision rational number types, continued fractions
representations, or something else that better approximates what programmers
expect.

~~~
ErsatzVerkehr
These alternatives would all have their own pitfalls.

------
dekhn
I dunno, given what Javascript foisted on a generation of programmers, we
should be thankful Java is only as bad as it is.

Go Kahan!

------
acjohnson55
I browsed to page 10, but didn't learn what the big problem was, and then gave
up.

~~~
amock
There are five points on page 3:

1\. Linguistically legislated exact reproducibility is at best mere wishful
thinking.

2\. Of two traditional policies for mixed precision evaluation, Java chose the
worse.

3\. Infinities and NaNs unleashed without the protection of floating-point
traps and flags mandated by IEEE Standards 754/854 belie Java’s claim to
robustness.

4\. Every programmer’s prospects for success are diminished by Java’s refusal
to grant access to capabilities built into over 95% of today's floating-point
hardware.

5\. Java has rejected even mildly disciplined infix operator overloading,
without which extensions to arithmetic with everyday mathematical types like
complex numbers, intervals, matrices, geometrical objects and arbitrarily high
precision become extremely inconvenient.

~~~
kaoD
#1-#4 don't give much information about the real problem.

#5 is just a matter of convenience. I'm sure the Java team gave a long thought
to adding operator overloading, and I can see why they choose not to include
it (Java is meant to be straightforward, with as little magic as possible).

The first 20 pages or so are too much "blahblah" about Java, not much about
how it's hurting everyone.

~~~
jerf
#5 isn't a matter of convenience if you're trying to do heavy mathematical
operations in Java. Deviation between conventional notation and programming
notation is a real problem, because it's a place where bugs can hide.

However, I would be very sympathetic to the argument that the solution is
"Don't do that". In 1998 and today, for what may be different reasons but
still comes to the same outcome, Java is not a language intended for highly
mathematical computations. It's probably better just to not pretend that it is
one, than to try to force it in. Nor would I even consider this a criticism of
Java (though I personally dislike it); compared to its focus area, trying to
support heavy-duty math would be a tiny little niche to it that would take
wildly disproportionate amounts of effort to support. Let people who care
about that niche operate in it.

My only point of agreement is that it is true that you ought to be able to get
easy feedback about NaN and other failures, indeed precisely _because_ in Java
you are unlikely to be doing the sorts of calculations in which those are
expected possibilities. $NaN or "infinity hours billed" is something you
generally want an exception for, ASAP. It would have been better to write that
in as a requirement for the VM and slowly emulate it on the few platforms that
didn't natively support it than leave it out. (A brief google search suggests
Java still doesn't support this, even today. If I got that wrong... great! I'm
glad it has come on board, but I stand by the idea that it should have been in
there from day 1, which is the real point I have here.)

------
japaget
What I found most interesting about the article was some of the mathematical
techniques presented. See page 44 for an accurate formula for angular
distances and page 48 ff. for an alternative formalism for the vector cross
product.

------
saejox
Cyrix. That's a word i didn't hear in at least a decade.

------
Aardwolf
Fully agree on the operator overloading complaint. It really is a LOT easier
when working with matrices, vectors and scalars to be able to type their math
expressions naturally rather than with functions like add(multiply(.....))

~~~
peterashford
What operator do you use for vector dot product? What about cross product (as
opposed to vector scaling)? What about the different precedence between
primitive operators and overloaded operators ('+' and '*' have the same
precedence in vector maths, no so in basic arithmetic).

Does adding mathematical notation to programming languages make the code
easier for non-mathematicians? Does having two different coding standards make
the language easier to use and maintain? What do you do when your set of
operator glyphs is less than the number of mathematical operations you need to
perform? What happens with your code when the operator overloads you use clash
with the operator overloads from a new library?

I don't see any language that does operator overloading well and a lot of
issues with making a good implementation. I like the idea I just see a lot of
problems with the reality. And unless you can solve the problem well, I think
it's the right thing to do the simplest thing that might possibly work.

~~~
Aardwolf
None for dot product.

The regular matrix and vector multiplication as * operator makes things a lot
more readable.

Dot and cross product are less common for 3D rendering (one or none per line
of code) so having that as a function doesn't hurt readibility.

~~~
peterashford
I like writing ray tracers, so I use dot product and cross product quite a
lot.

------
PaulHoule
Many of the problems they point out aren't just Java problems, they're FORTRAN
problems too.

------
peterashford
Another "everything Java does is evil" rant. No wonder HN is all over it. This
is tiresome.

If you want 64 bit precision - use doubles. That's not hard, is it?

