
How to Print Floating-Point Numbers Accurately (1990) [pdf] - tosh
https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf
======
kentonv
Back in 2007-ish I spent a bunch of time trying to figure out how to print
floating point numbers losslessly. Annoyingly, there's no printf() format
string that says "print exactly as many digits as needed so that it parses
back to the same value". You can specify a number of digits to use, but to
avoid printing unnecessary digits in all cases (e.g. "0.20000000000000001" for
0.2) you must ask for a maximum of 15 digits, but in some cases you will lose
data if you ask for fewer than 17 digits.

After asking around, I came across the Steele & White paper, and the
implementation known as `dtoa()` written by David M. Gay. At the time, this
seemed to be understood to be the "correct" answer.

But then I looked at the code:
[http://www.netlib.org/fp/dtoa.c](http://www.netlib.org/fp/dtoa.c)

Uh.

There's a lot to dislike about that code, but arguably the worst thing is that
it isn't thread-agnostic. It mutates global variables and protects them with a
global mutex. A global mutex lock, just to print a number! Whyyyyyyyy?

So then I tried something different: I wrote some code that would do sprintf()
with 15 digits precision first, then parse it with strtod() to see if it came
back exact. If not... then I did sprintf() with 17 digits precision... and
called it a day.

In benchmarks, this turned out to be just as fast as calling dtoa().

And so that's how the Protocol Buffers library deals with numbers when writing
TextFormat:

[https://github.com/google/protobuf/blob/ed4321d1cb3319998411...](https://github.com/google/protobuf/blob/ed4321d1cb33199984118d801956822842771e7e/src/google/protobuf/stubs/strutil.cc#L1174-L1213)

But the bigger lesson for me was: Transmitting floating-point numbers in text
is _awful_. People have no idea how ridiculously complex this is, because it
seems like it ought to be simple. When you send JSON with numbers in it, you
are probably invoking code that looks something like dtoa(), over and over and
over again. And that's just to write them out; I have no idea how complex the
parsing side is.

Please, folks, think of the CPU cycles. When sending numeric data, use a
binary format.

~~~
taneq
> When sending numeric data, use a binary format.

Binary representations of floating point number are a bit of a nest of vipers
too, though. You can _probably_ just pipe IEEE floats across the network as
bytes, in practice, but it's risky.

It's probably safer in many cases to just transmit things in fixed point.

~~~
dbaupp
I can think of a few possible problems with IEEE754-across-the-wire, such as:

\- not all hardware supports them

\- subnormal numbers are annoying

\- NaNs can be tricky to handle, such as signalling and (if payloads are used)
security risks through NaN payloads.

Which are you thinking of? Or something else?

~~~
jcelerier
> I can think of a few possible problems with IEEE754-across-the-wire, such
> as:

> \- not all hardware supports them

let's be reasonable : how many people who run architectures which don't have
any float would actually send floats to them over the network ?

~~~
gnufx
The parent didn't say "don't have any float". I have had to translate between
little endian IEEE754 and big endian IBM format, roughly copying what HDF did.
Protein crystallographers invented their own binary file format rather than
just using HDF, and didn't define the binary floating point format, so that
the files weren't portable initially.

------
oleks
Have you seen this POPL'16 paper?
[https://popl16.sigplan.org/event/popl-2016-papers-
printing-f...](https://popl16.sigplan.org/event/popl-2016-papers-printing-
floating-point-numbers-a-faster-always-correct-method)

~~~
tokenrove
Note that the claims in that abstract were retracted because the authors
messed up when benchmarking. Note the changed title and abstract here:
[https://cseweb.ucsd.edu/~lerner/papers/fp-printing-
popl16.pd...](https://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf)

~~~
simonbyrne
It is really annoying that CS conference proceedings don’t have a mechanism
for retraction or post-publication errata.

~~~
marcandrysco
We were able to update the paper on the ACM website:
[https://dl.acm.org/citation.cfm?id=2837654](https://dl.acm.org/citation.cfm?id=2837654),
although looking at it now, they did not update the title. I'll see if they
can fix that.

There is very exciting news coming up with printing floating point. Ulf Adams
from Google will be presenting a new algorithm called Ryu that appears to be
super fast, simple, and perfectly accurate. Assuming the claims are correct,
Ryu ought displace all of the current algorithms.

~~~
simonbyrne
Thanks! My comment was not meant as a criticism of your work (I still like the
paper, and do appreciate that you went to the effort of putting the amended
results online), but more of the CS conference publishing model.

I look forward to reading the Ryu paper.

------
bloak
Presumably this article from 1990 does not mention C99's hexadecimal floating-
point literals, for example:

printf("%a\n", 0x1p-10);

It's a shame that JSON doesn't accept them.

------
enf
I've been using Milo Yip's implementation of Grisu2, which references another
paper: [https://github.com/miloyip/dtoa-
benchmark](https://github.com/miloyip/dtoa-benchmark)

------
saagarjha
For anyone looking at a practical implementation, Swift recently migrated to a
Grisu2-based algorithm:
[https://github.com/apple/swift/commit/97a934c412a26f0222f57d...](https://github.com/apple/swift/commit/97a934c412a26f0222f57d5e4cd7467a516eb6af)

~~~
eboyjr
For reference and discussion, skimming the docs, the algorithm is described
here [0] and contains some improvements suggested by the "Errol paper"[1]

[0]: [http://www.cs.tufts.edu/comp/150FP/archive/florian-
loitsch/p...](http://www.cs.tufts.edu/comp/150FP/archive/florian-
loitsch/printf.pdf)

[1]:
[https://news.ycombinator.com/item?id=10915182](https://news.ycombinator.com/item?id=10915182)

------
RossBencina
See also:

William D. Clinger, "How to Read Floating Point Numbers Accurately," Proc. ACM
SIGPLAN '90, pp. 92-101.

pdf:
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45....](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45.4152&rep=rep1&type=pdf)

And David Gay's implementation:
[http://www.netlib.org/fp/](http://www.netlib.org/fp/)

------
SloopJon
I've been looking at key-value stores recently, and I wonder what people think
about collating numbers of heterogeneous types.

FoundationDB tuples have type codes that segregate values of different types,
so that the strings "1" and "2" sort before the integers 1 and 2, which sort
before the single-precision floats 1.0f and 2.0f, which sort before the
double-precision floats 1.0 and 2.0.

The database that I test supports double-precision IEEE floats, and a
proprietary decimal float with a signed 64-bit significand and signed 8-bit
exponent. When converted to string for use as keys, these collate as expected.
The price of this is that you don't get shortest representations of the sort
sought by this paper and others. Otherwise, a binary and decimal float that
compare unequal could convert to the same string.

I guess it's kind of unusual to use floats as keys, and more unusual still to
use both binary and decimal floats, but I wonder if there is another strategy
for collating them.

------
userbinator
That's the paper referenced in this one, which more people may have seen, also
about correctly converting floats to strings and vice-versa:
[https://www.ampl.com/REFS/rounding.pdf](https://www.ampl.com/REFS/rounding.pdf)

------
zuzuleinen
A useful website for understanding float issues [http://floating-point-
gui.de](http://floating-point-gui.de)

