> printing the number might just cut off the digits at some point.
I'm not sure, but I think it's not really about cutting off the digits (although that's the effect it has). At least with python, I think what happens is:
* repr gives you the decimal number with the exact value of the floating-point number, i.e. 0.30000000000000004
* print gives you the least-precise decimal number whose closest floating-point representation is equal to the floating-point number
That is, there are (infinitely) many decimal numbers such that the closest floating-point representation has the value 0.30000000000000004. One of these decimal numbers is 0.3, and of all such decimal numbers, this has the fewest digits. So that's the one that gets printed.
Someone pointed out that this isn't correct, and then deleted their comment. (?)
I now think the behiviour I ascribed to print, is actually what repr does. I'm not sure about print, but I guess it is just rounding to a certain number of places.
In this case, that was deliberate. I was using it to show the exact decimal value of the floating point approximations to decimal numbers, not to do decimal arithmetic.
There's some discussion here: https://docs.python.org/3.1/tutorial/floatingpoint.html. Basically: eval(repr(x)) == x is guaranteed, and some attempt is made to give the shortest such string representation for Python 3.1+ (and 2.7?), but it's not guaranteed on all platforms.
> For what its worth, perl6 passes this test with flying colours.
Do we really expect calculations on floating point numbers to be the same as calcualtions with real numbers?
The only problem I see is that often the floating point format is not specified properly. When one writes "0.3", what does it actually mean? Does the format the compiler uses internally conform to IEEE 754? Is it 32bit, 64bit, 80bit or something else?
The only problem I see is that often the floating point format is not specified properly. When one writes "0.3", what does it actually mean?
It's perfectly well-specified. If you are using double-precision floating point, "0.3" means 0x3fd3333333333333 (as bits), or 0.299999999999999988897769753748434595763683319091796875 (as an exact number: the closest representable number to 0.3).
Most languages do (though implementation bugs are common—plenty of languages where 64-bit precision is required actually end up being implemented in such a way that 80-bit precision is used on x86).
That said, a lot of low-level languages don't: C doesn't guarantee this, for example. C doesn't even mandate IEEE 754, and it merely requires that "float" is a subset of the values of "double" which is itself a subset of the values of "long double".
Perl 6 doesn't use IEEE 754 by default at all. It internally stores decimal literals as rational numbers, with numerator and denominator in separate fields, and it only resolves back to a decimal number when you convert it to a string.
In other words, there are about 5648798746465 possible interpretations of the "(\+|-|)[0-9]+.[0-9]+" format, and the problem is with mismatched assumptions ("of course I'm using the One Correct Interpretation, like everyone else!").
> Just adding 0.1 to 0.2 in the language of your choice can be misleading because printing the number might just cut off the digits at some point.
It's amazing that someone went to the trouble to register a custom domain for this and put up a nicely formatted page, yet doesn't explain the above: that it's not (just) a matter of representation, but printing precision.
What does it mean to "pass" this test? Just that the software is massaging its output to match human expectations rather than giving the most accurate (if not the prettiest) answer?
> For what its worth, perl6 passes this test with flying colours.
This is because Perl 6 decimal literals are stored in a built-in rational type called Rat, which internally stores numbers as ratios between integers. So that 0.1, 0.2, 0.3 are internally stored as the tuples (1,10), (2,10), and (3,10), and the decimal value you see is just the string representation and not the actual internal value.
If you want your decimal literal to be floating-point (called Num in Perl 6), you want to add e0 to the end.
That's interesting, I had no idea. There's probably a way more detailed Rosetta Code style project that could cover all these nuances, but for a single page demo, there's only so much detail I can get into before it would be more confusing than helpful.
If you'd like to flesh out the perl section a bit though, maybe include the differences between 5 and 6, I'd be happy to merge a pull request.
You've had a perl6 pull request outstanding for a while. The differences are far too big to list simply though. Ultimately Perl6 uses Rationals by default, not Floats.
Of course they do, just not a binary floating point representation. They can be represented with decimal floating point just fine and there are plenty of programs and libraries that can do this.
"Finance" uses lots of different things. Sometimes it uses symbolic math; sometimes it uses binary floating point, sometimes it uses decimal floating point, sometimes it uses arbitrary precision decimals or rationals, sometimes it uses integers in the smallest unit of concern for the application.
One it so use fixed point decimal with a 1/100 scaling instead of floating point. So yes, integers and then you shift the decimal point when you output.
But you have to watch out for interest calculations, you might earn a fraction of a penny on a daily basis but over a year it adds up and you can't throw away that accuracy.
So you might do all math as floating point, but with pennies instead of dollars.
Or you can do Binary Coded Decimal, which stores each group of decimal number after the decimal point exactly, in many ways it similar to fixed point decimal, since it is a fixed point.
Do they do USD interest calculations in something smaller than 1/10000 (decimill I think)?
The issue with BCD would be data store support and CPU support right? Do CPUs support doing math directly with BCD values and does your DB support doing math directly with BCD values?
In any case, using floating point for any currency calculations is a huge no-no, even though a lot of things do it anyway.
I work with Magento daily and all their currency math uses PHP floats and the rounding errors drive me bonkers.
Well, I know from doing payment gateway integration that many of them use integers for currency amounts. They are the smallest currency unit and for display you then convert based on the currency specific precision value.
But these numbers are perfectly computable! Sure they cannot be represented in the commonly used binary implementation of floating point numbers, but that doesn't mean computers cannot operate with these numbers at all.
Well, they can't be specified as IEEE 754 floats. A computer can still use those numbers in the form of decimal fixed-point or numerator-denominator formats.
>There are three binary floating-point basic formats (encoded with 32, 64 or 128 bits) and two decimal floating-point basic formats (encoded with 64 or 128 bits)
"Computers can only natively store integers, so they need some way of representing decimal numbers"
Technically, [digital] computers only natively "store" high and low voltages. The interpretation of a sequence of those signals as "integers" is entirely up to you.
In all seriousness, representing integers is not a given either (e.g. two's complement vs one's complement, BCD, bignum, sparse integers, etc.). If you are going down in abstraction levels to describe floating point representation, you should not just call integers "native" blindly.
Furthermore, floating point is not the only way to represent computation involving real values. It is indeed possible to represent computation on real values symbolically in many cases and lazily compute arbitrary precision results for output-printing purposes.
Technically, much storage does not need voltage being high or low either. Pits burned with lasers, and magnetic changes are also common ways to store stuff that don't involve voltage in the stored state.
Here you can see python has a fractions.Fraction type which can be used for rational number arithmetic.
Here we see that python has a nice behaviour for when printing floats.
>>> print(0.1 + 0.2)
0.3
Here is a common error where people feed the input of Fraction (and Decimal) with a float. Even though you use the Fraction here it still carries the floating point error through.
Here is the Decimal type which does not have this float problem. Some people have argued (including myself) that Decimal should be used by python instead of float. Since this is much friendlier to people doing things like adding these numbers together.
This is a good case for macros in Python, actually. Fraction('0.1') is usually what you want, so Fraction(0.1) should mean just that and if you do want the current behaviour, Fraction(float(0.1)) would do it. Same for Fraction(17/3) for example.
That depends on the number you want to represent. Some values can be represented exactly, others with varying degrees of precision because of the difference between what is actually stored and what you wanted to store. This will collapse a number of values into the same precision 'bucket'.
What is continuous or precise depends on the base you use. I can't think of any example in base 2, but I believe in base 3 you could represent numbers that have infinite digits in base 10.
If I'm not mistaken, 0.3333... would be 0.1 and 0.6666... would be 0.2. It probably gets nasty for a lot of simple base 10 numbers, so it isn't a good solution.
> Technically, [digital] computers only natively "store" high and low voltages. The interpretation of a sequence of those signals as "integers" is entirely up to you.
And then there's MLC NAND flash which stores bit sequences as different voltage levels within the same cell.
That's valid, but I was going for an ELI5 style explanation of a concept that surprised and confused me at the time. If you want to provide a better one, I'm happy to merge any pull requests.
when you aim for reddit style stuff like "ELI5", don't be surprised if HN tears you up. reddit likes things to be cool and cliquey. we like things to be correct.
Those languages which print 0.30000000000000004 are doing something which is silly as a default: they are printing 17 decimal digits, out of a type that only supports 15.
$ txr -p '(+ .1 .2)'
0.3
Why is that? Here, the underlying type is the C double type. The printed representation is obtained according to a default precision. That default is taken from the C constant DBL_DIG. For an IEE754 double, that constant is 15.
It is misleading to print more decimal digits out of a double than DBL_DIG; that is the constant which tells you how many decimal digits of precision double can reliably store. Thus I chose that constant as the default printing precision for floats: to give the programmer/user the maximum realistic precision. That is to say, print the decimal digits which are plausibly there, and not any fictional ones.
17 decimal digits requires a 57 bit mantissa -- ceil(log 10 / log 2) * 17). The 64 bit double type has only 52. So if you print 17, you're "making shit up".
This is a common but fundamentally wrong view of floating-point numbers. You are not "making shit up" by printing more that 15 digits of a floating-point number. Floats, doubles, etc. have precise values. As I said in another comment, the double represented by the literal `0.1` is precisely 0.1000000000000000055511151231257827021181583404541015625; similarly, `0.2` is precisely 0.200000000000000011102230246251565404236316680908203125 and `0.3` is precisely 0.299999999999999988897769753748434595763683319091796875. From these exact values, you can see why `0.1 + 0.2 != 0.3` – the left hand side is greater than 0.3 while the right hand side is smaller than 0.3. Printing only 15 digits is not doing programmers any favors: even though `0.1 + 0.2` will print as "0.3", that just makes the lie even worse – they will be even more confused when `0.1 + 0.2 != 0.3` even though both values print identically.
Your claim is fallacious, because a floating-point value denotes a range of the real number line, whereas you're insisting that, no, it stands for its literal value: the absolutely precise rational number that lies at the center of that range.
That is no more true than my current body temperature being precisely 36 4/10 degrees Celsius because the thermometer reads 36.1.
All numbers in that range alias to the same double, yet differ wildly in their decimal digits beyond the fifteenth. That "long tail" of digits is a meaningless residue arising from the arbitrary difference between the chosen center-of-range point and the actual number it approximates.
> Printing only 15 digits is not doing programmers any favors
Note that the ISO C function printf uses 6 digits of precision under the %g conversion specifier by default. So a team of experts can find it justifiable to severly truncate precision on printing. I find that justifiable also, like this: printing is not only for constants, and trivial expressions like 0.1 + 0.2, but for the results of complex calculations, which accumulate significant rounding errors. Six digits of precision is prudent: for many complex calculations, it will avoid misleading the user with too much precision. Fifteen decimal digits will be wrong after just a few operations; there is hardly any "headroom" to absorb error.
> just makes the lie even worse
If merely neglecting to reveal some aspect of the stark truth is tantamount to lying, then you're also lying when you pull a number with 54 significant decimal figures out of a 64 bit double. Revealing some truth without an explanation can also be construed as "lying", as in "[N]one of woman born shall harm Macbeth".
I personally find it convenient that when I bang the token 0.3 into my REPL, it comes back with a tidy 0.3. I know that there are several values of double which will print as 0.3, and don't compare with exact equality except in special circumstances (like when deliberately using integral values not far from zero).
> Your claim is fallacious, because a floating-point value denotes a range of the real number line, whereas you're insisting that, no, it stands for its literal value: the absolutely precise rational number that lies at the center of that range
Floating point numbers are _not_ intervals. If you read carefully any formal definition of floating point numbers (The IEEE standards, TAoCP Chapter 4, or Higham's _Accuracy and Stability of Numerical Algorithms_, to name just three possible references), you will see that floating point numbers by definition form an exact rational subset of the extended real line.
> All numbers in that range alias to the same double, yet differ wildly in their decimal digits beyond the fifteenth. That "long tail" of digits is a meaningless residue arising from the arbitrary difference between the chosen center-of-range point and the actual number it approximates.
You are not being clear about the set of real numbers a user may input and their ultimate representation as a floating point number. While it's true that many real numbers round to the same floating point number _f_, that's not the same as then saying that _f_ carries that interval around with it. The latter is false, since the intervals do _not_ propagate in floating point arithmetic. It's also false that the floating point number _f_ is the midpoint of the set of numbers that round to _f_; the precise set depends on the rounding mode and the granularity of the set of floating point numbers around _f_.
I didn't say they were intervals (I'm well aware of interval representations for numbers, which single point rationals are not).
> that floating point numbers by definition form an exact rational subset of the extended real line.
Sure, that's what they form; it's not what they (usually) denote.
> It's also false that the floating point number _f_ is the midpoint of the set of numbers that round to _f_;
It is close enough to being the case for my purpose in the grandparent article. Whether the cluster of numbers is lopsided one way or the other doesn't really detract from my point.
Your view has been repeatedly refuted by Kahan and others. See page 25 of [1], page 1 of [2], page 22 of [3] for just a few examples. Conceptually, the reason the interval view is wrong is that if floating point values represented intervals, then their arithmetic would be incorrect. For example, the sum of an interval of width w1 and an interval of width w2 should be an interval of width w1 + w2. That, of course, is not how floats work at all. The correct view is that floats represent exact rational values and operations on them, instead of being the true mathematical ones – under which floats are not closed – compute the closest float to the true result of each operation.
1.0, 2.0, and 3.0 are all represented perfectly and precisely in IEE754 floating point, as are all integers up through 2^24 (for 32-bit floats) or 2^53 (for 64-bit doubles).
2^0 is 1, so if the exponent is 0 then the mantissa times one is your value, so every integer has an exact representation.
If the exponent is 1, then the mantissa is multiplied by 2 to get the actual value. Put another way, the mantissa is the value divided by 2 and rounded, i.e. shifted right one bit.
> 1x10^7 + 2x10^7 != 3x10^7, right?
This particular case is interesting.
If we're talking about 64-bit doubles, then you have 53 bits of precision for integers. All those values are well within that range, so this arithmetic and comparison are done with precise integer values, and the expression will compare equal.
If we're talking about 32-bit floats, the range of precise integer values is -16,777,216 through 16,777,216, or -2^24 through 2^24.
1x10^7 is 10,000,000, within that range, but the other two values are outside the range.
So you might expect that != would be the answer here.
But if you test it, that isn't the case: the expression compares equal!
The reason: although 20,000,000 and 30,000,000 are outside the range where every integer has a precise representation, they are within the range where every even integer is precise: -33,554,432 to 33,554,432. Values within this range but outside the range of precise integers are rounded to a multiple of 2.
Similarly in the range -67,108,864 to 67,108,864, all integers which are a multiple of 4 are represented precisely.
Basically, as you go outside the range of precise integers, values get rounded to a multiple of 2, 4, 8, etc. as required.
When the mantissa overflows the available precision, it is shifted to the right enough so that it fits without losing the most significant bits. Instead, the least significant bits are discarded, and the exponent is incremented by the number of discarded bits.
Of course you could choose other values where the rounding doesn't work out in your favor, and then you'd get the unequal comparison you expect. A simple example:
33333333.0f + 1.0f != 33333334.0f
Here, the value 33,333,333 is rounded down to 33,333,332. Add 1 to that and you get 33,333,333, which is again rounded down to 33,333,332.
33,333,334 is represented precisely, so the comparison fails.
Stratoscope was already done a superb job explaining. But just to show why my question was not that dumb...
Let's say you want to represent "two-million" in 32bit float:
Your answer is: 0x1e8480 * 2^0.
However, I was expecting something like: 0x1.e848 * 2^X. But then "125-thousand" is 0x1.e848 * 2^Y, so how do you standardize what is the correct exponent for integers, X or Y? And how do you know there is not a pair (M,Z) != (0x1.e848, X) such that M * 2^Z is also two-million?
the fraction of a floating point is always "starts with on invisible 1.xxxx" so that forces unique representations for your exponent for any given number. Is it not clear why this is the case?
let's take a simple case where you have one bit of fraction.
So for any given exponent multiplication with a fraction with an invisible bit creates products that are hemmed between 1.0x2^n and 2.0*2^n so there's no collisions between any pair (e,f) in your representation.
> It is misleading to print more decimal digits out of a double than DBL_DIG
If you want round-trip conversion for all `double` numbers, i.e. from `double` to text and back, you need 17 digits of precision (%.17g).
Likewise, if you want round-trip conversion for all `float` number, you need 9 digits (%.9g).
The reason for this is that even though these types can really only reliably give 15 and 6 digits of precision, some of the numbers they can represent are closer to each other than that.
In C++, you can get the former limit using std::numeric_limits<double>::digits10, and the latter using std::numeric_limits<double>::max_digits10.
> If you want round-trip conversion for all `double` numbers, i.e. from `double` to text and back
I don't. Rather, I want round-trip conversion of all 15-digit-precision decimal numbers to the machine and back. If the machine calculates some numbers which are different from some of these but alias to them textually, I don't care.
Someone else might want the decimal text representation to provide bit-exact storage semantics for arbitrary doubles.
For that reason it might be a better design choice to obtain the default printing precision from a special variable, which itself defaults to 15, rather than a hard-coded default.
it supports somewhere between 15 and 16, actually. There are 52 bits of precision in a double's mantissa[1]. This gives you log_10(2^52) = 52*log(2)/log(10) ~ 15.654 digits of precision, that is, the 16th decimal place is accurate about 65% of the time.
You can also exactly represent in decimal the binary value stored, because as long as you have 52 decimal places, you have enough 2's in the denominator for all of the bits that you can represent (each 10 in the denominator pairs up with each 2 from each bit).
--
[1] Plus an implicit bit, but not in the fractional part of the mantissa.
$ ./txr
This is the TXR Lisp interactive listener of TXR 123.
Use the :quit command or type Ctrl-D on empty line to exit.
1> (tostring (+ 0.1 0.2))
"0.3"
2> (let ((*flo-print-precision* flo-max-dig)) (tostring (+ 0.1 0.2)))
"0.30000000000000004"
3> *flo-print-precision*
15
4> flo-dig
15
5> flo-max-dig
17
- 15 by default is reasonable for everyday programming; we don't need results like 0.30000....4 popping up in our faces.
- provide the constant which indicates how many decimal digits are needed to preserve the binary value exactly: this is needed for faithful storage and communication --- we wouldn't want a Sexp-based RPC call to behave differently from a local computation.
- provide the printing precision default as a special variable which can be overridden over a dynamic scope.
15 by default is reasonable for everyday programming;
we don't need results like 0.30000....4 popping up in
our faces.
That just changes the problem. You can't get rid of rounding error by representing decimal numbers in binary. 0.3 - 0.2 - 0.1 still won't show up as zero even if you only display 15 decimal digits.
I'm not about to change the floating-point type to a non-binary support.
I think that 15 digits is the "right" value for ordinary display purposes. If you present a literal value of up to 15 digits to the machine, it gets converted to some approximation which, when printed back with 15 digits of precision, gives you the same digits.
I have no "rounding error" because I haven't performed a calculation. I understand that when I enter 0.3 into the machine, the floating-point value isn't exactly 0.3. I have an error, because the representation can only provide a close approximation of 0.3.
Yet I know that this approximation is so close to 0.3 that I would like it printed that way. Now of course it will be printed that way if I round to, say, four places. But that will throw away precision for other numbers. If I round to DBL_DIG digits (15), I get the maximum precision, without having my input numbers altered into something else.
It is useful and elegant for the default printing precision to be as high as possible, yet such that numbers that I enter into a REPL are echoed back at me pretty as I entered them, modulo choice of notation.
No, of course, DBL_DIG doesn't guarantee that I can do any sequence of calculations, like 0.3 - 0.2 - 0.1 and still get back a result which is exact to DBL_DIG digits! There is no lower bound on how inaccurate a floating-point calculation can be, if it is carried through enough iterations!
Implied fixed material doesn't contribute precision. Otherwise we could pretend that there are twenty-four implied 1's and the precision is 76 bits.
Because the implied 1 cannot be zero, it isn't a bit; it doesn't carry information.
For instance if we define a four bit binary number type which has an implied 1, it can only represent values from 16 to 31. That's four bits of precision; the range is just displaced.
The information of the first bit is implied via the exponent.
Twenty four implied 1's would give you a precision of 76 bits, but you would restrict yourself to a very small subset of all possible numbers with that precision.
EDIT: Let's put it another way. Assume you have 3 decimal digits of precision for a float. The first digit cannot be zero. Would you then claim that you don't have 3 digits of precision, but rather log(10x10x9,10) = 2.95.., because the first digit only carries 0.95.. digits of information?
> binary floating points as the default representation for literals expressed as precise decimals.
For better or worse, it is a widespread practice appearing in numerous programming languages, including some widely used popular ones.
To conform with the practice, if you want, in your language, a way to use decimals to write exact fractional numbers, it's better to have an explicit notation for that like, say, 10.5r (rational number, another spelling for 21/2). Or hey: 10R5. Electrical Engineers will love you to death. :)
> For better or worse, it is a widespread practice appearing in numerous programming languages, including some widely used popular ones.
Well, yeah, I've been programming since the 1980s. I know that.
> To conform with the practice, if you want, in your language, a way to use decimals to write exact fractional numbers, it's better to have an explicit notation for that
Sure, given that your goal is to conform with that practice. I'm just saying its a bad practice that reflects a generally premature optimization that often harms correctness, so we shouldn't conform to it in new languages.
> [implicitly approximating numbers like `0.1`] is a widespread practice
Sure, but the key point is that it leads to approximation that may cause problems.
It makes much more sense to interpret `0.1` as meaning exactly that number. Same with, say, `29/7`. If someone wants approximation, then have an explicit notation for that, like, say, `1.23e45`.
Defaulting to a Binary Encoded Decimal is also silly because its not as performant and loses compactness. Fundamentally. (Having a composite base comes with problems). What is your suggestion, having a default floating point have a Binary or hexadecimal representation for literals? Let's say I want to input a planet's mass as 6e24 kg - quick what is that in your literals?
It doesn't have to lose performance; it's just the default representation. For instance, if you have
a + 0.1
where a is of binary floating point type, the 0.1 constant could denote an exact number. Because of the addition with a float, it gets coerced to the float's type. But that can be done at compile time. So effectively 0.1 is like a binary floating point constant in that situation.
But in another situation it remains exact, such as:
0.1 + 0.2
this gets folded by the compiler at compile time, and uses exact math producing an exact 0.3.
Exactness is just the default.
If your language has rationals expressed as digs/digs, you can imagine that those are used in their place:
a + 1/10;
1/10 + 2/10;
In other words, suppose that 0.1 is just another "spelling" for 1/10.
This will be no less performant than using integer constants in floating contexts, e.g:
double x;
// ...
x += 1; // not slower than x += 1.0!
> Defaulting to a Binary Encoded Decimal is also silly because its not as performant and loses compactness.
Defaulting to correctness makes a lot more sense than often premature time and space optimizations by default.
> What is your suggestion, having a default floating point have a Binary or hexadecimal representation for literals?
There's lots of good solutions (Scheme's, of having a numeric tower with literals using the minimal-scope exact representation -- e.g., integer if there is no fractional part, decimal if its a decimal literal, rational if it is rational literal not expressed as a decimal -- by default unless they have an modifier specifying intent to use an inexact representation) is the best.
the point is, there are still unrepresentables (1/3 e.g.) in binary encoded decimal. If it's all about literals, just make the user put the correct literal in, and make them understand that floats are imperfect anyways. If you're doing anything remotely useful with Binary Coded Decimal, probably about two or three operations in, you're going to lose resolution on your exactness. It doesn't buy you much. GIGO.
In the last half a year or so, I added major features such as: structs with good OOP support, an interactive REPL (based on "linenoise" with a lot of my hacks applied), and delimited continuations.
That... is not a known hardware floating point representation. There are way too many digits. This is a just-plain-buggy conversion routine I'm guessing? What is Numbers, and why isn't it using a proper arbitrary precision library if it doesn't want to use doubles?
Edit: sorry, my brain was still reading the leading 0.3 from the linked article. There is indeed nothing wrong with that representation except the nonstandard leading zeroes.
A double can hold 15-17 significant base 10 digits. The exponent can be +-10^308 (11 bits for the exponent, 52 bits for the fraction, 1 bit for the sign).
Yes yes yes, I know how to represent an IEEE value. It was an interpretation error: the linked article shows a zillion variants of numbers with a leading significant of 0.3 (i.e. all the leading zeros are significant) while the posted string above did not.
Here is the part that made me understand it all: "Floating-point representations have a base (which is always assumed to be even) and a precision p. If = 10 and p = 3, then the number 0.1 is represented as 1.00 × 10-1. If = 2 and p = 24, then the decimal number 0.1 cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4."
In Haskell/GHC it depends on what representation you chose. Default is Double, which exhibits this problem. But if I force it to Float I get the 'correct' result. Or if I use the 'Scientific' data type (which has arbitrary precision). Just like 'Int' vs 'Integer', where the former is restricted by the machine's word size, while the later is unbounded.
So effectively, in Haskell, it's both and neither.
A Float will round, as expected.
A double will return the correct value as it, should.
Shouldn't the Scientific, if it has arbitrary precision, arrive at the correct value too?
Clearly the only logical conclusion is that the Glasgow compiler, at its roots, virtualizes the entire quantum mechanics model as a compilation step, enabling superposition-as-a-feature.
My futile attempts at humour aside, this is quite an interesting thing to know and exactly the kind of intricate little detail I was hoping for.
The Scientific type does arrive at the correct value. 0.3 is the correct answer, and Scientific gives it to you. I don't know why you would expect something else from a type which implements arbitrary precision arithmetic.
Its important to understand that 0.1 + 0.2 is actually 0.3, and Scientific is giving the correct answer.
The Haskell type annotations aren't casting values that are interpreted as binary floating point (as might be the case in some languages), it is instructing the compiler how to treat the literals. Scientific is arbitrary precision decimal, so it can exactly represent 0.1 and 0.2, and return the exact result 0.3, rather than representing 0.1 and 0.2 as binary floating point approximations, adding them, and then producing some decimal approximation of the binary floating point result when asked.
I'm not sure what was the intended message of the table of examples. It is possible in many languages to get both types of ASCII representations for floating point: a nice human-readable 0.3 and the technically correct 0.30000000000000004. But some examples seem artificially biased towards one or the other. Only for Python both variants are shown. And then some examples use decimal representations, which are not floating point at all.
For instance, in Perl "print 0.1+0.2" will give you "0.3" just like in Python, while the printf format shown here will give you the non-rounded representation. And there are of-course decimal and rational number packages for Perl as well.
For those that arrive at the right answer, is it by mathematical correctness in the implementation or by rounding down?
i.e. Which are right and which are flukey in their wrongness?
Under any other circumstance I wouldn't care about this pointless detail, but seeing as how this is the top post on the front page, I'd like an excruciatingly detailed investigation please.
Don't make me not get round to doing it myself, sir.
If you want PHP to print it like the other languages you have to set the output precision high enough. The default, as you mentioned, is 14 but if you set it to 17 then you get the 'correct' output.
All of them may be correct as typically, multiple 'answers' can be correct. One could argue that given a number, one should print the shortest string that round-trips to that number, but that may mean that one prints "0.3" while the 'real' value is closer to 0.30000000000000004 or 0.29999999999999998 (a representation that one of the algorithms in the above paper gives) or possibly even equal to it.
Also, some of them may be incorrect for backwards compatibility reasons. In the eyes of some, that makes them correct again.
Don't be ridiculous. It's perfectly fine if you know what you're doing. For example, when you have a limited selection of floats (e.g. hashtable keys, or only numbers between 0 and 100, accurate to 1 decimal) - in these cases, strings would work just as well, but floats are more efficient (and compare correctly).
You've just redefined your problem to using floats as the underlying representation. In that case you don't even need floats...
In most other cases the correct way is to check if the absolute value of the difference between one value and another is smaller than some arbitrary cut-off, a small number typically denoted with 'epsilon'.
Until your code gets ported to a platform where float == double and it turns out the algo feeding your clever efficient comparison delivers slightly different values that previously collapsed into the same bucket. Relying on the particulars of an underlying representation requires deep knowledge of how those particulars are implemented (and I would happily agree with you that such knowledge is a requirement but more often than not when you come across a float comparison with equality you're looking at a bug. That you're the exception to the rule is fine with me, you seem to know what you're doing so don't take it personal).
The real problem is: most people don't have deep knowledge of how those details are implemented and take 'equality' for granted, especially if the visual representation of the numbers they are comparing tells them they should be equal which is not always the case.
The old adage goes: if you need floating point you don't understand your problem :)
The comment that sparked this sub-thread is a nice illustration of what can and does go wrong.
"In current versions, Python displays a value based on the shortest decimal fraction that rounds correctly back to the true binary value, resulting simply in ‘0.1’."
As illumen pointed out, Python has also got fractions.
For things like Pi however, there is no exact numerical representation and you'd probably want sympy or something similar.
Sure. But the inability to represent 1/3 in decimal is just the same as inability to represent 1/10 in binary. The binary case just seems different because we are using a foreign base (decimal) literal "0.1" to express the binary number.
There are a few broken links. I believe the original poster references the following paper by Robert Burger & Kent Dybvig, "Printing Floating-Point Numbers Quickly and Accurately":
I suspect in many instances it's down to the method doing the printing being "helpful". The page documents this behaviour with Python. I've never specifically used C#, but it seems to default to the "G" format[1], the alternative "R" format shows the expected result. You can play around with it here: http://ideone.com/hxD96J
Printing out is kind of a misleading check here, maybe a better idea would be to test equality with the constant 0.3. Of course the site is just a neat illustration and not a technical whitepaper.
Yes, I agree this page is misleading. It makes it look like C# is using rational numbers instead of floating point. In fact the "G" format defaults to 15 digits of precision for doubles, which is few enough to avoid ugly strings in many simple situations, but not all:
The printing routine. Internally, none of them can store 0.3 in a floating point variable because it is impossible to do so in exponential notation with base 2 (which is what IEE 754 uses).
Some languages may optionally use an exact representation (e.g., Scheme or Python), but that's not the general case.
There is one langauge that once I complained to the authors that it was broken: Lua.
Lua uses floating point by default, it can't make integer math at all, I made a game using Lua, back then DirectX had a bug where it would switch the floating point mode of the processor without permission, and sometimes the result was some extreme FPU imprecision.
Since Lua uses only the FPU for maths, even for integers, the result was that my game had lots of extremely bizarre results sometimes on Windows (while on Linux and OSX it worked as expected), it happened only once, but I saw 5+6 result in 13...
It was a really hard thing to debug, specially because of the flaming (when I asked about this on Lua IRC and forums, people flamed me endlessy, happily one guy in particular suggested me to check if it was the DX issue, and indeed it was).
> Lua uses floating point by default, it can't make integer math at all
JavaScript does this too. The fact is that with 64 bit floating point numbers, you can exactly represent every integer from -9007199254740992 to 9007199254740992, which is larger than the range of `int` on most systems anyway. 5+6 resulting in 13 is not an artifact of using floating point numbers. CPUs or GPUs don't randomly lose precision like that.
32 bit floating point numbers can exactly represent every integer from -16777216 to 16777216.
By the way, your reply was one of the most common replies to me that quickly turned into flaming (people offending me in all way they could to tell me Lua was perfect, and I was seeing things, because 32-bit is enough for the integers I was using...)
Can it calculate 1/3 exactly? You can in trinary: It's 0.1.
Binary, decimal, trinary, or any other numbering system is no more accurate or "exact" than any other. They all have fractions that they cannot represent.
An explanation of the following useful fact:A fraction p/q is equal to a finite length floating point number in base b <=> q divides b^n <=> the prime factors of q are prime factors of b.
Proof if q divides b^n, then q * c = b^n => p/q = (p * c)/(q * c) = (p * c)/b^n = (p * c)e-n in base b.
Example: Express the fraction 7/18 (base 10), in base 60 and convert it to a floating point in base 60.
60 = 2^2 * 3^1 * 5^1, 18 = 2^1 * 3^2, look for c such that 18 * c is a power of 60. Observe that the factors of 60^2 all have exponents that are greater or equal than those of 18, so we can multiply 18 by the required factors to get (60^2). This way: 60^2 = (2^2 * 3 * 5)^2 = 2^4 * 3^2 * 5^2 = 18 * c = (2^1 * 3^2) * (2^3 * 3^0 * 5^2)= 18200, so we need to multiply by 200 the numerator and denominator of the initial fraction to get a new fraction whose denominator is a power of 60. Now 7/18 = (7 200)/(18 * 200) = 1400/(60^2) and 1400 = 23 * 60 + 20.
Finally 7/18 = 1400/(60^2) = 0.2360 in base 60.
We use the notation 00,01,02, ... 59 for the digits of base 60, so 2360 is a two digits number in base 60.
The asterisk used for multiplication is not shown in HN, don't know why.
I remember, once upon a time... a bug where we were trying to find the average credit rating (AAA, AA+, AA, etc) of a portfolio of companies. And we had this issue where, in a simple degenerate case, the average of AA+ and AA+ became AA (AA+ is expected). In the end, it was an issue of floating point error accounting! (This was in C++ btw on a probably not-so-familiar platform).
TLDR;
floor(13.0) != 13
floor(13.0) == 12
Each credit rating was mapped to a integer. We'd take the floating point average of all the integers and then floor it. The integral answer would be used to map back to a credit rating.
Let's say AA+ was mapped to the number 13. Well floor( (float(13) + float(13)) / 2.0) == 12. Not 13.
That's because floating point 13.0 is really 12.99999999999.
The fix was to add accounting for floating point error by adding in ULP (https://en.wikipedia.org/wiki/Unit_in_the_last_place) which in essence tips the representation of 13.0 to 13.00000000001 (or whatever the real number is).
This might be true on really weird platforms, but with IEEE-754 floating point this isn't true. Integers within a big range are exactly representable without any rounding error. Doing floor((float(13) + float(13))/2.0) should be exactly 13. It is entirely possible that the weird platform was supposed to be using IEEE-754 arithmetic but had a buggy math library (depressingly common).
In some cases, string conversion is different, and the same calculation is going on, but the output is different.
In some cases, the internal representation isn't binary floating point at all, so the errors resulting from approximating an exact decimal with a binary float don't occur.
I would like to see a language use decimal by default. Under the hood, it could use floating point for certain types of calculations where less precision is required. If I want to know the answer to 0.2 + 0.1, chances are I want the answer to be 0.3. If I'm writing performance-sensitive code, maybe I can drop down a level and opt for float(0.2) + float(0.1). Are any languages doing this currently?
Perl 6 uses rationals by default. They have the advantages of being base-agnostic, able to accurately represent any recurring digit expansion accurately regardless of eventual base, and also faster (since, especially if you normalize (convert to lowest terms) lazily, most operations are just a few integer instructions with no branching, looping, or bit-twiddling involved).
Scheme has rationals, but decimal literals aren't rational by default. You have to type #e in front of it, or express it as a fraction.
It really ought to be fixed at the implementation level (how much stuff would it really break if floatin point errors went away?), but failing that I'd love a macro that did it for me. I can't figure out how to write one robustly.
Floating point is (usually) about "closest representable value". That is, the normal floats form a finite subset of the real numbers, and a calculation like x + y will pick the closest element to the true result.
In this case, adding the closest floating point value to "0.1" (0x1999999999999a / 2^56) and the closest one to "0.2" (0x1999999999999a / 2^55) gives
(Rational numbers with a denominator that's a power of two have a finite decimal representation, and that's the full one for this number.)
However, when printing, one is either interested in a close-enough rounded value (e.g. like some languages print 0.3 by default, or how %.3f will explicitly limit to 3 digits), or an exact representation. The exact representation should be round-trippable, so when you read it back in, you get exactly the same value. In practice this means printing a float z as a decimal value such that the closest float to the exact decimal value (i.e. as a real number) is z. Obviously the full string above works because it is exact, but it is nice to be as short as possible. In this case, the floats immediately before and after our number are exactly
Of course you can always (use numbers), but then you have to use (/ 1 20) and (/ 2 10). "0.3" is still considered "inexact", because of the decimal point. I don't like this - finite decimals are perfectly rational numbers and trivially convertible, why can't the interpreter treat them as such?
Errr that's exactly what the page implies; it does not say nor imply "those that display .3 are doing it correctly internaly".
One of the cardinal rule of floating point in computing is that you never do direct comparison between values, whatever the language, so your test should never ever be used.
That doesn't mean a language cannot be allowed to infer what the correct display of a floating point value should be.
For C# i see a different behaviour as in his example.
double test = 0.1 + 0.2; //will be 0.30000000000000004 if you check in the Debugger
Debug.Print(test.ToString());//will print 0.3
The Common Lisp version is slightly misleading because computations are made by default with single-float types, whereas there is also short-float, double-float and long-float.
default for READ-DEFAULT-FLOAT-FORMAT is implementation dependent and can be changed by user.
In CL type float has subtypes single-float, double-float, short-float, and long-float. "Any two of them must be either disjoint types or the same type; if the same type, then any other types between them in the above ordering must also be the same type. For example, if the type single-float and the type long-float are the same type, then the type double-float must be the same type also."
If there is only one float representation it has to be `(typep x 'single-float)` but if the internal representation is different (double-float for example) it can be all of the types. At least this is how I understand the spec.
Indeed. Note also that there are recommended minimum precisions and exponent sizes, which are followed in practice by CL implementations. That means that if double and long are the same type, the precision and size is at least the one recommended for long-float.
Calculators have traditionally used base 10 internally, not base 2, what is the base of the numbers on that TI? That's the answer, if it's base 10, .1, .2 and .3 are all exact.
edit: The OP has accepted my pull request and this explanation is now directly available on the website.
It's a shame that the web page doesn't explain why floating point numbers are inaccurate when it comes to decimals. It's actually pretty simple. When you have a base 10 system (like ours), it can only express fractions that use a prime factor of the base. The prime factors of 10 are 2 and 5. So 1/2, 1/4, 1/5, 1/8, and 1/10 can all be expressed cleanly because the denominators all use prime factors of 10. In contrast, 1/3, 1/6, and 1/7 are all repeating decimals because their denominators use a prime factor of 3 or 7.
In binary (or base 2), the only prime factor is 2. So you can only express fractions cleanly which only contain 2 as a prime factor. In binary, 1/2, 1/4, 1/8 would all be expressed cleanly as decimals. While, 1/5 or 1/10 would be repeating decimals.
So 0.1 and 0.2 (1/10 and 1/5) while clean decimals in a base 10 system, are repeating decimals in the base 2 system the computer is operating in. When you do math on these repeating decimals, you end up with leftovers which carry over when you convert the computer's base 2 (binary) number into a more human readable base 10 number.
Spot on. I would add to this explanation that the real confusion comes from the fact that in most languages when you write `0.1`, it doesn't mean 0.1 = 1/10 – instead it means the closest number to 0.1 of the form n/2^k where n < 2^53. Namely 0.1000000000000000055511151231257827021181583404541015625, which we can represent in decimal because 2 divides 10. And, of course, the illusion that `0.1` is actually 0.1 is supported by the great efforts [1] made to print floating-point values with the least number of digits necessary to reproduce them, so `0.1` still looks like 0.1 when you print it even though it isn't.
In decimal if you only have 5 digits for mantissa and you write 2/3, it becomes 0.66667 which is a little bit more than 2/3. Similarly, if you write the binary number 0.0000'0000'1 in decimal with 5 digits for mantissa, it becomes 0.0039063 instead of 0.00390625. If in some faraway land they use binary math, they would likely ask why such a simple binary number got rounded off.
THANK YOU! That is one of the best explanations on that matter I have seen. I've never considered why floating points are inaccurate, I just took it for a given (which I hate doing). Now I understand why, and so my own wisdom has increased ever so slightly. Today is a good day.
Notice the repeating pattern? The exact number must have infinite number of binary digits. Wherever you cut (whichever fixed number of bits you want to keep) you lose something.
To live in a world where information is not only accessible, but where you can also contribute to it easily is truly amazing. I use Git daily for source control, but I am always amazed when I see it used to manage content in an open source way.
So, this is a great explanation. I have never heard it expressed this way and I love it.
However, I am a little confused. In your first paragraph you say that 10's prime factors are 5 and 2, so 1/10 can be expressed cleanly. In paragraph two you say that binary only has a prime factor of 2, but 10 is divisible by 2, so why doesn't 1/10 express cleanly in binary?
In the fraction 1/10, the denominator is 10.
The prime factors of 10 are 2 and 5.
Binary does not contain the prime factor of 5 necessary to represent 1/10 cleanly as a decimal. It only contains the 2.
Similarly, the prime factors for 1/6 would be 3 and 2. In a base 10 number system, you only have the 2. So you end up with a repeating decimal (0.1666666).
I have been teaching programming for years, and love to point out that floating-point arithmetic isn't accurate. Many experienced programmers are shocked to discover the problems with floats.
But I never before had a good answer for which numbers would be accurate, and which wouldn't. This explanation was crystal clear and helpful. Thanks for providing it.
Au contraire, `0.1` and `0.2` are non-repeating decimals.
Their approximations may be repeating of course, and some languages indulge in such approximations, but there's nothing intrinsic about `0.1` that requires that it be approximated prior to storage rather than stored with 100% accuracy.
Many lisps do what it takes to store rationals as rationals, maintaining 100% accuracy. Perl 6 likewise treats literals of the form `1/10` or `0.1` as the rationals they are, again maintaining 100% accuracy.
Yes, but in order to have a non-repeating decimal expansion, the denominator must share _all_ of its factors with the base. That is, only fractions that, when reduced, have a denominator of the form 2^n*5^m (for natural numbers n, m) will terminate in base 10. The explanation given could probably be clearer.
Here's an example using perl5:
perl -e 'print 0.1+0.2' yields '0.3', which looks OK, however
perl -e 'print 0.1+0.2-0.3' reveals '5.55111512312578e-17'
So, it's better to print the result of 0.1+0.2-0.3
For what its worth, perl6 passes this test with flying colours.