Hacker News new | past | comments | ask | show | jobs | submit | page 2 login
0.30000000000000004 (30000000000000004.com)
765 points by beznet 6 days ago | hide | past | web | favorite | 400 comments



Also the subject of one of the most popular questions on StackOverflow: https://stackoverflow.com/q/588004/5987

While it's true that floating point has its limitations, this stuff about not using it for money seems overblown to me. I've worked in finance for many years, and it really doesn't matter that much. There are de minimis clauses in contracts that basically say "forget about the fractions of a cent". Of course it might still trip up your position checking code, but that's easily fixed with a tiny tolerance.

when the fractions actually dont matter... its so painless just to just store everything in pennies rather than dollars (multiply everything by 100)

It’s not painless. E.g. dividing $100.00 by 12 month in integer cents requires 11 $8.33 and one $8.37 (or better 4x(2x8.33+8.34), depending on definition of ‘better’). You can forget this $0.04, but it will jump around in reports until you get rid of it – it requires someone’s attention anyway, no matter how small it is. Otoh, in unrounded floating point that will lead to a mismatch between (integer) payments and calculations. In rounded fp it’s the same problem, except when you’re trying very hard for error bits to accumulate (like cross-multiplying dataset sums with no intermediate rounding, which is nonsense in financial calc and where regular fixpoint integers will overflow anyway).

What I’m trying to show here is that both integers and floating point are not suitable for doing ‘simple’ financial math. But we get used to this Bresenhamming in integers and do not perceive it as solving an error correction problem.


This struck home with me when one day a friend and I bought the same thing and he paid a penny more.

I realized something I didn't ever notice or appreciate in 20+ years: oh yeah, they can't just round off the penny in their favour every time. And the code that handles tracking when to charge someone an extra penny must be irritating to have developed and manage. All of a sudden you've got state.


What kind of thing and store was it?

That's one of the worst domain name ever. When the topic comes along, I always remember about "that single-serving website with a domain name that looks like a number" and then take a surprisingly long time searching for it.

I have written a test framework and I am quite familiar with these problems, and comparing floating point numbers is a PITA. I had users complaining that 0.3 is not 0.3.

The code managing these comparisons turned out to be more complex than expected. The idea is that values are represented as ranges, so, for example, the IEEE-754 "0.3" is represented as ]0.299~, 0.300~[ which makes it equal to a true 0.3, because 0.3 is within that range.


> That's one of the worst domain name ever.

Maybe the creator's theory is that people will search for 0.30000000000000004 when they run into it after running their code.


It may be the worst domain name ever, but the site only exists because I thought that using "0" as a subdomain was a neat trick, and worked back from there to figure out what to do with it.

FWIW - the only way I can ever find my own website is by searching for it in my github repositories. So I definitely agree, it's not a terribly memorable domain.


It's the first result for "floating point site" on Google. Sure the domain itself is impossible to remember, but you don't have to remember the actual number, just what it stands for.

Remember filter bubble. My first result is not your first result. (although in this case it happens to be, but we both probably search a lot on programming)

Also did it in an InPrivate window to confirm, which is still somewhat targeted but far less so than on my actual account. It's still first.

And, at the end of the day, even if there's a filter bubble and it's the reason I see it first, then so what? The people looking for this site are likely going to fit into the same set of targeted demographics as you and me and most people on this site. So unless you also want to cater to 65-year old retirees that don't care about computer science and what floating numbers are, then why does the filter bubble even matter?


> My first result is not your first result.

It would be if you both used DuckDuckGo, though :)


> That's one of the worst domain name ever. When the topic comes along, I always remember about "that single-serving website with a domain name that looks like a number" and then take a surprisingly long time searching for it.

That's why we need regular expressions support in every search box, browser history, bookmarks and Google included.



just add 0.1 and 0.2 in fp32 (?) accuracy if you can't remember the name :)

This is the double-precision IEEE sum. A single-precision result would have (slightly less than) half as many digits.

This is a good thing to be aware of.

Also the "field" of floating point numbers is not commutative†, (can run on JS console:)

x=0;for (let i=0; i<10000; i++) { x+=0.0000000000000000001; }; x+=1

--> 1.000000000000001

x=1;for (let i=0; i<10000; i++) { x+=0.0000000000000000001; };

--> 1

Although most of the time a+b===b+a can be relied on. And for most of the stuff we do on the web it's fine!††

† edit: Please s/commutative/associative/, thanks for the comments below.

†† edit: that's wrong! Replace with (a+b)+c === a+(b+c)


Note that the addition is commutative [1], i.e. a+b==b+a always.

What is failing is associativity, i.e. (a+b)+c==a+(b+c)

For example

(.0000000000000001 + .0000000000000001 ) + 1.0

--> 1.0000000000000002

.0000000000000001 + (.0000000000000001 + 1.0)

--> 1.0

In your example, you are mixing both properties,

(.0000000000000001 + .0000000000000001) + 1.0

--> 1.0000000000000002

(1.0 + .0000000000000001) + .0000000000000001

--> 1.0

but the difference is caused by the lack of associativity, not by the lack of commutativity.

[1] Perhaps you must exclude -0.0. I think it is commutative even with -0.0, but I'm never 100% sure.


I tried to determine how to perform IEEE 754 addition (in order to see whether it's commutative) by reading the standard: https://sci-hub.tw/10.1109/IEEESTD.2019.8766229

(Well, it's a big document. I searched for the string "addition", which occurs just 41 times.)

I failed, but I believe I can show that the standard requires addition to be commutative in all cases:

1. "Clause 5 of this standard specifies the result of a single arithmetic operation." (§10.1)

2. "All conforming implementations of this standard shall provide the operations listed in this clause for all supported arithmetic formats, except as stated below. Unless otherwise specified, each of the computational operations specified by this standard that returns a numeric result shall be performed as if it first produced an intermediate result correct to infinite precision and with unbounded range, and then rounded that intermediate result, if necessary, to fit in the destination’s format" (§5.1)

Obviously, addition of real numbers is commutative, so the intermediate result produced for addition(a,b) must be equal to that produced for addition(b,a). I hope, but cannot guarantee, that the rounding applied to that intermediate result would not then depend on the order of operands provided to the addition operator.

3. "The operation addition(x, y) computes x+y. The preferred exponent is min(Q(x), Q(y))." (§5.4.1). This is the entire definition of addition, as far as I could find. (It's also defined, just above this statement, as being a general-computational operation. According to §5.1, a general-computational operation is one which produces floating-point or integer results, rounds all results according to §4, and might signal floating-point exceptions according to §7.)

4. The standard encourages programming language implementations to treat IEEE 754 addition as commutative (§10.4):

> A language implementation preserves the literal meaning of the source code by, for example:

> - Applying the properties of real numbers to floating-point expressions only when they preserve numerical results and flags raised:

> -- Applying the commutative law only to operations, such as addition and multiplication, for which neither the numerical values of the results, nor the representations of the results, depend on the order of the operands.

> -- Applying the associative or distributive laws only when they preserve numerical results and flags raised.

> -- Applying the identity laws (0 + x and 1 × x) only when they preserve numerical results and flags raised.

This looks like a guarantee that, in IEEE 754 addition, "the representation of the result" (i.e. the sign/exponent/significand triple, or a special infinite or NaN value - §3.2) does not "depend on the order of the operands". §3.2 specifically allows an implementation to map multiple bitstrings ("encodings") to a single "representation", so it's possible that the bit pattern of the result of an addition may differ depending on the order of the addends.

5. "Except for the quantize operation, the value of a floating-point result (and hence its cohort) is determined by the operation and the operands’ values; it is never dependent on the representation or encoding of an operand."

"The selection of a particular representation for a floating-point result is dependent on the operands’ representations, as described below, but is not affected by their encoding." (both from §5.2)

HOWEVER...

6. §6, dealing with infinite and NaN values, implicitly contemplates that there might be a distinction between addition(a,b) and addition(b,a):

> Operations on infinite operands are usually exact and therefore signal no exceptions, including, among others,

> - addition(∞, x), addition(x, ∞), subtraction(∞, x), or subtraction(x, ∞), for finite x (§6.1)


> Also the "field" of floating point numbers is not commutative, (can run on JS console:)

OK.

    >> x = 0;
       0
    >> for (let i=0; i<10000; i++) { x+=0.0000000000000000001; };
       1.0000000000000924e-15
    >> x + 1
       1.000000000000001
    >> 1 + x
       1.000000000000001

You've identified a problem, but it isn't that addition is noncommutative.

Yeah, what is demonstrated here is that floating point addition is nonassociative.

Your example shows that floating-point addition isn't associative, not that it isn't commutative.

Isn't that more of an associativity problem than a commutativity problem, though?

1.0 + 1e-16 == 1e-16 + 1.0 == 1.0 as well as 1.0 + 1e-15 == 1e-15 + 1.0 == 1.000000000000001

however (1.0 + (1e-16 + 1e-16)) == 1.0 + 2e-16 == 1.0000000000000002, whereas ((1.0 + 1e-16) + 1e-16) == 1.0 + 1e-16 == 1.0


Yep. The TL;DR of a numerical analysis class I took is that if you're going to sum a list of floats, sort it by increasing numeric value first so that the tiny values aren't rounded to zero every time.

Really? It wasn't to use Kahan summation?

https://en.wikipedia.org/wiki/Kahan_summation_algorithm


Hah! Well, yeah, that too. But if there's a gun to your head, sorting the list before adding will get you most of the way there with the least amount of work.

I feel like it should really be emphasised that the reason this occurs is due to a mismatch between binary exponentiation and decimal exponentiation.

0.1 = 1 × 10^-1, but there is no integer significand s and integer exponent e such that 0.1 = s × 2^e.

When this issue comes up, people seem to often talk about fixing it by using decimal floats or fixed-point numbers (using some 10^x divisor). If you change the base, you solve the problem of representing 0.1, but whatever base you choose, you're going to have unrepresentable rationals. Base 2 fails to represent 1/10 just as base 10 fails to represent 1/3. All you're doing by using something based around the number 10 is supporting numbers that we expect to be able to write on paper, not solving some fundamental issue of number representation.

Also, binary-coded decimal is irrelevant. The thing you're wanting to change is which base is used, not how any integers are represented in memory.


Agree. All of these floating point quirks are not actually problems if you think of them as being finite precision approximations to real numbers, not in any particular base. Just like physical measurements of continuous quantities. You wouldn't be surprised to find an error in the 15th significant figure of some measurement or attempt to compare them for equality or whatever. So don't do it with floating point numbers either and everything will work perfectly.

Yes, there are some exceptions where you can reliably compare equality or get exact decimal values or whatever, but those are kind of hacks that you can only take advantage of by breaking the abstraction.


If you only use decimals in your application, it actually is a fix because you can store the numbers you care about in exact precision. Of course it's not really a fix if you're being pedantic but for a lot of simple UI stuff it's good enough.

One small tip about printf for floating point numbers. In addition to "%f", you can also print them using "%g". While the precision specifier in %f refers to digits after the decimal period, in %g the precision refers to the number of significant digits. The %g version is also allowed to use exponential notation, which often results in more pleasant-looking output than %f.

   printf("%.4g", 1.125e10) --> 1.125e+10
   printf("%.4f", 1.125e10) --> 11250000000.0000

And %e always uses exponential notation. Then there's %a, which can be exact for binary floats.

One of my favorite things about Perl 6 is that decimal-looking literals are stored as rationals. If you actually want a float, you have to use scientific notation.

Edit: Oh wait, it's listed in the main article under Raku. Forgot about the name change.


That’s only formatting.

The other (and more important) matter, — that is not even mentioned, — is comparison. E. g. in “rational by default in this specific case” languages (Perl 6),

  > 0.1+0.2==0.3
  True
Or, APL (now they are floats there! But comparison is special)

      0.1+0.2
  0.3
      ⎕PP←20 ⋄ 0.1+0.2
  0.30000000000000004
      (0.1+0.2) ≡ 0.3
  1

Exactly what are the rules for the "special comparison" in APL? That sounds horrifying to me.

Assume the values could be equal if the relative error of the operation is greater than a small predefined value (called “⎕ct”, comparison tolerance, and you can change it).

but this is not an equivalence relation. You may have a=b and b=c but a!=c

it's horrifying!


The runner up for length is FORTRAN with: 0.300000000000000000000000000000000039

And the length (but not value) winner is GO with: 0.299999999999999988897769753748434595763683319091796875


Those look like the same length

Huh? The fortran one is 38 characters long with 33 0s after the 3. The go one is 56 characters long with 15 9s after the 2.

I’m on mobile. Must be the issue.

Postgresql figured this out many years ago with their Decimal/Numeric type. It can handle any size number and it performs fractional arithmetic perfectly accurately - how amazingly for the 21st Century! Is comically tragic to me that all of the mainstream programming languages are still so far behind, so primitive that they do not have a native accurate number type that can handle fractions.

> how amazingly for the 21st Century!

Most languages have classes for that, some had them for decades in fact. Hardware floating point numbers target performance and most likely beat any of those classes by orders of magnitude.


> It's actually pretty simple

The explanation then goes on to be very complex. e.g. "it can only express fractions that use a prime factor of the base".

Please don't say things like this when explaining things to people, it makes them feel stupid if it doesn't click with the first explanation.

I suggest instead "It's actually rather interesting".


For including words in a sale pitch, I'd agree.

But this isn't a sales pitch. Some people are just bad at things. The explanation on that page require grade school levels of math. I think math that's taught in grade school can be objectively called simple. Some people suck at math. That's ok.

I'm very geeky. I get geeky things. Many times geeky things can be very simple to me.

I went to a dance lesson. I'm terribly uncoordinated physically. They taught me a very 'simple' dance step. The class got it right away. The more physically able got it in 3 minutes. It took me a long time to get, having to repeat the beginner class many times.

Instead of being self absorbed and expect the rest of the world to anticipate every one of my possible ego-dystonic sensibilities, I simply accepted I'm not good at that. It makes it easier for me and for the rest of the world.

The reality is, just like the explanation and the dance step, they are simple because they are relatively simple for the field.

I think such over-sensitivity is based on a combination of expecting never to encounter ego-dystonic events/words, which is unrealistic and removes many/most growth opportunities in life, and the idea that things we don't know can be simple (basically, reality is complicated). I think we've gotten so used to catering to the lowest common denominator, we've forgotten that it's ok for people to feel stupid/ugly/silly/embarrassed/etc. Those bad feelings are normal, feeling them is ok, and they should help guide us in life, not be something to run from or get upset if someone didn't anticipate your ego-dystonic reaction to objectively correct usage of words.


When faced with criticism about your lack of inclusivity, what's to gain by doubling down in order to intentionally exclude people? The argument you are presenting always feels disingenuous because you imply that there is something lost in the efforts to be more inclusive.

The idea that you care about the growth of people you are actively excluding doesn't make a whole lot of sense. In this example we're talking about word choice. The over-sensitivity from my point of view is in the person who takes offense that someone criticized their language and refuses to adapt out of some feigned interest for the disadvantaged party. The parent succinctly critiqued the word choice of the author and offered an alternative that doesn't detract from the message in the slightest.

The lowest common denominator is the person who throws their arms up when offered valid criticism.


> because you imply that there is something lost in the efforts to be more inclusive

Yes there is something lost. I included it in my post but I'll repeat it: People who aren't good at math are 'shielded from the truth' (they objectively suck at math because they can't grasp something that is objectively simple in the domain of math). Again, feeling bad about not grasping something simple is the necessary element for a humbling experience. Humbling experiences aren't meant to feel great. For me, I've learned the most with humbling experiences. I honestly believe most people in the first world need more of them.

The suggested language is more inclusive, that's an advantage to sales, but less clear, that's a disadvantage to communication/learning. Personally, I like learning and want to see things optimized for that.

BTW; I loved the sly way of you implying that: A- I took offense (I am not, nor do I see anything in my comment that says I'm offended) and B- That I'm the lowest common denominator because of A. It's a subtle way of attacking me and not my point. It says a lot about both the person doing the attack and the strength of their argument that they have to resort to ad-hominems. Though I will credit you with using a smartly disguised one.

Also, you are speaking to me as if I was the website author. I'm not the OP of the article, which if you read TFA you would see the actual author changed in favor of the suggestion.


The problem is that almost everything is simple once you understand it. Once you understand something, you think it's pretty simple to explain it.

On the other hand, people say "it's actually pretty simple" to encourage someone to listen to the explanation rather than to give up before they even heard anything, as we often do.


I understand prime factors just fine, but I'd never think it's "simple" to bring them up when I'm explaining how decimal points work.

Thanks for this.

Yep, I've thrown 10,000 round house kicks and can teach you to do one. It's so easy.

In reality, it will be super awkward, possibly hurt, and you'll fall on your ass one or more times trying to do it.


Ditto as I now feel stupid.

I read the rest of your reply but I also haven’t let go of the possibility that we’re both (or precisely 100.000000001% of us collectively) are as thick as a stump.


To be fair, this is also done in every other STEM field, and CS is no exception.

We could all learn a lot more from each other if everything wasn't a contest all the time.


I learned it in college, but immediately forgot it after the exam. Why? It wasn't simple.

I had to use google translate for this one, because I didn't suspect the translation to my language to be so literal.

My take is that this sentence is badly worded. How do these fractions specifically use those prime factors?

Apparently the idea is that a fraction 1/N, where N is a prime factor of the base, is rational in that base.

So for base 10, at least 1/2 and 1/5 have to be rational.

And given that a product of rational numbers is rational, no matter what combination of those two you multiply, you'll get a number rational in base 10, so 1/2 * 1/2 = 1/4 is rational, (1/2)^3 = 1/8 is rational etc.

Same thing goes for the sum of course.

So apparently those fractions use those prime factors by being a product of their reciprocals, which isn't mentioned here but should have been.


Thanks for the suggestion. I've updated the text.

Awesome, thanks!

>Why does this happen? It's actually rather interesting.

Did the text change in the last 15 minutes?




It's nice but it's extremely overkill for understanding this particular problem.

there are only two kind of problems, trivial problems and those that you don't know how to solve (yet).

I still remember when I encountered this and nobody else in the office knew about it either. We speculated about broken CPUs and compilers until somebody found a newsgroup post that explained everything. Makes me wonder why we haven't switched to a better floating point model in the last decades. It will probably be slower but a lot of problems could be avoided.

Unless you have a floating point model that supports arbitrary bases, you're always going to have the issue. Binary floats are unable to represent 1/10 just as decimal floats are unable to represent 1/3.

And in case anyone's wondering about handling it by representing the repeating digits instead, here's the decimal representation of 1/12345 using repeating digits:

  0.0[0008100445524503847711624139327663021466180639935196435803969218307006885378
  69582827055488051842851356824625354394491697043337383556095585257189145402997164
  84406642365330093155123531794248683677602268124746861077359254759011745646010530
  57918185500202511138112596192790603483191575536654515998379910895099230457675172
  13446739570676387201296071283920615633859862292426083434588902389631429728635074
  92912110166059133252328878088294856217091940056703118671526933981368975293641150
  26326447954637505062778452814904819765087079789388416362899959497772377480761441
  87930336168489266909680032401782098015390846496557310652085864722559740785743215
  87687322802754151478331308221952207371405427298501417577966788173349534224382341
  02875658161198865937626569461320372620494127176994734710409072498987444309437019
  03604698258404212231672742]

Nice example. For those who do not understand why it is so long, a denominator multiplied by a period must be all-nines. E.g. 1/7 = 0.(142857), because 142857x7 = 999999, so that 0.(142857)x7 = 0.(999999) = 1 back again. For some simple numbers N their nearest 999...999/N integer counterpart is enormously huge.

> Unless you have a floating point model that supports arbitrary bases

See also binary coded decimals.

https://en.wikipedia.org/wiki/Binary-coded_decimal


That's not a floating point.

From the article:

> Programmable calculators manufactured by Texas Instruments, Hewlett-Packard, and others typically employ a floating-point BCD format, typically with two or three digits for the (decimal) exponent.


Then that's how they're encoding the components of the float. BCD itself is not a floating-point, it's just a different way of encoding a fixed-point or integer. If all you want to do is use floating point but expand the logarithm and mantissa then that's completely tangential to whether or not they're stored as BCD or regular binary values.

> Binary floats are unable to represent 1/10 just as decimal floats are unable to represent 1/3.

That is true, but most humans in this world expect 0.1 to be represented exactly but would not require 1/3 to be represented exactly. Because they are used to the quirks of the decimal point (and not of the binary point).

This is a social problem, not a technical one.


Decimal floating point is standardized since 2008:

https://en.wikipedia.org/wiki/Decimal_floating_point#IEEE_75...

But it's still not much used. E.g. for C++ it was proposed in 2012 for the first time

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n340...

then revised in 2014:

http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3871.ht...

...and... silence?

https://www.reddit.com/r/cpp/comments/8d59mr/any_update_on_t...


It's not important to most people, because decimal floating point only helps if your UI precision is exactly the same as your internal precision, which almost never happens.

Seeing the occasional 0.300000000000004 is a good reminder that your 0.3858372895939229 isn't accurate either.


> It's not important to most people

One can argue that nothing is important to most people.

The correct calculations involving money, up to the last cent, are in fact important for people who do them or who are supposed to use them. I've implemented them in the software I've made to preform some financial stuff even in eighties, in spite of all the software which used binary floating point routines. And, of course, from the computer manufacturers, at least IBM cares too:

https://www.ibm.com/support/pages/decimal-floating-point-use...

Apparently, there are even processors which supports these formats in hardware. It's just still not mainstream.


There is no "better floating point model" because floating point will always be floating point. Fixed point always has been and always will be an option if you don't like the exponential notation.

> Fixed point always has been and always will be an option

Not really. It would be really cool if fixed point number storage were an option... but I'm not aware of any popular language that provides it as a built-in primitive along with int and float, just as easy to use and choose as floats themselves.

Yes probably every language has libraries somewhere that let you do it where you have to learn a lot of function call names.

But it would be pretty cool to have a language with it built-in, e.g. for base-10 seven digits followed by two decimals:

  fixed(7,2) i;
  i = 395.25;
  i += 0.01;
And obviously supporting any desired base between 2 and 16. Someone please let me know if there is such primitive-level support in any mainstream language!

COBOL was created to serve the interests of the financial industry, therefore COBOL has fixed point as a first class data type.

Every programming language that has come since has been designed to be a general purpose programming language, therefore they don't include fixed point as a first class data type.

Therefore the financial industry continues to use COBOL.

Every time someone some tries to rewrite some crusty COBOL thing in the language de jure, they'll inevitably fuck up the rounding somewhere. The financial industry has complicated rounding rules. Or better yet, the reference implementation is buggy and the new version is right, but since the answers are different it's not accepted.


You don't need special support from the language, fixed-point arithmetic is effectively the same as integer.

DOOM's implementation of fixed-point is a good example which should work on any platform as long as sizeof(int) >= 4.

https://github.com/id-Software/DOOM/blob/master/linuxdoom-1.... https://github.com/id-Software/DOOM/blob/master/linuxdoom-1....

Addition and subtraction will work normally. Multiplication also works normally except you need to right-shift by FRAC_BITS afterwards (and probably also cast to a larger integer type beforehand to protect against overflow).

Division is somewhat difficult since integer division is not what you want to do. DOOM's solution was to cast to double, perform the division, and then convert that back to fixed-point by multiplying by 1.0 before casting back to integer. This seems like cheating since it's using floating-point as an intermediate type, but it is safe to do because 64-bit floating point can represent all 32-bit integers. As long as you're on a platform with an FPU it's probably also faster than trying to roll your own division implementation.


Writing a fixed point class in C++ is pretty trivial [1]. It would have semantics and performance comparable to integer.

Fixed point classes have been proposed many times as addition to the standard library, but have yet to be voted in.

[1] see my comment elsethread about trivial problems.


Floating point is fundamentally a trade off between enumerable numbers (precision) and range between minimum/maximum numbers, it exists because fast operations on numbers are not possible with arbitrary precision constructs (you can easily have CPU/GPU operations where floating point numbers fit in registers, arbitrary precision by its very nature is arbitrarily large).

With many operations this trade off makes sense, however its critical to understand the limitations of the model.


> Makes me wonder why we haven't switched to a better floating point model in the last decades. It will probably be slower but a lot of problems could be avoided.

Pretty much all languages have some sort of decimal number. Few or none have made it the default because they're ignominiously slower than binary floating-point. To the extent that even languages which have made arbitrary precision integers their default firmly keep to binary floating-point.


> Few or none have made it the default because they're ignominiously slower than binary floating-point.

You can strike the "none". Perl 6 uses rationals (Rat) by default, someone once told me Haskell does the same, and Groovy uses BigDecimal.


Many languages have types for infinite-precision rational numbers, for example Rational in Haskell.

Wait, an entire office (presumably full of programmers) didn’t understand floating point representation? What office was this? Isn’t this topic covered first in every programming book or course where floating point math is covered?

This was in the 90s. We were all self taught and I had never met anyone who had formal CS education.

> Makes me wonder why we haven't switched to a better floating point model in the last decades.

The opposite.

Decimal floating points have been available in COBOL from the 1960s, but seem to have fallen out of favor in recent days. This might be a reason why bankers / financial data remains on ancient COBOL systems.

Fun fact: PowerPC systems still support decimal-floats natively (even the most recent POWER9). I presume IBM is selling many systems that natively need that decimal-float functionality.


Decimal floats are a lot older than COBOL. Many early relay computers (to the extent there were many such machines) used floating-point numbers with bi-quinary digits in the mantissa. https://en.wikipedia.org/wiki/Bi-quinary_coded_decimal

Being a lot slower is a worse problem than being off by an error of 2^60. And if it isn't, then you simply choose a different numeric type.

In JavaScript, you could use a library like decimal.js. For simple situations, could you not just convert the final result to a precision of 15 or less?

  > 0.1 + 0.2;
  < 0.30000000000000004

  > (0.1 + 0.2).toPrecision(15);
  < "0.300000000000000"
From Wikipedia: "If a decimal string with at most 15 significant digits is converted to IEEE 754 double-precision representation, and then converted back to a decimal string with the same number of digits, the final result should match the original string." --- https://en.wikipedia.org/wiki/Double-precision_floating-poin...

That is why I only used base 2310 for my floating point numbers :-). FWIW there are some really interesting decimal format floating point libraries out there (see http://speleotrove.com/decimal/ and https://github.com/MARTIMM/Decimal) and the early computers had decimal as a native type (https://en.wikipedia.org/wiki/Decimal_computer#Early_compute...)

The multiplication of the first 5 primes ;)

This is part of the reason Swift Numerics is helping to make it much nicer to do numerical computing in Swift.

https://swift.org/blog/numerics/


Swift also has decimal (so does objective-c) which handles this properly. See https://lists.swift.org/pipermail/swift-users/Week-of-Mon-20... to see how swift's implementation of decimal differs from obj-c.

what is the number representation in swift? Looking at your link it seems to be plain ieee floats. In that case, would't it have the same behavior?

This is a great shibboleth for identifying mature programmers who understand the complexity of computers, vs arrogant people who wonder aloud how systems developers and language designers could get such a "simple" thing wrong.

" vs arrogant people who wonder aloud how systems developers and language designers could get such a "simple" thing wrong."

I never heard anyone complain that it would be simple to fix. But complaining? Yes - and rightfully so. Not every webprogrammer need to know the hw details and don't want to, so it is understandable that this causes irritation.


Interesting, I searched for "1.2-1.0" on google. The calculator comes up and it briefly flashes 0.19999999999999996 (and no calculator buttons) before changing to 0.2. This happens inconsistently on reload.

Swi-Prolog (listed int he article) also supports rationals:

  ?- A is rationalize(0.1 + 0.2), format('~50f~n', [A]).
  0.30000000000000000000000000000000000000000000000000
  A = 3 rdiv 10.

This specific issue nearly drove me insane trying to debug a SQL -> C++/Scala/OCaml transpiler years ago. We were using the TPC-H benchmark as part of our test suite, and (unbeknownst to me), the validation parameters for one of the queries (Q6) triggered this behavior (0.6+0.1 != 0.7), but only in the C/Scala targets. OCaml (around which we had built most of our debugging infrastructure) handled the math correctly...

Fun times.


I wish high level languages (specifically python) would default to using decimal, and only use a float when cast specifically. From what I understand that would make things slower, but as a higher level language you're already making the trade of running things slower to be easier to understand.

That said, it's one of my favorite trivia gotchas.


When did RFC1035 get thrown under the bus? According to it, with respect to domain name labels, "They must start with a letter" (2.3.1).

Long, long ago. 3com.com wanted to exist.

Amazingly, 3.com apparently didn't want to exist.

All-digit host names have been allowed since 1989.

https://tools.ietf.org/html/rfc1123#page-13

One aspect of host name syntax is hereby changed: the restriction on the first character is relaxed to allow either a letter or a digit. Host software MUST support this more liberal syntax.


Huh. Thanks! I really missed the memo there. I wonder why 1035 doesn’t mention that it is updated-by 1123.

The same document defines `in-addr.arpa` domains that have numeric labels.

The mandate of a starting letter was for backwards compatibility, and mentions it in light of keeping names compatible with email servers and HOSTS files it was replacing.

Taking a numeric label risks incompatibility with antiquated systems, but I doubt it will effect any modern browser.


Ages ago I guess. 1password doesn't start with a letter either.

Fixed-point calculations seem to be somewhat of a lost art these days.

It used to be widespread because floating point processors were rare and any floating point computation was costly.

That's not longer the case and everyone seems to immediately use floating point arithmetic without being fully aware of the limitations and/or without considering the precision needed.


As soon as I've started developing real-life business apps I've started to dream about a POWER which is said to have hardware decimal type support. Javs's BigDecimal solves the problem on x86 but it is at least an order of magnitude more slow than FPU-accelerated types.

Well, if your decimals are fixed-point decimals, which is the case in finance, decimal calculations are very cheap integer calculations (with simple additional scaling in multiplication/division).

I just use Zarith (bignum library) in OCaml for decimal calculation, and pretty content with performance.

I don't think much domains needs decimal floating point that much, honestly, at least in finance and scientific calculations.

But I could be wrong, and would be interested in cases where decimal floating-point calculations are preferable over these done in decimal fixed-point or IEEE floating-point ones.


Why doesn't everybody do it this way then? We would probably have a transparent built-in decimal type in every major language by now if there were no problems with this.

> Why doesn't everybody do it this way then?

Why? Fintech uses decimal fixed-point all the way, there are libraries for them for any major language. Apps like GnuCash or ledger use them as well.


But Java has BigDecimal in its standard library and it's soooo slow I doubt it is implemented this way.

In the Go example, can someone explain the difference between the first and the last case?

There's a link right below. It seems like

1. Constants have arbitrary precision 2. When you assign them, they lose precision (example 2) 3. You can format at as a arbitrary precision in a string (example 3)

In that last example, they are getting 54 significant digits in base 10.


Thanks. What I didn’t realize is that although the sum is done precisely, the resulting 0.3 will be represented approximately once converted to float64. In the first case formatting hides that, in the last it doesn’t.

I think in the last example, it's going straight from arbitrary precision to 54 significant digit, bypassing float64 entirely, hence why it looks different from the middle example.

Mods: Can we have a top level menu option called "Floating point explained"?

Not surprisingly Common Lisp gets it right. I don’t mean this is snark (I don’t mean to imply you are a weenie if you don’t use lisp) but just to show that it picked a different kind of region in the language design domain.

The thing that surprised me the most (because I never learned any of this in school) was not just the lack of precision to represent some numbers, but that precision falls off a cliff for very large numbers.

Computer languages should default to fixed precision decimals and offer floats with special syntax (eg “0.1f32”).

The status quo is that even Excel defaults to floats and wrong calculations with dollars and cents are widespread.


TL;DR - 0.1 in Base 2 (binary) is the equivalent of 1/3 in Base 10 meaning, it’s a repeating decimal that causes rounding issues (0.333333 repeating)

This is why you should never do “does X == 0.1” because it might not evaluate accurately


I love how Awk, bc, and dc all DTRT. I wonder what postscript(/Ghostscript?) does.

Whoo go Ada, one of the few to get it right. Must be the goto for secure programming for a reason.

Take that Rust and C ; )


Happy to see ColdFusion doing it right. Also, good for Julia for having the support for fractions.

for Smalltalk, the list is not complete, it has scalled decimals and fractions too: 0.1s + 0.2s = 0.3s . (1/10) + (2/10) = (3/10)

Those Babylonians were ahead of their time.

Use Int types for programming logic.

Except when, you know, you can’t.

Curious, when can't you?

My mental model of floating-point types is that they are useful for scientific/numeric computations where values are sampled from a probability distribution and there is inherently noise, and not really useful for discrete/exact logic.


Right; for the former integer arithmetic won't do.

Yep, absolutely (and increasingly often people are using 16-bit floats on GPUs to go even faster).

But the person you replied to said programming logic, not programming anything.

Honestly I think if you care about the difference between `<` and `<=`, or if you use `==` ever, it's a red flag that floating-point numbers might be the wrong data type.


Why is D different than the rest?!

bc actually computes this correctly, and returns 0.3 for 0.1 + 0.2

This has been posted here many times before. It even got mocked on n-gate in 2017 http://n-gate.com/hackernews/2017/04/07/

Please check some of the online papers on Posit numbers and Unum computing, especially by John Gustafson. In general, Unums can represent more numbers, with less rounding, and fewer exceptions than floating points. Many software and hardware vendors are starting to do interesting work with Posits.

Probably one of the more in depth technical discussions of the pros and cons of the various proposals that John Gustafson has made over the years:

https://discourse.julialang.org/t/posits-a-new-approach-coul...


IEEE floating-point is disgusting. The non-determinism and illusion of accuracy is just wrong.

I use integer or fixed-point decimal if at all possible. If the algorithm needs floats, I convert it to work with integer or fixed-point decimal instead. (Or if possible, I see the decimal point as a "rendering concern" and just do the math in integers and leave the view to put the decimal by whatever my selected precision is.)


Depends on the field. 99.9000000001% of the time, the stuff I do is entirely insensitive to anything after the third decimal point. And for my use cases, IEEE 754 is a beautiful stroke of genius that handles almost everything I ask from it. That's generally the case for most applications. If it wasn't, it wouldn't be so overwhelmingly universally used.

But again, there are clearly plenty of use cases where it's insufficient, as you can vouch. I still don't think you can call it "disgusting", though.


IEEE is deterministic and (IMO) quite well thought-out. What specifically do you not like about it?

The fact that the most trivial floating-point addition of 0.1 + 0.2 = 0.300000000000004 was insufficient to make this seem HUMAN-nondeterministic to you? (I mean sure, if you thoroughly understood the entire spec, you might not be surprised by this result, but many people would be! Otherwise the original post and website would not exist, no?)

It’s kind of a hallmark of bad design when you have to go into a long-winded explanation of why even trivial use-case examples have “surprising” results.


⅓ to 3 decimal places is 0.333. 0.333 + 0.333 = 0.666, which is not ⅔ (to 3 decimal places, that is 0.667). That is all that is happening with the 0.1 + 0.2.

The word you're looking for is "surprising," which is a far cry from non-deterministic. IEEE 754 is so thoroughly deterministic that there exists compiler flags whose sole purpose is to say "I don't care that my result is going to be off by a couple of bits from IEEE 754."


You don't need to thoroughly understand the entire spec, nor do you need to know that 0.1 + 0.2 = 0.300000000000004. "Computers can't really represent floating point numbers exactly" is generally good enough. (Also: you added "human" as a qualifier; you didn't have that before so I responded to your statement as it was written.)

you may dislike IEEE floats for many reasons, but not for being non-deterministic. Their operations are described by completely deterministic rules.

Fixed point is perfectly OK, if all your numbers are within a few orders of magnitude (e.g. money)


I agree with this view, there's nothing more disgusting than non-determinism. The way computers rely on assumptions for the accuracy of a floating number is one that's contrary to the principles of logical thinking.

> The way computers rely on assumptions

The way people rely on assumptions.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: