Hacker News new | past | comments | ask | show | jobs | submit login

Or your numbers are wrong because you didn't wrap in parentheses.

Example, put these two formulas in excel:

=(4/3 - 1)*3 - 1

=((4/3 - 1)*3 - 1)




The online version, at least, seems to be wrong when you do wrap it in parentheses:

=(4/3 - 1)*3 - 1 = 0

=((4/3 - 1)*3 - 1) = -2.22045E-16


Check out page 3-5 of this paper called How Futile are Mindless Assessments of Roundoff in Floating-Point Computation? [0]. It explains what is happening here better than I can and is where I got this example from.

[0]: https://people.eecs.berkeley.edu/~wkahan/Mindless.pdf




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: