Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Is that the central shortcoming in binary floating-point math?
1 point by newbie-02 on Aug 23, 2024 | hide | past | favorite
Hi, I'm struggeling with bin-FP imprecision ( think not the first one who does ), and think to have identified one central weakness.

IEEE 754 standard delivers consistent arithmetic for a wide range of 'english plumber metrics' ( imperial system of measurement, halves, quarters, eights, sixteenths and so on ). Up to where it runs out of bits and switches into rounding.

Alas decimal fractions are encoded in a way that most of them fall in that weak category, and operation results often need additional fit-in rounding.

Rounded results aren't qualified for associativity and distributivity, the math with them is 'broken', is not qualified arithmetic from mathematical POV.

In consequence the problems for calculations with decimal fractions are not limited to extreme cases or peripheral areas, but occur freely scattered even in the simplest calculations. ( 2 times 64 bit fail to add 0.2 and 0.1 correctly, embarrassing )

Alas plenty of programs / users try substituting decimal fraction math with bin-FP, and then go from trap to trap.

1. question: is the above analysis / estimation / pinpointing correct?

2. question: is someone aware of a source in literature or web where this has been formulated explicit and clearly?

3. PLEASE! no discussions about how negligibly small the deviations are, they mark the end of mathematical correctness and the beginning of problems. That's it.



Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: