Hacker News new | past | comments | ask | show | jobs | submit login

It doesn't take much imagination to construct an example. Knowing 0.1 isn't exactly represented make a formula with it that should be exactly a half cent. Depending on fp I precision it will be slightly above or below a half cent and rounding will not work as expected. We found this in prod at a rate of hundreds to thousands of times per day it only takes volume to surface unlikely errors.



I thought I was clear that the result was rounded. The fp imprecision affects the correctness of rounding e.g. half to nearest even.


It is the rounding that seems to be an issue here.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: