Ah, yeah. The performance of modern FPUs really kills, and it's changing a lot of the assumptions I used to have about performance.
Anecdote: In state of the art physics code, it's common to use integers for coordinates [1]. Recently, I was working on a BSc. toy project, and I used ints for coordinates as is common. With both ints and floats it's important to be careful around functions like `tan` that go to infinity, but of course floats are more forgiving, so I prototyped some of the code using doubles.
I ended up comparing performance and it wasn't even funny. Double precision arithmetic was anywhere between 3x (where a good int algorithm is known) all the way to 100x faster (if the int algorithm is cordic, for example) than integers.
The number 1 supercomputer in the world has 8 million GPU cores (or is it 8 million GPUs? Execution Units?) and 600k CPU cores.
They are doing floating point math on GPUs. Numerical analysis [1] is used to create the best accuracy possible. This is something that has been done for thousands of years, basically since math has had problems with no "exact" solution.
> you probably don't use that for things like excel
I'm not sure when I read it, but long ago I read some kind of AMA from a MS engineer working on Excel saying that his greatest achievement was working on the team that made the Excel's DAG solver trivially parallelizable. In the same thread, he mentioned that offloading to the GPU was being looked into. I guess it never came to fruition.
On the other hand the CPU has floating point math too, and floating point math is MUCH faster than decimal math.
So his point holds if you replace GPU with CPU, but his use of GPU is likely inaccurate.