We are speculating here now. I though this discussion was about the status quo; not what we imagine soon to be.
Even if the bandwidth bottleneck gets solved in the future ... scenes from movie sets usually are big enough so transformations are stored as double precision matrices to avoid objects starting to jitter when they are far away from the origin.
Have you checked double precision GFLOPs on you favorite GPUs lately? And then compared those and their prices to some Ryzen 3970X CPU specs and their prices?
Just for example in the PRMan 3.9 release notes from 2001, it says under 'Miscellaneous Changes': "Some parts of the renderer now use double-precision floating point arithmetic, to avoid round-off error."[1]
Your model-to-camera to camera matrix needs to be only single precision, usually.
But your model-to-world matrix needs to be double precision to avoid jittering.
So to calculate the former you use double precision and then you can truncate the resulting matrix to single precision for use e.g. in shaders. Everything dandy by then, even for GPUs.
But first you need to get there somehow.
So if you have a gazillion instances in your scene, particles, blades of grass, leaves of tress, spaceships, whatever, you need a gazillion matrix multiplications with double precision to build your acceleration structure and to actually start generating pixels.
It's one of many reasons why GPU based renderers performance goes to shit, particularly on time to first pixel, when scenes of such complexity get thrown at them. Contemporary GPUs have comparatively shitty f64 performance.
Edit: added example PRMan changelog backing up the claim in the 1st sentence. This was most likely considering just xforms. For ray-tracing specific issues and f64 see e.g. the PBR book on solutions to ray intersection precision challenges.
Even if the bandwidth bottleneck gets solved in the future ... scenes from movie sets usually are big enough so transformations are stored as double precision matrices to avoid objects starting to jitter when they are far away from the origin.
Have you checked double precision GFLOPs on you favorite GPUs lately? And then compared those and their prices to some Ryzen 3970X CPU specs and their prices?