Not a mac/UMA discussion point, but is it time to add additional, installable, DDR5 to GPUs? I can see this as a win/loose. PCIe 5x16 is close to maxing out the bandwidth available from high end dual channel DDR5 now, but not quite. I'm not a hardware person but I suspect putting it on the card could lead to significant performance improvements over using system ram so allowing systems like this, where MOE weights are shed, to get even higher performance than just adding that DDR5 to the system. Bigger models become closer to reality and it provides more of a pathway for developing technologies that take advantage of it. Of course the loose side is that you just put a lot of specialized ram on a card instead of into the system where it could be used for other things. I could see a place for a 16GB card with 64GB(or more) of DDR5 especially if we start seeing MOE and similar technologies really start being designed for this concept.
Probably not with DIMM modules, as longer traces mean higher latency (speed of light is ~30 cm in 1ns). GDDR typically uses larger buses (more wires) for higher bandwidth, even more so for HBM, so DIMM would be hard. Maybe CAMM would be up to the task?
It certainly seems feasible from an engineering perspective (though it does make cooling harder), at least for mid-range, not H100-class HW, but it prevents market segmentation, so EOMs may not be too interested (as long as no competitor does it).
