Ollama works fine with multi-gpu setups. Since rocm 6.3 everything is stable and you can mix different GPU generations. The performance is good enough for the models to be useful.
The only thing which doesn't work well is running on iGPUs. It might work but it's very unstable.
Good to know. Still, is it viable option?
Buying i.e. AMD Threadripper for 2.5k$, motherboard and ram for 2k$ and i.e. 4 GPUs for 4k$ to have total of 96GB of VRAM? Total should be around 10k$ which is roughly price of Intel GPU specifically for AI if I am not mistaken? Which option would be better performance wise.
I did never saw a comparison anywhere and this is too much money to make fun experiment over the weekend.
> 10k$ which is roughly price of Intel GPU specifically for AI
Huh? The largest vram card that Intel has is the A770 which is around $350. What exactly are you trying to compare against? Are you doing inference only or training?
The only thing which doesn't work well is running on iGPUs. It might work but it's very unstable.