How many people do you think are 'into' ML enough to spend $1k+ on a SOTA gpu and associated hardware? I am slowly getting there having moved from Colab to my own set up...
However... Ethereum enabled/lured many everyday joes into buying 4+ $1k cards due to the incentive structure of; 'buy mining rig then buy a lambo.' Setting up a GPT-NeoX is orders of magnitude more difficult than pointing GPU compute at Nicehash. I really have a hard time thinking ML will have any meaningful uptake in that regard because the incentive structure isn't the same and its much, much harder.
Big cloud seems to be going there own way in regards to compute. GPU's are great for ML, but that doesn't mean they will always hold the crown. TPU's, NVMe storage, and chiplets may find a better path with newer software.
I just don't see how Nvidia really thrives without drastically reducing price (less margin). I don't think they are dead, but they are in big trouble as are many companies.
Correct, MBP's can run stable diffusion and other ML workloads on non-nvidia hardware. I clearly see this becoming a trend. GPT-J, Neo and NeoX run really well on Colab TPU's, again these are not made by Nvidia.
Training is dominated by Nvidia, I will not question that as most papers I have seen say something similar. I will say that I do not believe training will always be dominated by Nvidia's datacenter options. Two things that will hasten the withdraw from Nvidia; Cuda and hardware advances around the motherboard (ASICs, RAM proximity, PCIe lanes, data transfer planes, etc).
Think about this... what if a company released an ML training/Inference ASIC that used regular DDR4/NVMe, performed like 4 x A100's and cost $8000? Would you be interested? I would! I don't think this is too far off, there has to be someone working on this outside of Google, Apple and Meta.
However... Ethereum enabled/lured many everyday joes into buying 4+ $1k cards due to the incentive structure of; 'buy mining rig then buy a lambo.' Setting up a GPT-NeoX is orders of magnitude more difficult than pointing GPU compute at Nicehash. I really have a hard time thinking ML will have any meaningful uptake in that regard because the incentive structure isn't the same and its much, much harder.
Big cloud seems to be going there own way in regards to compute. GPU's are great for ML, but that doesn't mean they will always hold the crown. TPU's, NVMe storage, and chiplets may find a better path with newer software.
I just don't see how Nvidia really thrives without drastically reducing price (less margin). I don't think they are dead, but they are in big trouble as are many companies.