I dunno... $8000 builds a 64c/128t 256 GB RAM workstation with the same GPU these researchers used (https://pcpartpicker.com/list/P6WTL2). That's arguably in the realm of home computer for just about anyone making $90,000 and above, I would think; I would also think anyone working in those fields could command at least that salary or greater, unless they're truly entry level positions. Seems it would be a reasonable investment for someone actively working in the area of machine learning / artificial intelligence.
They used an Nvidia 2080 TI, which is expensive today, but rumours are that the entry level 3XXX series will be similar to the 2080 TI in performance. That should be dropping in about a month!
In a ML setup your GPU is going to be the workhorse. I'd spend less on the CPU and memory and spend more on the GPU. Throwing something together REALLY quick (I'm being lazy and not checking) I'd go something closer to this https://pcpartpicker.com/list/T9Ds27 because of the graphics card. But getting a Ti would improve a lot just for GPU memory. You could get a lot done with a machine like this though (I assume it would be good for gaming too. Disclosure: not a gamer).
tldr: upgrade GPU, downgrade CPU and ram to keep similar pricing.
This depends quite a lot on the domain. In some image processing tasks you can actually be cpu-bound during dataloading. So either you get tons of RAM and preload the dataset, or you use more cores to queue up batches. You still need a good GPU and generally I'd agree to prioritise that first.
You can get CPU bound, I am $100 under and you could put money towards that or RAM. I did also leave a path to upgrade RAM. But that said, I've been working with image generation a lot lately and CPU really isn't my bottleneck.
If doing image processing (or NLP with rec nets), I wouldn't save on VRAM. 11GB minimum (2080/1080TI). Otherwise you can't even run the bigger nets with good image resolution.
I think the more important cuts that went unmentioned are the CPU cooler and mobo. You could arguably cut the CPU cooler completely, since these Ryzen CPUs include one.
The mobo cut is also a pretty useful savings, though it will be an obstacle to multiple-GPU setups.
Also, the parent comment misleadingly suggests that a 3900X costs less than $300. That seems like an error in pcpartpicker, since clicking through reveals a true price of $400+.
well an m5ad.24xlarge is 96 threads and 384G with your own 2xSSD (saves on EBS bandwidth costs). So fewer threads but a bit more memory. (we'll guess that is a 48 core EPYC 7642 equivalent with 96 threads since there is no 96 core version)
How does having your own SSD save on bandwidth? You have to send the data to and from the cores where the computation is taking place.
There are certainly some workloads where it makes sense to own your own storage and rent computation, but you can't assume that by default for a "powerful AI" workload.
There is no exact cloud equivalent - the researchers used commodity hardware for their GPU, something that NVIDIA doesn't allow for use in data centres.
The closest you can get on AWS (more like System #3 in the paper with 4x GPUs) would be something like a p3.8xlarge instance [1] that'll cost you $12.24 (on demand) or $3.65 to $5 (spot price, region dependent) [2].
A single GPU instance (p3.2xlarge) only 16 vCores, though) will cost you $3.06 on-demand or $0.90 to $1.20 (spot).
They mention multiple architectures, including down to a 10 core, 128GB ram, and 1080Ti setup. Next setup is improved CPU, double ram, and a 2080Ti. Third setup is 4 2080Tis. See table A.1
For shallower models CPUs aren't overshadowed by GPUs as much - but beyond a certain # of parameters the CPU loses out as GPUs do vector math highly efficiently.
"The team also tested their approach on a collection of 30 challenges in DeepMind Lab using a more powerful 36-core 4-GPU machine. The resulting AI significantly outperformed the original AI that DeepMind used to tackle the challenge, which was trained on a large computing cluster."
Well, they presumably tested the same CPU with 4 GPUs (2080 Ti I think) - maybe they wanted to compare.
Paper: https://arxiv.org/abs/2006.11751
Source: https://github.com/alex-petrenko/sample-factory