Is calling a 48-core machine a "personal computer" a bit of a stretch, or am I missing something?
A couple years ago, we picked up a HP Z800 workstation with two 12-core processors and 24GB of RAM for $400.
The main downside is that if the power supply (proprietary) or motherboard die, there's not easy fix for it other than buying another of the same model.
I wanted something to host my private and experimental stuff. A friend of mine who tracks used server auctions pointed me at some servers that were Dell systems clearly custom built for one of the large SV companies. I paid $450 for a box with 72GB of RAM and 16 cores. At that price, I just bought a second one for spares. Which I haven't needed to use in the 4 or 5 years I've been using it.
It has been great.
I'm just a regular computer user.
(Mind you, it doesn’t take any more technical knowledgeability than most users to bargain-hunt like you have; but knowing that the product can be of varying quality at a given price, and caring enough to put in the time to get something of good quality, makes you an exception, in the same way that people who e.g. comparison-shop for quality suits or shoes are an exception. It’s a general mentality that most people don’t subscribe to—neither the “buying the best” of the rich, nor the “economizing” of the poor, but rather the optimizing ROI of the economics-minded. And even an economist will only apply this kind of thinking to cases where they actually believe this thing will affect their life enough that the research time-cost is worth it—i.e. when it’s a hobby of theirs. Economics-minded computer hobbyists are a rather more niche set than “regular computer users!”)
I've used my 2014 MBP with a 4 core i7 (plus hyperthreading) plus 3 desktop i5 boxes for over 24 hours straight on the same XGB task before. The 8 CPU threads on the MBP lets it outperform more recent machines on this kind of task.
I think the point of "regular" was that it's not exactly some extreme budget needed to do this.
I mean, yes, I can see how an individual can get one. And it is definitely neat to see them pushed to their limits. But calling yourself a regular computer user with a machine that strong is a stretch.
You're also one of "those" people
That's the rub: deep learning is restricted to CUDA/cuDNN/NVIDIA GPUs, which exclude a large amount of desktops with AMD graphics (e.g. all modern Macs).
Current cloud GPU prices have dropped enough such that this approach may be pretty effective with spot/preemptible instances.
Developing competitive CUDA equivalent is much harder task than what you assume. There needs to be lots of tooling and need for optimizations in software to make these low level libraries high performance. The amount of testing and benchmarking to make the right choices may take many people working full time. Implementing the functionality is not enough.
AMD has had apis for a long time and it has an has an open source deep learning stack but it's not good enough. AMD also has CUDA to HIP converter but the results are not competitive and miss features.
Both AMD and Intel are probably getting there eventually and Nvidia is cashing the monopoly phase before the prices stop.
i’d be thrilled to have the option of doing deep learning on an AMD card even if it ran at 1/3 the speed on a comparable (but probably somewhat more expensive) NVIDIA card. it would open up a lot of options even if it were still less economical in throughput/dollar.
if nothing else, how many machine learning researchers would like to prototype things on their MacBook Pros?
They are even ok for some kinds of training (eg, if you are doing transfer learning with a fixed embedding/feature representation and have a pretty small parameter space to learn).
But it's so easy to fire up a cloud instance at Paperspace or somewhere and push some code across.
And whose fault is that?
I have a replication including cloud formation scripts here https://github.com/cshenton/neuroevolution
Can u elaborate more on the network requirements?
How much bandwidth is required? (I presume latency is less of a problem)