
Build a fast deep learning machine for under $1K - kqr2
https://www.oreilly.com/learning/build-a-super-fast-deep-learning-machine-for-under-1000
======
gcp
Contrary to the claims there, the CPU does matter: if the network is
relatively small, it's easy to get bottle-necked feeding the GPU.

Same thing applies to storage, for the same reasons. There's a reason AMD is
selling deep learning cards with Flash drives built in.

The choice of CPU cooler is silly. The CPU chosen is a 65W model, which will
be cooled quietly even by the boxed cooler (and we don't need to care about
overclocking with a non-K). By his own admission, the rig draws 250W, so the
idea that 650W must be cooled is nonsense, and even then, only a part of that
is due to the CPU.

It's probably worth looking more closely at the GPU, notably, if that cooler
stays quiet when the card is under full load.

You do not need a monitor, keyboard or mouse to use such a machine. You can
just ssh into it. You might need to borrow a set to set up the BIOS.

If you "run out of memory for your applications" on the GPU, have you
considered simply lowering the mini-batch size? (See, this is why we got a
good CPU, it removes the hurt from feeding smaller batches)

~~~
nl
_There 's a reason AMD is selling deep learning cards with Flash drives built
in._

For those not familiar with them, this is the Radeon Pro SSG[1]

This has a built in M2 drive, but this is used as video card memory, NOT main
storage. It could possibly allow big (HUGE) models or batches, but doesn't
prove fast IO is often a bottleneck.

Practically no one[2] is using these for deep learning. They are marketed to
the oil and gas modelling and visualization market and notably the product
page (linked above) makes no mention of neural networks or deep learning, and
it has never been mentioned in AMD's publicity.

Yes, it _might_ be good for deep learning. And yet, modern deep models do want
more memory. But there is a lot that needs to happen for this to be useful:
AMD needs to release something like CuDNN and they need to make sure OpenCL is
supported as well as CUDA is.

[1] [https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-
ssg-g...](https://arstechnica.com/gadgets/2016/07/amd-radeon-pro-ssg-graphics-
card-specs-price-release-date/), [http://www.amd.com/en-us/press-
releases/Pages/amd-radeon-pro...](http://www.amd.com/en-us/press-
releases/Pages/amd-radeon-pro-2016jul25.aspx)

[2] I'm sure you'll find someone. But show me a published paper or any kind,
or even benchmarks showing the use of the extra memory somewhere.

~~~
deepnotderp
MIOpen is AMD's cuDNN alternative.

~~~
rockdoe
Which does not actually exist. It's announced but unreleased.

------
mrmondo
My home server has an i7 4790K, 32GB DDR3 and... a GTX 1080 Gamer Edition
that's doing... well, bugger all. Why is the graphics card in there you might
ask? Well, it was sitting on my desk collecting dust as I wait patiently for a
Pascal driver for eGPUs on macOS but Nvidia had delayed these for long enough
that I thought I'd chuck it in my home server, it is being used by ffmpeg for
x265/HVEC transcoding with Plex media server but I'm really not even touching
it's capabilities - not even close when transcoding a 120GB+ 4K 120FPS video
and streaming it wirelessly to my TV that's running a Plex client. The server
is running CentOS 7 and has 10x 8TB SATA drives in BTRFS RAID 10 for storage,
two Intel NVMe 750 series 1.2 TB SSDs for the OS as well as a couple of 2TB
Crucial MX300 SATA SSDs that aren't currently in use for much.

So, I have been sitting here wondering how I can make better use of it while
it's practically parked in my server while I wait for some Pascal drivers to
drop. I'm not so interested in robots / live video processing but it'd be neat
if I could leverage its power for something fun or to assist the servers
compute performance in some other ways that might be useful or interesting. I
was running KVM with PCIe pass through passing the GPU to a guest VM running
Steam as a steam streaming box but I got sick of having an OS on the network
that I had to worry about Virus's, Malware and annoyingly slow and unreliable
updates being installed when you least want it to happen and I got lazy and
went back to playing PlayStation when I needed some time out from constructive
work / research.

Open to ideas to any software I could tinker with if anyone has any
interesting suggestions?

~~~
j_s
Can you ballpark today's price tag for such a setup? There seem to be several
pricing tiers floating around in this discussion and I'd appreciate the chance
to see how your setup fits in without having to shop for it.

~~~
mrmondo
Well it's all cheap kit with the exception of the disks, I guess you could
just go online and have a look at what those parts cost you in your country
etc...

------
dsacco
I think there should be a disclaimer here that while, yes, this will _work_ ,
it will quickly become suboptimal (perhaps even dysfunctional) for a lot of
work as you scale your hobby into something resembling more professional work.
I personally would not start with a thousand dollar machine if you have any
intention of doing serious computation in the future. While you can trade out
parts pretty easy on a custom built machine, you don't want to be in a
position where you have to trade out most of the core parts 6 months in.

For reference, I built a home PC that I successfully do deep learning and data
analysis on (mostly tensorflow and scipy stack) for about ~$10k. It's liquid
cooled, has 15 fans, four radiators, an i7-6900K CPU, 128GB RAM, four GTX 1080
GPUs (controversial), four TBs of HDD space and 1TB of SSD space. I don't
recommend you _start_ with this at all, but my point is that porting your
hardware from point A to point B will be a pain if it comes to it.

I used the guide here as a reference about 8 months ago when I built it:
[http://graphific.github.io/posts/building-a-deep-learning-
dr...](http://graphific.github.io/posts/building-a-deep-learning-dream-
machine/). My purpose in doing this was, essentially, to pay for electricity
rather than AWS/GCP/Azure compute resources (and in that regard it's been very
successful!).

I know I'm hijacking a thread here to talk about building home machines for
professional deep learning work when this story is clearly not intended for
that, but I wanted to throw in this perspective so that it's understood this
is very different from just "build this machine to start out and upgrade it
later." There's a law of diminishing returns here, but in general my point is
that I do _not_ think this is a minimum for "start doing deep learning
effectively at home." If you want to learn hands on deep learning cheaply, my
opinion is that it would be more efficient to use compute resources from a
cloud provider before diving into this with a home-based custom machine.

tl;dr: The demographic of folks who probably want/should/need to build a home
deep learning machine probably has little overlap with the demographic of
folks who want to do it non-professionally, or at least with only $1k in
resources.

~~~
dragandj
On the other hand, by the time anyone starting just now learn enough ML to be
dangerous, the price of the $10,000 hardware would drop to $5,000 or less, so
starting with the cheapest yet decent option seems a better approach...

~~~
dsacco
Fair point. I'm just quibbling about where "decent" begins, and adding in the
point about ease of upgrades. Replacing a GPU is pretty easy. Replacing a
motherboard because you don't have enough PCIe slots will require tearing down
the majority of the machine and rebuilding it.

The latter is more likely to happen if you start down this path at rock bottom
prices.

~~~
lloyd-christmas
I began building one a year and a half ago, starting with a 5930k, 1x980ti,
32GB. This put me in a position to incrementally upgrade it starting at
~$2.5k. That being said, at this point the smallest AWS p2 instances are
equivalent to my current setup, and it would take 150 days worth of running my
hardware to make it cost-effective vs. AWS.

~~~
visarga
When you build your first DL box you also get a moral boost. If you think
every second is counted and penalized against you, you might use it less (such
as when you're using cloud servers). But if it is already yours bought and
paid for, then you can afford to run any crazy idea. A good compromise would
be a local DL box and extra virtual machines on demand, for when you need to
run extensive hyperparameter searches. If you make the leap and invest, at
least you commit your money into the direction of ML, strengthening your
resolve.

~~~
lloyd-christmas
On the flip side, dropping $2.5k for a hobby you haven't investigated yet and
then looking back 6 months later on the computer collecting dust will likely
dissuade you from investing any time into potential future hobbies. I play
piano, and I'd never encourage anyone to go drop $2.5k on an electric keyboard
just to provide a motivation. A decision to pick up a new hobby/skill should
be based on desire, not a handcuff.

------
brudgers
If I were building a GPU box on the cheap, I'd start with a used Dell
Precision T7xxx series off of Ebay for a <$300 including RAM and a Xeon or
two. They have 1100 watt power supplies and lots of slots and the RAM is
usually ECC. Odds are it will come with DVD burner and possibly a hard disk
and a Windows license...again all for a couple of hundred dollars.

Leaving $700 for GPU's while providing reliable high end hardware pretty much
designed to run GPU's for the base platform.

------
PascLeRasc
I'm not sure why mITX was chosen - in my experience it's almost always more
expensive and leaves no room for additional GPU expansion (Or network
adapters/whatever card you might need) in the future. You can get 1151 mATX
motherboards for more like $60-70 on Amazon instead of the $125 they paid.

Also, paying $125 for 16GB of ram but not spending the extra $20 to have your
CPU be able to overclock? I'm not sure where OP is building but I can find
that much RAM for $20 less than they paid, and overclocking isn't really that
difficult or unreliable these days.

~~~
coroxout
I was wondering that too. For something like this maximum room for future
expansion seems good. In the past I've used mITX to build low-power, low-noise
machines, rather than building a high-powered processing beast.

On the other hand, it made the article more interesting for me. I can't really
justify a home machine learning rig, but I was thinking I might soon replace
that old mITX machine.

(edit - just reached the pics - really not sure why you'd go with a mini form
factor and then add a 6" heatsink/fan!)

~~~
PascLeRasc
I also prefer mITX and I have one I've tried to optimize for low noise and
efficiency. Just seems ill-suited for this purpose.

------
singularity2001
In Europe it's also very common to just choose the components in a web shop
and have them assembled, also for about $1000. (Not looking at Dell)

Anyone interested in deep learning should go that path instead of burning
their money on Amazon.

~~~
ProxCoques
+1 on that. There are a number of online build-to-order shops here in the UK
which offer a truly vast range of component combinations. Better still, the
order process automatically checks for compatibility with, for example, your
choice of motherboard.

Once you have the configuration you want, you just pay up and it arrives
professionally assembled ready to rock. I don't think I'll ever by an off-the-
shelf machine again.

~~~
IanCal
I found the premium on these was quite high, although even just buying all the
components from a single place added a fair amount to my recent build.

Compatibility, price checking and searching are all available with
PcPartPicker (UK site:
[https://uk.pcpartpicker.com/](https://uk.pcpartpicker.com/)) which saved me
an astonishing amount of time.

The hour or two building the machine were worth the few hundred it saved me,
but everyone has different priorities.

~~~
Coding_Cat
I've found that the premium depends on the store and model. Some seem to do
relative pricing, others absolute. In the latter case, if you want a big rig
it's best to let someone build it for you.

Often times in those cases you also get great service because a) if you build
PCs for a living you're a computer geek and it's fun to build an insane PC and
b) they often use their biggest systems as advertisement. At least, that's
what I've seen.

+cable management is like black magic to me. If I were to take the money saved
as payment for me to get it as nice as those places get it, I'd be below
minimum wage.

------
kordless
I've been considering a "thing" that allows monetization of an AI-enabled
machine by others who need the capacity to do larger scale testing. Anyone
interested in this? My contact info is in my profile.

~~~
minimaxir
You won't be able to match price/expenses (fixed with hardware or variable
with upkeep) of AWS/GCP due to their economy of scale. Even startups who are
doing similar things sublicense from AWS.

------
amelius
But how do you scale cheaply to more GPUs?

~~~
mtanski
Get a ATX instead of ITX mobo. It's likely is cheaper as well.

------
RichardHeart
If by "super fast deep learning machine" you actually mean, mid tier gaming
computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to
7x the video cards in a full size ATX computer using pci-e risers, while
paying less for the board.

Don't lose 5 percent of your cpu overclock or more by having an itx friendly
cooler instead of a D15 noctua or better.

Choose your video card wisely, AMD does some things better than NVIDIA
depending on what software you're running.

If you chose an ITX platform for deep learning fun, uh, you should really add
the couple shoeboxes extra space to your platform and have 4x-7x the power
available to upgrade into.

~~~
siscia
Try to bring good in the conversation.

What piece you would have use? Why? What tradeoffs you have considered?

You sound expert in the field, try to share your knowledge with the community
in a constructive way so that we can all benefit from it.

I develop software for living and still I haven't any clue of what you said,
while I followed quite well the article and I thought he was making reasonable
choice.

Please show me better.

~~~
croon
Not parent, but I've built my own computers for almost 20 years.

As the articles author himself questions, he should have gotten the marginally
more expensive CPU, and definitely the GPU with more RAM.

* The overclockable CPU doesn't just mean that you very easily could get a 10%+ performance boost without much work, but also that you often (depending on your specific chip) can lower the voltage and make it run much cooler/quieter, which is something I increasingly care about if I'm using it a lot.

* He writes that fitting the model in RAM is basically the most important part, but then saves less than a dinner out by basically halving the RAM on the GPU he bought. The chip is otherwise the same though, so performance is only dependent on whether you fill that RAM or not.

* The Noctua D15 the parent mentioned is a CPU cooler where Noctua is a long standing high performant brand, and D15 is a specific model with a 15 inch fan and comparable sized heat sink. There are of course other brands, but I myself also usually end up with Noctuas. The reason it's important is that however fast you can dissipate heat from the cpu/case, the less chance of throttling, and the larger headroom for potential overclocks you get.

Airflow and room for larger heatsinks is also why he recommended not going for
an ITX. A linked benefit is again the potential for a quieter system.

I haven't gone much into ML (yet), but I currently have a system with:

* I7 6700K (the difference to I5 6600K being higher base clock and hyperthreading, which is more important to computational work than to gaming, so if you have the money, definitely go for the I7)

* 32GB DDR4 (as author mentioned, RAM is cheap). The clock/timings on RAM isn't really as important, but try to find the best you can find for a given price point.

* An Nvidia GTX 1080: It's not Titan X or Z, but almost, at less price. It definitely blows the budget for a $1000 system, but I agree that the entire 10 __series is good.

If the limit is a firm $1000, I would get something like this:

[https://pcpartpicker.com/list/XHV9Fd](https://pcpartpicker.com/list/XHV9Fd)

And if more funds is available, I'd get more storage and RAM, then a better
CPU, then a better GPU, then maybe bump the chassis up to an R5 (same brand),
possibly another motherboard. In that order. There's always something better,
so you compromise based on budget.

~~~
mamcx
Was years ago when I build my last PC and I'm out of the game.

I'm tempted in build a hackintosh:

[https://www.tonymacx86.com/threads/hackintosh-cutting-
edge-k...](https://www.tonymacx86.com/threads/hackintosh-cutting-edge-kaby-
lake-nvme-ddr4-4k-viable.214249/#post-1433129)

\- Intel i7 Kaby Lake \- No decided on motherboard. The one that cause me less
trouble (for hackintosh) is fine. \- GTI 750ti (have) or buy a pascal nvidia.
\- NVMe drive if possible \- 32 GB RAM. \- Probably a Thermaltake CORE P3
case. Not decided.

I was thinking in use a Liquid Cooler but wonder if the Noctua could be
better/less noise?

~~~
croon
> I was thinking in use a Liquid Cooler but wonder if the Noctua could be
> better/less noise?

Depends, is your option of water cooling an All-In-One-solution that have
become popular in recent years? Their performance is on par or slightly better
than a large heat sink + large low speed fan(s), but they're not generally
quieter, as you still have fans for them, as well as a pump.

I considered those options when building mine too, and as I wans't too
enthusiastic about assembling my own water cooling system, I went for a large
air cooled heatsink instead (the D15). No risk of leakage or pump failure, and
proven performance/low noise.

~~~
mamcx
I was planning in use Corsair Hydro H60
([http://amzn.to/2k9a9O1](http://amzn.to/2k9a9O1)).

However, I wish to have a quiet system, and my brother have it and it sound
louder than I wish. I don't plan on overcloking.

~~~
croon
Yeah, that's one of those AIO water coolers I was referring to.

[http://www.anandtech.com/show/5054/corsair-hydro-
series-h60-...](http://www.anandtech.com/show/5054/corsair-hydro-
series-h60-h80-and-h100-reviewed/6)

The "Silver Arrow" is an air cooler from Thermalright that is pretty
equivalent to a Noctua or a Phantek or be Quiet! etc.

In that review it beats the H60 in both temperature and noise. The H60 is more
than twice as loud.

So yes, unless you assemble your own water cooling system, I'd say definitely
go for a regular heatsink+fan.

------
throwawayish
If you're on a budget it makes little sense if any to buy things like a
motherboard, CPU or memory new.

