
Ask HN: Building a powerful ML workstation vs. consuming it via cloud providers? - tzhenghao
I can&#x27;t decide between buying hardware, assembling and building a &quot;gaming&quot; rig for deep learning applications, and consuming compute resources via a cloud provider? Which is a more economical method?<p>I&#x27;d prefer the latter since I won&#x27;t have to worry about maintaining hardware, but I heard the prices are pretty high and there&#x27;s slowdown on some AWS GPU instances? Thoughts?
======
Eridrus
How much are you going to use your workstation for ML/games?

If you're just getting started and don't have a specific application in mind
you should just play with some smaller problems, use Colab and/or use the $300
in GCP credits Google keeps advertising.

If you're going to be doing this for an extended amount of time, then buying
hardware will be better; you can get some good deals on second hand high end
GPUs on eBay.

The main risk is that you buy a bunch of gear, and then don't use it.

------
selljamhere
TL;DR - Building your own rig is fun, but can be a pain to maintain. There is
a path on Google Cloud to start ML projects for free, and scale up when you
need to.

I think you're hitting the biggest up/down sides on the nose.

There is an innate joy to building your own rig (I'm guilty of it myself), and
it does provide a fair amount of flexibility. The two main downsides that come
to mind are the up front cost for parts, and the maintenance required to keep
the system up to date. I've had success with a CentOS rig running nvidia-
docker[1]. The annoyances mount over time though, as OS and Nvidia driver
releases can be a pain to keep up to date, especially if you're working with
several of the major frameworks. This is the main reason I've moved most of my
ML projects to the cloud.

Cloud has many benefits including scalability and minimal maintenance. There
are many options in the cloud ML space, but my current favorite leverages
Google's cloud. I'll get into detail below, but the main reason is that they
have several tools that build a natural progression and will grow with you as
your projects mature.

First Steps:

I'd recommend checking out Colab[2][3] to get your feet wet. It includes a
FREE GPU instance, and is a Jupyter based notebook, so you can easily utilize
any of the python based frameworks. (1/2 of a K80, if memory serves. 12 GB) As
with all free resources, there are some caveats. Colab is designed for
interactive usage, so they don't encourage long running processes. However,
the general consensus online is that you can get around 12 hours of training
in at a time before the instance restarts itself, which is plenty of time to
kick off a job and save checkpoints to cloud storage for resuming later. All
in all, it's a great tool get a project off the ground. One bit of strangeness
(from a software engineer's perspective): the notebooks are stored in google
drive. This adds an interesting possibility for collaboration (it's in the
name, right?), but leaves something to be wanted in terms of source control.

Growth:

If Colab isn't cutting it, there are a couple next steps. If you need more
compute power, or have long running training jobs, the Colab UI can actually
be run locally backed by your own GCE instance (with attached GPUs)[4]. If you
need additional services (databases, map/reduce, etc) Datalab[5] is Colab's
bigger beefier cousin, with all sorts of integrations across the Google Cloud
Platform, as well as 3rd party providers.

EDIT: Link formatting

[1] [https://github.com/NVIDIA/nvidia-
docker](https://github.com/NVIDIA/nvidia-docker)

[2]
[https://colab.research.google.com/notebooks/welcome.ipynb#re...](https://colab.research.google.com/notebooks/welcome.ipynb#recent=true)

[3]
[https://research.google.com/colaboratory/faq.html](https://research.google.com/colaboratory/faq.html)

[4] [https://research.google.com/colaboratory/local-
runtimes.html](https://research.google.com/colaboratory/local-runtimes.html)

[5] [https://cloud.google.com/datalab/](https://cloud.google.com/datalab/)

