
Ask HN: Anybody using Amazon Machine Image for AWS Deep Learning? - DrNuke
Hello, I do not own GPU hardware and am trying to understand if public amis on AWS are worth the fuss at professional level or at least good enough for the likes of course.fast.ai moocs. Newest here https:&#x2F;&#x2F;aws.amazon.com&#x2F;marketplace&#x2F;pp&#x2F;B06VSPXKDX for Ubuntu. Any experience? Thanks!
======
bhouston
GPUs in the cloud right now are so pricy, in part I think because NVIDIA is
making people put in K40s and K80 - the Quadro and Telsa class cards that are
super expensive.

We started to put on premise machines that have GPUs in them to reduce our
costs. We can get a desktop class Core i7 machine with a GeForce 1060 6GB for
around $800 all in, where as on Amazon you have to have a G2 2.xlarge on
demand it is $400/month, or $3478 one year term (that is for 1/4 of a G2
8.xlarge achine, thus you get one of its 4 Quadro K5000 4GB cards to
yourself.) I could put 2 1060s in that machine for another $200 or so. This
just beats AWS pricing in <4 months of use.

My expectation is that these 1060s will be good for two years, maybe even
three.

Because our GPU tasks are low bandwidth and non-critical we can run them on
our existing office internet. Our rent also includes power. So there are not
too many additional costs over the bare machine costs.

~~~
mabbo
I think you're definitely winning on price- provided you plan to run things
continuously for that time period. The AWS GPU instances are probably not
aimed at you though.

It's far more likely to be aimed at the Master's student in ML who has this
new awesome model but it takes goddamn forever to train on his laptop and he
just needs 3 days of time with a big-ass GPU instance.

He spins up the instances, runs his model, spins it down and is out what, $100
maybe? That fits into his limited budget.

Additional benefit: if I buy a kickass GPU, I'm going to reduce my overall
productivity in research because now I can finally play some of these hot new
games on full graphics!

~~~
chrisfosterelli
> It's far more likely to be aimed at the Master's student in ML

As a counter example, I'm a master's student in ML and my lab wants to push
more for an onsite machine since it's easier to expense one machine than a
bunch of monthly AWS bills for each student in the lab :)

Additionally, most Master's students (at least in Canada) already have access
to one or more very large compute clusters for jobs like this. I have used AWS
a few times when I was short on time though!

------
jph00
To answer your question about the
[http://course.fast.ai](http://course.fast.ai) MOOC: the course provides an
AMI that has everything you need pre-installed. Just follow the directions in
the setup video, which includes a script that sets up the instance for you.

To save around 80% (but slightly less convenient) there are two ways to use
spot instances provided on [http://forums.fast.ai](http://forums.fast.ai) that
are largely automated and work well.

In part 2 of the course you'll learn how to set up your own server. As others
have mentioned that's much better value if you're using deep learning somewhat
regularly.

(I'm the founder/teacher of this course.)

~~~
shoshin23
I can totally vouch for the ease with which you can setup your deep learning
AMI. I followed the instructions on the site and the scripts provided by
Jeremy abstract a lot of the stuff away. Super cool work!

------
rocketcity
I have been using it a good bit for some of the deep learning assignments on
Siraj Raval's channel
([https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A](https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A)).
I have had a really good experience so far. The trick is to find the AMI id
for whichever region you want to use the image in and then just put in a spot
request for your instance. I just checked the current going rate for a P2.XL
in us-east-1 and it was at 0.2355 per hour.

~~~
ShirsenduK
Siraj is _the_ bomb! Fellow wizard here :)

------
Fripplebubby
I've been using this AMI for CNTK since the AMI was made available recently.
It's been working well for me - they seem to update the AMI for big tagged
releases of the libraries, and all the CUDA drivers are good to go out of the
box. Personally I found it easier to use than Docker images, and both are
easier than manual install.

With a spot request you can get a p2.xlarge for around $0.22/hr, which for me
is sufficient for moocs and side projects, but for beefier instances the
prices get high.

------
KaiserPro
I'm using them at $work. But they are expensive, but so is EC2.

Unless you are able to get a decent deal (far better than reserved instances)
then for raw CPU/GPU power hosting your own is better. Its the whole package
that makes AWS attractive. (RDS, S3, instant spinup)

Spot prices is a good place to start if you are doing batches. just make sure
you checkpoint little and often.

However if you are on your own, with limited funds, Get a second hand
workstation (dell or hp z620/600) they tend to have decent PSUs, loads of ram
and decent support, and a couple of Geforces with more Vram. (Quadros you are
paying for the double float performance and ram. transfer Bandwidth and speed
are the same or slightly better on geforce class cards.)

------
minimaxir
A helpful on-topic note: Amazon has 1x K80 cloud GPU instances for $0.90/hr
(not prorated), but very recently, Google announced GPU instances of the same
caliber ([https://cloud.google.com/compute/docs/gpus/add-
gpus](https://cloud.google.com/compute/docs/gpus/add-gpus)) for ~$0.75/hr
(prorated to the minute), although Google Compute Engine requires configuring
the GPU drivers first.

------
uberneo
If you are just getting started then
[https://www.floydhub.com/](https://www.floydhub.com/) is a good place as they
provide initial 100 mins on GPU for free and then later on approx 50% cheaper
than AWS

~~~
neil1023
Correction: 100 hours _

------
chaosmail
I used the ami with a p2.large for a deep learning project at university. In
my opinion it was quite pricey and reading from disk storage was
disappointingly slow (we had a 60gb dataset). What i found really great was
using the same machine first as a t2.micro instance for setup and testing s3
connection, a m4.large for some preprocessing and finally the p2 for deep
learning, all on the same AMI and SSD.

------
kshnell
www.paperspace.com/ml has much more affordable GPUs than AWS and a new Pascal
card. There's a pre-configured Deep Learning template and a couple interesting
features like a browser based terminal and desktop (you can still SSH if you
prefer).

~~~
dkobran
Paperspace co-founder here -- happy to discuss our AWS alternative which is
focused on affordability, simplicity and performance. Recently, we've had a
huge influx of people in moocs offloading their work to our GPUs/cloud. Would
love any feedback you have.

------
dhruvp
Hey OP! I've been using it extensively and it's definitely more than good
enough for most tasks you'll be doing. If you're going this route, I'd highly
recommend following the Keras' authors guide on how to set up the instance and
connect to it with a jupyter notebook: [https://blog.keras.io/running-jupyter-
notebooks-on-gpu-on-aw...](https://blog.keras.io/running-jupyter-notebooks-on-
gpu-on-aws-a-starter-guide.html). This will boost your productivity a ton in
the long run.

Best of luck!

------
ap46
Just get the GTX 1060 6GB variant & that should be enough RAM to run most
models without suffering pagination.

------
chrischen
A pascal titan x is the best card for deep learning.

Even my gtx 1080 is faster, albeit with slightly less ram than the K80 or M60s
available in the cloud.

A titan X would fix the ram issue though.

The most economical would probably be to colocate some titan X boxes. Not only
cheaper, but more percormant than anything you can get from IaaS right now.

------
lekker
GPU hardware on Amazon is quite pricey. I would be careful since it can get
out of hand rather quickly.

------
farhanhubble
I'm using p2.xlarge for Udacity Self Driving card nanodegree and Fast.AI
assignments. It's super pricey as everyone has mentioned but I prefer using
AWS to owning my own rig because I can download large datasets quickly.

------
hackpert
Yeah. I mostly end up using p2.xlarge spot instances, and they have worked
fine for my jobs so far. Be sure to checkpoint often and you'll be okay with
spot instances.

------
Baeocystin
No. CUDA-capable hardware is dirt cheap, and once you have your own, you can
run experiments 24/7.

------
benmanns
I don't have experience with the Amazon AMIs for deep learning, but I'll give
the tip that you can use spot instances with limited lifespans (x instance at
$y price for at most z hours) to reduce cost and the risk you forget to shut
something down.

~~~
Fripplebubby
Doing spot requests is a great way to keep the costs down if you just want to
rent a GPU for a while. But bear in mind that GPU instance types tend to spike
in price all-at-once when somebody brings a huge compute job online, which
will exceed your spot request's max bid with no warning and immediately
terminate your instance. So make sure whatever you're doing can be
interrupted!

------
gm-conspiracy
So, what is a recommended, buildable GPU rig (bang for buck)?

i7 w/ dual 1060 GPUs?

~~~
lloyd-christmas
my year and a half old rig is 2x 980ti and a 5930k, which is also my generic
home computer. I only use AWS when it's a long running job and the weather is
hot. It can really heat up your home on long running jobs, so keep that in
mind.

~~~
StavrosK
I've been wondering: Is information processing "free" in terms of energy? I.e.
if I put 100 watts in my GPU to make it do computations, will it heat my room
for 100 watts' worth of heat?

I'm guessing yes, because where else is the energy going to go, but then we
should probably be using heaters that are made of old/cheap GPUs, rather than
just a resistor.

~~~
ivan_ah
As a Canadian, I always feel good running compute-intensive jobs in the
winter. It's heating as usual, with a side-benefit of computation.

~~~
Neliquat
I have a system to duct in, out or recurculate inside air as the seasons
change. I can heat my home with 'waste' energy at idle, or switch to cool
outside air for heavy jobs or late night gaming. In summer I exhaust outside,
pulling cooler air from underground. It is slowly spawning its own system,
only 1.5 years in.

