
Linode GPU Instances - gr2020
https://blog.linode.com/2019/06/19/introducing-linode-gpu-instances/
======
Who_me
Hey peeps full disclosure I work as one of Linode's RnD engineers. I want to
try to get to as many of these as I can.

One of the biggest questions is why the Quadro RTX 6000? Few things:

1\. Cost it has the same performance as the 8000. The difference is 8 more GB
of RAM that comes at a steep premium. Cost is important to us as it allows us
to be at a more affordable price point.

2\. We have all heard or used the Tesla V100, and it's a great card. The
biggest issue is that it's expensive. So one of the things that caught our eye
is the RTX 6000 has a fast Single-Precision Performance, Tensor Performance,
and INT8 performance. Plus the Quadro RTX supports INT4.
[https://www.nvidia.com/content/dam/en-zz/Solutions/design-
vi...](https://www.nvidia.com/content/dam/en-zz/Solutions/design-
visualization/quadro-product-literature/quadro-rtx-6000-us-
nvidia-704093-r4-web.pdf)
[https://images.nvidia.com/content/technologies/volta/pdf/tes...](https://images.nvidia.com/content/technologies/volta/pdf/tesla-
volta-v100-datasheet-letter-fnl-web.pdf) Yes, these are manufactures numbers,
but it caused us pause. As always, your mileage may vary.

3\. RT cores. This is the first time (TMK) that a cloud provider is bringing
RT cores into the market. There are many use cases for RT that have yet to be
explored. What will we come up with as a community?!

Now with all that being said, there is a downside, FP64 aka double precision.
The Tesla V100 does this very well, whereas the Quadro RTX 6000 does poorly in
comparison. We think although those workloads are important, the goal was to
find a solution that fits a vast majority of the use cases.

So is the marketing true to get the most out of MI/AI/Etc? Do you need a Tesla
to get the best performance? Or is the Tesla starting to show its age? Give
the cards a try I think you'll find these new RTX Quadros with Turning
architecture are not the same as the Quadros of the past.

~~~
jamesblonde
If you really want low cost to compute for Deep Learning and you needs lots of
compute and don't want to pay for V100s, then the AMD Vega R7 is the card for
you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!),
works with Tensorflow (pip install tensorflow-rocm), and about 60% of the
performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so
still quite good). Put lots of them in servers with PCI 4.0, and you can do
great things. Here's a recent talk on it:

[https://www.youtube.com/watch?v=neb1C6JlEXc](https://www.youtube.com/watch?v=neb1C6JlEXc)

~~~
danieldk
_If you really want low cost to compute for Deep Learning and you needs lots
of compute and don 't want to pay for V100s, then the AMD Vega R7 is the card
for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!),
works with Tensorflow (pip install tensorflow-rocm), and about 60% of the
performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so
still quite good)._

Two of my colleagues use high-end AMD GPUs to train RNNs and transformers with
tensorflow-rocm. There are still some nasty bugs (e.g. [1]), so it is
currently not for everyone. _However_ , given how far they have come compared
to 1-2 years ago, it is very likely that in a year or so, they are a real
competitor to NVIDIA for compute. That competition was long needed.

[1] [https://github.com/ROCmSoftwarePlatform/tensorflow-
upstream/...](https://github.com/ROCmSoftwarePlatform/tensorflow-
upstream/issues/325)

~~~
jamesblonde
Agreed, it is not quite prime-time yet. They are trying to upstream all the
ROCm stuff in TensorFlow, and when it gets into mainline and stabilizes, i
agree that it has great potential for take-off - particularly from price-
sensitive researchers and large companies who need huge GPU farms.

~~~
ksec
Two Questions.

I wonder if Google is in any way helping AMD in the TensorFlow and ROCm?

What happen when Intel join the GPU race in 2020. Making their own _ROCm_
again?

------
picozeta
I would go with Hetzner: [https://www.hetzner.com/dedicated-
rootserver/ex51-ssd-gpu](https://www.hetzner.com/dedicated-
rootserver/ex51-ssd-gpu)

GTX1080 for 100$ a month. Grantend, it is older, but it still works for DL.
Let's say you do 10 experiments a month for ~20 hours. Thats 0.5$/hour and I
don't think it is 3 times faster.

If you then want to do even more learning the price goes even down.

//DISCLAIMER: I do not work for them, but used it for DL in the past and it
was for sure cheaper than GCP or AWS. If you have to do lots of experiments
(>year) go with your own hardware, but do not underestimate the convenience of
>100MByte/s if you download many big training sets.

~~~
svd4anything
How does data in/out work in practice with them? I see this 4 Tbit bandwidth
but do you happen to know what that translates to and what happens if you
exceed that?

Also check availability shows a 5 day wait current: “EX51-SSD-GPU for
Falkenstein (FSN1): Due to very high demand for these server models, its
current setup time is approximately up to 5 workdays.*” Or maybe there are
other regions/dcs.

~~~
indalo
I have like 18 of their auction servers that are unmetered at 1gbps and really
make that bandwidth sweat. I've never had issues honestly, and they've never
tried to dreamhost me. I love it.

------
m0zg
Still way too much money when a 2x 2080Ti comparably specced machine under my
desk costs less than 2.5 months of their billing rate, and 4x 1080Ti servers
in my garage cost about 1 month of their 4-GPU machine _and_ have more SSD
storage. This pricing is totally insane, especially if not billed per-minute
(which in Linode's case it is not) and if there are no cheaper
preemptible/spot instances.

~~~
svd4anything
I’m starting to think one can adopt the simple rule of switch to a DIY build
whenever there enough work to keep a GPU busy for 2 months, otherwise if the
workload is intermittent then better strategy is leasing, especially
considering the purchase cost/performance is constantly dropping.

~~~
dillonmckay
It would seem there could be a market for multi-month rentals of a ML/gaming
rig, with return shipping/packaging?

------
ilaksh
Looks amazing. Linode has worked really well for me over the years.

One thing I noticed when recently trying to get a GPU cloud instance, the high
core counts are usually locked until you put in a quota increase. Then
sometimes they want to call you.

So I wonder if Linode will have to do that or if they can figure out another
way to handle it that would be more convenient.

I also wonder if Linode could somehow get Windows on these? I know they
generally don't do anything other than Linux though. My graphics project where
I am trying to run several hundred ZX Spectrum libretro cores on one screen
only runs on Windows.

------
keytarsolo
That pricing isn't too bad. They come with decent SSD storage too, which is
key for the large datasets that make a GPU instance worthwhile.

Linode skews more towards smaller scale customers with many of their offerings
so I think the GPUs here make sense. The real test will be how often they
upgrade them and what they upgrade them too.

------
hmart
I love Linode support. There are cheaper places but I have my Key VPSs there.

------
dkobran
Interesting to see another cloud provider go with Quadro chips. NVIDIA
repackages the same silicon under several different brands (GeForce, Quadro,
GRID, Tesla) and we ([https://paperspace.com](https://paperspace.com)) have
found Quadro to offer the best price/performance value. Despite minor
performance characteristics, such as FP16 support in the Tesla family, Quadros
can run all of the same workloads eg graphics, HPC, Deep Learning etc. If
you’re interested in a similar instance for less $/hr, check out the
Paperspace P6000.

------
minimaxir
Huh. Given that cheap cloud GPUs are nowadays sought for training AI,
launching with a Workstation-oriented GPU is an odd product decision.

~~~
ksec
Are there any difference? Seems to support CUDA as well, I don't see anything
wrong with it.

Also seems to be a lot cheaper than AWS counterpart.

~~~
minimaxir
It seems like a Quadro RTX 6000 is comparable performance-wise to a P100
([https://askgeek.io/en/gpus/vs/NVIDIA_Quadro-RTX-6000-vs-
NVID...](https://askgeek.io/en/gpus/vs/NVIDIA_Quadro-RTX-6000-vs-
NVIDIA_Tesla-P100-SXM2)), although there are no tests for the ML angle.

A P100 on GCP is $1.46/hr alone, so maybe Linode is a good deal if the
performance is indeed comparable.

~~~
elabajaba
RTX 6000 is significantly faster than a P100 outside of FP64, and is the
fastest or 2nd fastest GPU outside of FP64 work [1] (the GV100 is sometimes
faster, sometimes slower than the RTX 6000 but costs more). For FP64, GV100
based GPUS are quite a bit faster than P100s.

Also, you should really ignore pretty much all of the comparison sites that
show up when you search for computer component comparisons as they're nearly
all awful. The one you posted doesn't show a single benchmark comparison
between them, and compares numbers like clock speed which isn't comparable
between architectures, or memory clock speed instead of memory bandwidth
leading to the laughable conclusion that the RTX 6000 has "9.9x more memory
clock speed: 14000 MHz vs 1408 MHz" vs the P100 when the P100 uses HBM2 and
has 732.2 GB/s vs 672.0 GB/s of actual memory bandwidth.

[1]
[https://www.reddit.com/r/hardware/comments/9xx6cz/nvidia_qua...](https://www.reddit.com/r/hardware/comments/9xx6cz/nvidia_quadro_rtx_50006000_teardown_review/)

~~~
svd4anything
[https://www.techpowerup.com/gpu-specs/quadro-
rtx-6000.c3307](https://www.techpowerup.com/gpu-specs/quadro-rtx-6000.c3307)

Has this section:

Theoretical Performance Pixel Rate 169.9 GPixel/s Texture Rate 509.8 GTexel/s
FP16 (half) performance 32.62 TFLOPS (2:1) FP32 (float) performance 16.31
TFLOPS FP64 (double) performance 509.8 GFLOPS (1:32)

Perhaps for at least rough performance comparisons it is a good start.

My impression the current favorite card by DIY types is an RTX 2080 Ti.

[https://www.techpowerup.com/gpu-specs/geforce-
rtx-2080-ti.c3...](https://www.techpowerup.com/gpu-specs/geforce-
rtx-2080-ti.c3305)

However I think nvidia will take legal action against any service provider
trying to use those in a cloud or server offering.

------
thenightcrawler
Isn't AWS cheaper?

edit: could be wrong thought I read of AWS being .65 dollars an hour for deep
learning GPU use. edit2: Did a quick look, the .65 dollars doesn't include the
actual instance, so its around 1.8 an hour on the low end, I think this
cheaper.

~~~
perennate
p2.xlarge comes with an NVIDIA Tesla K80 GPU for $0.90/hr, but this is now an
"old" GPU and the RTX Quadro 6000 should have much higher performance (but I
was unable to find any machine learning benchmarks).

p3.2xlarge has NVIDIA Tesla V100 GPU which is NVIDIA's most recent deep
learning GPU, but it's $3.06/hr.

That said, AWS is among the most expensive providers if you just need a deep
learning GPU (but obviously AWS offers a lot of other useful things). For
example, OVH Public Cloud has Tesla V100 for $2.66/hr. And comparable NVIDIA
GPUs that are not "datacenter-grade" should be even cheaper; AWS, GCP, Azure,
etc. are unable to offer them because of contracts when they buy e.g. the
Tesla V100.

~~~
zachruss92
The K80s are super outdated now. Google used to offer them for free for 12hrs
a day on their Colab platform, but they upgraded them to using the Tesla T4s.
Note you can get a K80 on GCP (unreserved) for $.45/hr.

This is a good deal.

------
coherentpony
Does anybody know if there are any cloud instances with AMD GPUs?

~~~
azinman2
What would you want it for over nvidia in the cloud?

~~~
noir_lord
Testing if nothing else, I'd you are shipping to be people running AMD GPUs
then that would be useful without having to buy a card and another machine for
it to go in

------
zonidjan
Oh, hey! It only adds 1/5th of the GPU's purchase price.

------
MuffinFlavored
Can these be used for crypto mining at any level of efficiency? I was able to
mine GRLC back in the day on AWS spot instances at a VERY mild degree of
profitability.

~~~
walrus01
not really, most cryptocurrency is at the stage where the only thing effective
is a combination of custom ASICs and nearly free electricity. About twelve
months ago I looked into mining ethereum with state of the art GPUs and it
would not have had a reasonable ROI unless I was literally paying $0.00 per
kWh. And that was before its value per coin dropped a lot.

~~~
kohtatsu
Let's replace baseboard heaters with cryptominers.

------
bufferoverflow
Can you rent them by minute?

~~~
perennate
Linode bills per-hour: [https://www.linode.com/docs/platform/billing-and-
support/bil...](https://www.linode.com/docs/platform/billing-and-
support/billing-and-payments/)

------
ThomWilhelm3
Will this profitably mine bitcoins? :p

~~~
beatgammit
No.

