
My low cost provider of GPUs has run out of capacity. Good alternatives? - etaioinshrdlu
https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu
======
etaioinshrdlu
Hetzner has provided a quality service for a long time but now they have no
more GPU servers available. That makes it hard to scale.

Any other options out there without going bankrupt on AWS?

~~~
reallydontask
Have you contacted them to see when they might get them back in stock?

It might be a temporary blip in supplies, etc ...

~~~
hanfer
Yeah write them a ticket. They are really supportive, pretty sure they will
come up with something for you. (as did they for us, when we were in a similar
bind)

~~~
etaioinshrdlu
Already did. They just said, currently out of stock, working on making it
available as soon as possible.

No ETA provided.

------
blodovnik
Aws and Google spot instances are easily comparable price wise to hetzner but
it really depends on what you are doing, which you haven't said.

For example a g3 spot instance can be as low as 19 cents per hour $138/month.

Google GPU instances I've run for 14 cents an hour.

I think these aren't the lowest prices either.

~~~
etaioinshrdlu
The main use case is on demand neural net inference 24/7 availability.

Kind of like hosting a website that must be on 24/7, this neural net must be
on 24/7.

It runs on the order of 50x faster on an nvidia GPU compared to a CPU.

But it will sit idle most of the time.

Price per hour per GB of GPU ram is the most important metric when choosing a
server.

~~~
somuchtyler
I really like nocix servers, they are out of Kansas City. They got their
"i7-6700K 32GB + 2x 480GB SSD + GTX 1080" for $105/mo
[https://www.nocix.net/cart/?id=338](https://www.nocix.net/cart/?id=338)

~~~
etaioinshrdlu
This sounds great but they are also out of stock!

------
jonatron
Colocation with ebay'd servers and graphics cards. Some people don't know
there's somewhere in between building a datacentre and cloud servers.

~~~
NietTim
That's not a low cost solution though, and hard to just turn off the cost when
you're done with your day

~~~
parliament32
That's not really how Hetzner's dedis work: you typically pay a setup fee
(that's about 1/2 of a month of usage) and you have to cancel a full billing
cycle (usually a month) in advance. You're meant to run them for months/years
at a time, not just spin them up/down on-demand.

------
matthew-wegner
Elsewhere in the thread, it sounds like your needs a little more robust. But
for anyone reading who might be interested:

\- Tesla K40 12GB cards are ~$130 on US eBay, and have been for awhile

\- These cards are passive, though, intending to be used in wind tunnel
servers

\- However, you can put on 40mm screamer fans to cool them in a normal
desktop. Something like
[https://www.thingiverse.com/thing:3032044](https://www.thingiverse.com/thing:3032044)
or just wire-tie to the back grill

High pressure 40mm fans are LOUD, though. I run a desktop with two K40s in my
garage, and I can hear it a tiny bit standing outside in my driveway. (I
mostly use that box for neural net art experiments like various style transfer
workflows)

~~~
bitL
K80 are for ~$400, 24GB, 2xGPU, newer architecture, a bit better choice.

~~~
matthew-wegner
It shows up to the OS as two 12GB cards still. So it kind of depends on
whether the higher cost is worth the extra slot (in my case it isn't). Total
power is less than 2X on the card versus one K40, although I'm not sure
offhand if that indicates less performance or just power savings from shared
components.

As mentioned in a peer comment, K40 aren't the fastest cards either, but the
12GB is really nice for some use cases.

~~~
bitL
I have one K40 as well; you should be fine, even Haas F1 team built a CFD
supercomputer full of them fairly recently ;-) Kepler is pretty good in FP64.

------
chx
[https://www.serverhunter.com/?search=BF6-4AC-E30](https://www.serverhunter.com/?search=BF6-4AC-E30)
says the cheapest provider with a GTX GPU is Ikoula with an 1070 at 101.66 USD
and then Hetzner and then Hostkey. [https://www.hostkey.com/gpu-
servers](https://www.hostkey.com/gpu-servers)

It doesn't find [https://dedispec.com/gpu.php](https://dedispec.com/gpu.php)
these but I do :) I also read somewhere they even accept shipped in GPUs. I
know they did with disks.

~~~
ServerHunter
Sorry about that! While we had the Dedispec GPU servers listed, our spider
system didn't correctly detect the presence of a GPU, so it didn't show up as
such. It has now been resolved and they show up in your search results.

Thanks for using Server Hunter! :)

~~~
etaioinshrdlu
Also missing some GPU servers from [https://www.hostkey.com/gpu-
servers](https://www.hostkey.com/gpu-servers) ... although you got some of
them.

------
andrewl-hn
Folks at CleverCloud launched CleverGrid today
[https://www.clevergrid.io/](https://www.clevergrid.io/)

I know a few of their engineers personally, really experienced and trustworthy
engineers. They are based in France.

------
mciancia
I wonder how are they dealing with nvidia EULAs about using geforce series
cards in datacenters.

Maybe that is the issue?

~~~
Donald
They use older cards without the restrictions.

~~~
ntenenz
The EULA is attached up the drivers. Newer versions of CUDA often require
newer drivers, so you bump into the issue regardless of when you buy the GPU.

------
lukeqsee
Linode just launched their instances in Newark on public beta.

[https://www.linode.com/gpus](https://www.linode.com/gpus)

I'm not affiliated, just love their service!

~~~
tmikaeld
~ 8.5x higher price than Hetzner.

~~~
lukeqsee
Yes, they're at a much higher price point, but it's also way cheaper than AWS.

I certainly understand the draw of Hetzner's price point!

~~~
abrichr
How is Herzner able to be so much cheaper than the competition?

~~~
tmikaeld
They have addons for most support issues, including priority incidents -
throughout the years we've been with them, it can take ~2-5 hours before
someone even reply to complete server downtime.

We mostly host proxmox nodes and use HA, so it's not much of an issue when
it's used like this.

OVH is very similar, although, Hetzner is often quicker.

We only use server-grade (Like Xeon or EPYC), not the desktop ones (No ECC
memory).

If one need many IP's, OVH always wins in price due to it being a one-time
cost, while Hetzner charges monthly.

------
baybal2
I think Ebay and TaoBao are now filled to the brim with discarded mining GPUs

~~~
blodovnik
Can you link some examples?

~~~
baybal2
My first search at taobao gives gtx1080 for 300 usd at first position

------
samscully
Vast AI ([https://vast.ai](https://vast.ai)) are good, 1x GTX 1080 comes to
about $170/month. You are charged by the hour unlike Hetzner.

There are obviously downsides to renting time from random individuals though,
so it's not suitable for a server-like workload. Good for development
notebooks or training.

~~~
gwern
I recently used Vast.ai for ~3 weeks to run an anime BigGAN (
[https://www.gwern.net/Faces#biggan](https://www.gwern.net/Faces#biggan) ) and
the stability/uptime was pretty much 100%: it never went down or caused
problems. (I had problems, but they were all due to the BigGAN.) As long as a
little downtime isn't too big a deal or you can script changing instances
(they have a CLI tool), you probably could run as a server.

------
ydau
Lambda Labs: [https://lambdalabs.com](https://lambdalabs.com) has 1080 Ti GPUs
for rent. We also sell GPU workstations and servers for AI.

~~~
minimaxir
I wouldn't call $0.80/GPU/hr "low cost" compared to other options.

------
julienfr112
Wasn't it forbidden by the driver licence to use 1080 on cloud ? Nvidia tries
to bully you to buy ultra expensive Tesla card for that. Maybe the germans
don't care ?

~~~
brohee
Not enforceable in Europe IIRC. So they rightfully don't care.

~~~
etaioinshrdlu
I believe you but can we get some hard evidence? Everyone in this thread
agrees but it would be nice to know for sure.

------
tech_man7
Golem has GPUs for very cheap on its network so ask here
[https://www.reddit.com/r/GolemProject/](https://www.reddit.com/r/GolemProject/)
or
[https://twitter.com/golemproject?lang=en](https://twitter.com/golemproject?lang=en).
Golem website is www.golem.network

~~~
etaioinshrdlu
I don't like that website, it looks almost like a blockchain ICO site. Low on
details, high on hype.

I don't see prices listed. Something seems wrong.

This company does something pretty similar minus the blockchain and is much
more straightforward: [https://www.zerosix.ai/](https://www.zerosix.ai/) ,
same with [https://vast.ai/console/create/](https://vast.ai/console/create/)

~~~
tech_man7
It's not one of those ICO scams. Maybe you'd prefer their GitHub page in which
the team has written over 100,000 lines of code.
[https://github.com/golemfactory/golem](https://github.com/golemfactory/golem).

It's a marketplace so you name your own price and at the moment, it's used for
CGI rendering, but next week there are more use cases being added,
specifically WASM.

------
cryptofits
Hetzner support is pretty bad

took them like 6 days to answer my ticket

~~~
mythz
Not the experience I've had with Hetzner (been with them for 5+ years),
basically all responses have been within a day, 2 at most. Although their
servers have been pretty reliable so I haven't had to contact them much.

