
Rent out GPUs to AI researchers and make ~2x more than mining cryptocurrencies - mparramon
https://www.reddit.com/r/gpumining/comments/86ofw2/_/
======
boulos
Disclosure: I work on Google Cloud (and launched Preemptible VMs).

Cool! Like you, I wish people would make productive use of spare cycles.

Can I suggest you add a note comparing this to BOINC/gridcoin as well? I think
your marketplace is a better idea, but because of the security implications
dgacmu pointed out downthread, it shouldn’t be treated lightly.

Also, I really like your white paper
([https://vectordash.com/primer](https://vectordash.com/primer)) as it’s
nicely formatted and has pretty icons. I think you should update the
comparisons to the cloud providers though. Comparing a Pascal-series (1080 Ti)
part to a Kepler part (K80) isn’t particularly fair, as NVIDIA charges about
the same for that part as the later P100s and V100s. That is, people should be
comparing what the best price/performance ratio is, and all three providers
have better options. There’s a secondary question about whether the right
comparison is to AWS Spot / Preemptible VMs / Low Priority VMs, but I’m not
sure from a quick skim what the likely preemption rate will be with your
setup.

Finally, let me echo dgacmu in saying it’s too bad that NVIDIA forces all
providers to only provide the expensive parts. The best price/performance is
certainly the consumer parts (that 1080 Ti compares favorably to a P100 at a
fraction of the cost), but both the EULA and otherwise prevent it. You’re not
running a direct service / providing things in a data center which is a cool
workaround! But you should definitely be more clear about the risks of letting
untrusted third-party code to execute against your GPU.

Again, great project!

~~~
navinsylvester
We at Gridmarkets have cracked this one(spare cycles) and are now further
enhancing the model.

~~~
largote
That's a nice word-salad you got there.

~~~
mykull
Talking to VCs will do this to a person. I dont know why they can't just speak
normal English, but spewing marketese seems to be a rite of passage.

------
dpwm
There's a lot of comments on that reddit thread about how awesome this would
be as a service.

But there's a big problem of trust with this for ML.

How do I know you actually ran what I paid you for and not just generated
random data that looks right in the shape I wanted it?

You could farm it out to two people and if the results disagree, then payment
is decided by a third. But then you've just doubled the (paid) workload, and
you've not really solved collusion by a significant portion of the workers.

~~~
Samin100
It's pretty much letting people with GPUs become a sort of 'mini-cloud
provider'. There's no job queue or fancy distributing computing setup. We just
let you SSH into a container on someone's computer, and pay for the time used.

I was doing a lot of fun deep learning projects on the side & would often
Venmo my friend who was mining Ethereum to use his GPU to train my models. He
made more and I paid less than AWS spot instances or Paperspace.

This is just a fun side project hoping to let people who want to train their
deep learning models do it cheaply (on other people's computers!)

~~~
sangnoir
> It's pretty much letting people with GPUs become a sort of 'mini-cloud
> provider'. There's no job queue or fancy distributing computing setup. We
> just let you SSH into a container on someone's computer, and pay for the
> time used.

I had the same idea a few days ago - but in my head, the process would be
wrapped up as a "cryptocurrency" where the AI researchers pay real money and
the "proof of work" is useful/"real" work. I ran into 2 issues regarding
trust: the first is that how do you verify that the hardware owner is running
the real job an not NOOP'ing and sending false results? The second issue is
how do you protect the hardware from malicious jobs? GPUs have DMA access -
how do you stop task submitters from rooting your box and recruiting it into
an AI botnet (for free)?I ended up dismissing the idea, but if you could work
out these 2 issues, there's money to be made...

~~~
jonhohle
> the first is that how do you verify that the hardware owner is running the
> real job an not NOOP'ing and sending false results?

Consensus. Have _n_ nodes perform the same work (if it’s deterministic), and
only accept (and pay) if all the nodes match - or at least the nodes that were
part of the majority

I don’t think this would be considerably different from SETI or folding@home,
which have been going on for around twenty years.

For my senior project in college one of our ideas was a distributed render
farm that operated like what we’re talking about. There were some additional
issues there (transfering assets, preventing node owners from extracting the
output [say a studio was “letting” fans donate compute time to render a
feature film], etc).

~~~
dmurray
It would work well for problems that are computationally hard to solve, but
easy to verify solutions for. Unfortunately, such problems are ubiquitous in
cryptocurrency, but rare in machine learning.

~~~
chillee
Well, not really. Training a model is one such task. It's hard to train a
network but easy to verify that it has good performance (training vs
inference).

The real problem here, I believe, and I've seen this idea pop up several times
on hackernews, is that almost no machine learning tasks are progress free.

If the cryptocurrency is just paid out to the person who solves the task first
in a non-progress free problem, then the person with the fastest GPU would
mine all the coins and nobody else can participate. One of the key ideas
behind proof of work is that if two people have the same compute, and person A
has a headstart, if person A has not succeeded by the time person B starts,
they'll have the same probability of mining a block.

People seem to be just jumping on the crypto bandwagon and trying to come up
with "useful" proof of work, but it's a pretty difficult task.

------
Samin100
Creator of Vectordash here! If you have any questions about the platform, feel
free to ask away!

P.S. I'm @samin100 on Twitter if you enjoy tweets about GPUs!

~~~
Flammy
I saw people on the Reddit thread saying the Ubuntu requirement was a
showstopper for a lot of them.

You might look into the Win 10 linux support, Ubuntu is one o the supported
distros. Not sure if it would have full access to resources, have just used it
a bit at work and setup was super simple.

~~~
Samin100
I remember a few weeks ago a when some Microsoft software engineers came to
the career fair at Univ. of Maryland. One of them was working on Bash on
Ubuntu on Windows, so I ended up bugging him for about an hour on when they'd
support GPU passthrough for Ubuntu on Windows.

He said it was on their list of most requested features, but refused give me a
data on when it would be ready :/

------
rb2k_
I wonder when people will start just renting a 99 Euro server with a GTX 1080
from [https://www.hetzner.de/dedicated-rootserver/ex51-ssd-
gpu](https://www.hetzner.de/dedicated-rootserver/ex51-ssd-gpu) and sell it for
$367.2 / month on Vectordash

------
eridius
This uses LXC Containers. I've always heard that containers are not a
replacement for a good security model. What's the risk that someone could use
Vectordash to attack the host computer?

~~~
Samin100
LXC unprivileged containers are actually pretty secure by design. The
Canonical LXC page does a pretty great job of explaining why:
[https://linuxcontainers.org/lxc/security/](https://linuxcontainers.org/lxc/security/)

~~~
dgacmu
Please (OP, and others allowing access to your machines), be cautious on this
front. While LXC unprivileged containers are a start at isolation, LXC
containers + GPU passthrough has a much, much larger attack surface (the
nvidia binary blob drivers are complex and out of your control). The most
common ways of giving GPU access to a container involve installing the CUDA
drivers in the host and simply allowing the container to access them.

Depending on your threat model, this may be borderline OK, or may be insanely
high risk. We've seen driver exploits in the recent past:

[http://nvidia.custhelp.com/app/answers/detail/a_id/3140/~/se...](http://nvidia.custhelp.com/app/answers/detail/a_id/3140/~/security-
bulletin%3A-nvidia-unix-graphics-driver-exploit-advisory).

that can vector through the GPU to gain access to arbitrary host memory (aka,
you're in deep trouble). The only "right" way to give access to your GPUs
requires IOMMU support and running in a true guest VM, which Nvidia's consumer
cards and drivers are explicitly prevented from doing (because they'd like you
to buy the Tesla versions, thanks).

This may be a totally acceptable risk on an isolated mining rig, but people
should be aware of the heightened risks they're exposing themselves to.
Breaking your machine is _well_ within the capabilities of state-sponsored
actors on this one, and from time to time (just after vulnerabilities get
announced), could be decently within advanced-script-kiddie zone.

~~~
Samin100
This is a very, very valid point. I'm going to mention this on the website &
even advise people to run the client on an isolated machine (instead of their
daily driver gaming rig).

Thank you for pointing this out.

Also, one of my friends working on this project is a sophomore in the CS dept
at CMU, and given your interest in distributed systems & DL, would it be
possible to meet up for a couple of minutes and discuss security a bit more in
depth? (if yes, I'll shoot you an email)

~~~
dgacmu
Happy to. Drop me a note.

~~~
rememberlenny
^^^ This is why HN is amazing.

------
tudorconstantin
Great idea and I am glad somebody actually tries to make an implementation.

But unfortunately this won't replace mining: the large scale mining farms have
high end GPUs in their rigs, but the rest of the HW is very low end, because
that works perfectly for mining and they want their ROI as low as possible.

I have a 6 GTX 1070 GPU rig, which would be decent GPUs for AI/ML, but the
rest of the rig has a Celeron CPU, 4gb RAM,and a 5400 RPM HDD. Oh, and to be
able to see all the GPUs, I had to downgrade all the PCIEx slots to 1x.

I am curious what kind of ML tasks would be able to fully utilize these GPUs,
on such a low end hardware.

~~~
QasimK
The ones that train on a GPU? I'm not sure the rest of the system will have
that much of an impact.

------
ISL
We're coming into the warm seasons in the northern hemisphere, but if this
effort survives into autumn, it'll be quite tempting for home-heating.

Stoked to see distributed-compute as a paying service making another try. One
of these days, it is going to fly.

~~~
evanb
People are trying this for mining: [https://techcrunch.com/2018/03/08/qarnot-
unveils-a-cryptocur...](https://techcrunch.com/2018/03/08/qarnot-unveils-a-
cryptocurrency-heater-for-your-home/)

------
PascLeRasc
"Airbnb but for GPUs" is a buzzphrase I never could have imagined being real.

------
doall
Considering for hosting. One concern is that I have to expose my IP address to
unknown users. I don't know if it's possible, but it will be great that if I
can hide my location and purely lend my GPU with less security concerns.

~~~
Samin100
You don't need to expose your IP address at all! The AI/ML researchers can
only connect to one of our proxy servers instead of directly connecting to
your machine. This means hosts do not have to mess with port forwarding or
networking (even if your computer is behind a university firewall, you can
still be a host!)

~~~
doall
Wow that's great! Another question. It is always possible that I have some
downtime (unstable network connection, accidentally shutdown my pc, power
outage, etc) and it may become unfair that users having significantly less
downtime (better network, UPS, etc) having same earnings with users with much
more downtime. Is there any plan to deal with such kind of concern?

By the way, Vectordash is the most interesting idea & service I have seen in a
while :)

------
corysama
[https://medium.com/@rendertoken](https://medium.com/@rendertoken) aims to do
the same thing, but for 3D render farm customers.

------
senatorobama
How do I know if I will make a profit given electricity costs?

~~~
Samin100
This website has a pretty good calculator:

[https://whattomine.com](https://whattomine.com)

Just replace your miner earnings with the earnings listed on
[http://vectordash.com/hosting](http://vectordash.com/hosting)

------
Danieru
This sounds pretty awesome. I wish I had perminent internet to the house so I
could offer my rigs. I hope this service takes off, it should be a much better
use of electricity and gpus than mining. Please do keep developing this, with
the down turn in mining there should be a large pool of potential gpus. If
this service can connect those gpus to researchers then that should be a
lucrative business Ling term.

------
ezoe
As others have already pointed out, there are trust issues.

The seller can fake the computation to return bogus result is one thing.

But even if there is no malicious intent, the resulting computation is still
ended up bogus result, the malfunction of the hardware.

Commodity hardware isn't that reliable and there are so many commodity GPUs in
the wild that looks like working but return incorrect result.

~~~
alkonaut
all of these issues can be worked around by random validation, sumbitting
all/some blocks to multiple people (redundant calculation). This is no
different from existing efforts such as SETI@Homr and Folding@Home. The
difference being the compensation which increases the incentive for fraud.

The key to the trust issue is not trusting the remote computer, because you
can’t. You have to validate somehow.

~~~
ezoe
That solution requires that all computation is deterministic and the result of
computation can be easily comparable. I think not all the computation is that
simple.

~~~
alkonaut
It only assumes that including incorrect results isn’t catastrophic for the
end result, and that using mostly correct results will give a mostly correct
outcome. If that isn’t the case I’d compute every block 2..n times instead.

------
inetknght
GPUs are already too expensive for gamers and now you want to do this?!

~~~
proofofmoon
AI research demand is a lot easier for AMD/Nvidia to project, which allows
them to ramp up production and keep prices stable.

Side note: AI research is more beneficial to society than running a few
instances of Crysis 9 at 4K 120HZ.

~~~
lovelearning
... _potentially_ more beneficial. Also potentially very harmful depending on
what it gets used for.

~~~
blattimwind
Let's just say that AI research _can be_ useful, and thus this use of GPU time
and electricity _can be_ useful; in contrast to the alternative alternative-
GPU-use, which is not useful in the first place.

