
Poor Man's Cluster - FilingTrader
http://www.regressionist.com/2020/07/05/poor-mans-cluster/
======
hknapp
Any ideas on how to get GPU compute on the cheap? I see some old teslas on
ebay but not sure if that would be more effective than newer geforce cards.

Not for commercial use

------
a012
Glad to see Moosefs mentioned, I've used it a fee years ago to reuse old PCs
for storage. If you aren't hosting data with strict SLA, a master and standby
nodes work great.

~~~
FilingTrader
MooseFS is great. I actually tried LizardFS first, and it was painfully slow.
Only later, after settling on MooseFS did I read that erasure coding can slow
things down, and erasure coding was the only thing I tried with LizardFS. So,
I still can't say for sure that MooseFS is faster. Anyway, it appears to be
better supported at the present time, although I did like that LizardFS can
have multiple hot masters without paying for the pro version.

~~~
gravypod
Have you looked at Ceph ad a storage backend? It allows per-file erasure
coding configuration and has a lot of cool features. It can also manage raw
block devices from what I'm told.

It's really awesome you've gotten a chance to build up this cluster. I wish
more startups realized that, despite the initial price tag, some of this is
way better than what you'll get from AWS. Your NVMe is likely 10x to 100x
faster than premium ssd storage at a fraction of the price. If you ever use
GPUs you'll be saving loads of money.

It's hard to convince people though that bare metal isn't scary.

~~~
FilingTrader
I haven't tried Ceph. It didn't seem as appealing, judging from comments I
found (mostly on Hacker News):
[http://www.regressionist.com/2020/06/20/reviews-of-
distribut...](http://www.regressionist.com/2020/06/20/reviews-of-distributed-
filesystems/) But it's also hard to tell exactly what people are reviewing.
Evidently CephFS is less mature than Ceph object store.

But, yeah. This just all seems so much cheaper than AWS. There are other
advantages, too. Like, a lot of stuff is better with the low latency that
comes with having the cluster on-location. Python plots are more pleasant to
interact with (ssh -X). NoMachine remote desktop is faster, etc.

Also, it's a bit like an all-you-can-eat buffet. You pay once, and then you
can just relax and enjoy. I have enough storage now that I don't need to
stress about costs before deciding to cache intermediate data. And I don't
need to worry about whether I really need those older results that I still
have saved on disk that I'm paying each minute to store on AWS. Those kind of
thoughts are just gone.

I do have one GPU now, a 1080 Ti ($450 on eBay). I am ready to house a total
of 6, but I'm waiting to buy more until I really need them.

