
Ask HN: How to build a distributed dataset system - albirt
In my computer vision startup, we just got to the point where running datasets on a single development machine takes hours. This hampers progress, since one needs to know performance metrics before making changes to the product.<p>We&#x27;re looking into how to make this process faster by distributing the workload across many machines.<p>My question is, given a job (e.g. run a program on a dataset and collect some results), what are some practical frameworks that are out there that one can configure and use with relative ease. Note that none of us is a cloud expert but we are very comfortable with linux.<p>thanks in advance.
======
PaulHoule
Send me an email, it is on my HN profile page.

