
Launch HN: FloydHub (YC W17) – Heroku for Deep Learning - saip
Hi HN! I’m Sai, one of the cofounders at FloydHub (<a href="https:&#x2F;&#x2F;www.floydhub.com" rel="nofollow">https:&#x2F;&#x2F;www.floydhub.com</a>). We&#x27;re building FloydHub to be a “Heroku for deep learning”. We are in the current batch (W17) at YC. But I still like to think of FloydHub as being an HN incubated startup.<p>10 months ago, I was working at Microsoft and doing a lot of deep learning (DL) there. While the DL community is terrific, I was often frustrated by how difficult it was to get started and build upon others’ work. For example, running any popular Github project often started with an exercise in dependency hell. As I untangled these for myself, I wrote up some notes on setting up popular DL frameworks, which unexpectedly started trending on HN after someone posted it there (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11697571" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11697571</a>). That&#x27;s when I realized that engineering was a huge bottleneck in deep learning and a problem worth solving after all.<p>I’ve since quit my job and have been working fulltime for the last 9 months on building FloydHub to make deep learning easier. Our goal is to let the data scientists focus on the science, while we handle the engineering grunt work (provisioning and scaling infra, running reproducible experiments, enabling sharing and collaboration, supporting DL frameworks with zero setup, shipping trained models to production easily, etc.) Lots of interesting challenges - happy to talk about them!<p>We have a lot of work ahead, but we’re excited to share with you what we have so far! Looking forward to your feedback.
======
netvarun
I was one of their earliest beta customers. Despite the initial quirks, the
experience has been nothing but magical. It allowed me to go from code to
training/model generation in one command, without any of the devops nonsense
to do deal with. _While digging into deep learning, I honestly felt that the
devops stuff was actually more complicated than the math /backprop/neural nets
stuff._

It felt like a Heroku moment for me. They have the potential to do to
Tensorflow what Heroku did to Rails. Super simple deploy!

Obviously their vision is much broader (with an entire eco-system/'hub',
reproducibility, etc.), but to me atleast the first part is super useful and
exciting!

One advantage they have is that GPUs are INSANELY expensive on the cloud -
they can actually make it cheaper for everyone with clever binpacking and
proper termination.

My advice is that in the initial stage, they should partner with all the Moocs
to ensure that all deep learning students are using Floyd. It's cheaper,
faster and the students can focus on the science. And they provide, 100 free
hours!

Disclosure: I've known the guys for quite a while.

~~~
francogt
I think partnering with MOOCs is a great idea. I've been taking the Deep
Learning Foundations Nanodegree from Udacity and thought it would be awesome
if we got some sort of discount to try this out. Course developers have been
trying to get us AWS credits for a while now. What would be the best way to
contact the guys?

~~~
saip
Hey! Glad you brought this up. We've gotten quite a few requests from students
enrolled in Udacity courses and have been helping get them set up & run their
class projects on Floyd.

Here's our instructions for the Self Driving Car Engineer nanodegree program:
[https://github.com/floydhub/CarND-Term1-Starter-
Kit](https://github.com/floydhub/CarND-Term1-Starter-Kit). Happy to do the
same for your class as well! How can I reach you? Feel free to to mail us
directly: founders@floydhub.com.

We've also reached out to folks at Udacity to see if we can offer any official
support for the courses.

~~~
francogt
Thanks for the reply! I just sent you guys an email. Will take a look at what
you did for the Self Driving Car Nanodegree.

------
sly010
Sidenote: We always say the hardest things in software is cache invalidation
and naming things, but I also found the problem to be dependency tracking and
reproducibility. It's just so hard to get started with ANY non trivial
software project and your os/tools/libs expire way before you finish the
projects. Software should be cheap and repeatable, but for some reason it
takes active maintenance and is therefore very expensive. If a superhuman
intelligence looked at us from afar, we would probably look like how ants look
to us: Millions of small workers with very inefficient probabilistic behavior.
Sure, we get the job done, but very slowly with a lot of waste. That said,
ants lived for ever, so maybe it is the right thing to do ;)

Either way, I often find myself choosing backward compatibility and stability
over innovation and polish and choose to learn vim and bash instead of
replacing the silver bullet every year.

Shameless plug, I am working on a platform similar to FloydHub, but for
frontend engineers [0]. The problem is a real one.

[0] [https://pipez.io](https://pipez.io)

~~~
narenst
I can definitely resonate with this. Deep learning is in such an early stage,
the frameworks and tooling are still maturing and evolving rapidly. This makes
it really hard to reproduce other's work. Maybe there will be one winner in
the frameworks war (Tensorflow?) and things will be better.

Pipez sounds really useful, good luck!

------
terrabytes
I’m a Data Scientist and I work in a large-ish corp. We have a dedicated
engineering team who take care of all of our infrastructure needs - we mostly
focus only on data science. I would expect most medium and large enterprises
to have the same setup. Why would they use FloydHub?

~~~
geoah
Smaller companies and startups, or medium sized companies that just start with
ml/dl will find it very useful.

Currently working in a small startup and our data scientists (just a team of
6) are frequently fighting over the gpus in the office for time and online
services atm are overpriced for our small budget.

~~~
narenst
Agreed, that is the target audience we are going after with FloydHub. If you
have some time to chat about your startup, please send me an email
naren[AT]floydhub.com. I would love to get your feedback and see what would
make Floyd useful for companies like yours.

------
whodunser
I have been using your docker container for 6 months or so now, thanks for
putting it together :)

The jupyter jobs look neat, but I assume they are charged continuous time?
Would be cool if somehow that only ended up charged for compute time, but I
understand that would be difficult.

Are these instances guaranteed to be in a given region, for if I wanted to
route more complex debug output / intermediate files to S3?

~~~
saip
Thanks - glad you found it useful! The attention and feedback that I got from
building dl-docker has been terrific. Definitely one of the reasons we started
working on this seriously :)

~~~
penagwin
Little off topic, does DL-docker work in Ubuntu on Windows for the GPU?

~~~
saip
You mean bash on Windows? The CPU version of dl-docker will work on Windows!
But unfortunately, the GPU version does not. You would need GPU passthrough,
which is currently not supported. See [https://github.com/NVIDIA/nvidia-
docker/issues/197](https://github.com/NVIDIA/nvidia-docker/issues/197)

------
moflome
Hi Naren, really glad to see this launch, your "Deploy your Trained Models"
that feature is awesome... what about external dataset access? I would like to
access the Google YouTube ML dataset [0], how could that be done within
FloydHub without uploading? Also, perhaps related, have you thought of teaming
up with Kaggle [1]?

[0] [https://research.googleblog.com/2016/09/announcing-
youtube-8...](https://research.googleblog.com/2016/09/announcing-
youtube-8m-large-and-diverse.html) [1]
[https://www.kaggle.com/c/youtube8m](https://www.kaggle.com/c/youtube8m)

~~~
narenst
Hi, Thanks a lot! We are always adding to our public datasets (with
appropriate licences). So we would be very happy to set it up for you on
Floyd. Agreed that uploading large datasets is not quick. Instead we recommend
you download them directly in to Floyd. See:
[http://docs.floydhub.com/home/managing_output/](http://docs.floydhub.com/home/managing_output/)
for an example.

Re: Kaggle, we haven’t had a chance yet but that sounds like a great idea.

~~~
supermdguy
I was able to download a kaggle dataset using kaggle-cli.

------
transcranial
Looks really amazing! Reproducibility of models and experiments is huge. It
should be almost a requirement if one is going to publish results claiming
SotA, etc. Seems like you could become the GitHub for deep learning in
addition to the Heroku for deep learning.

~~~
saip
Thanks, that's an awesome comment! Agreed. IMHO, end-to-end reproducibility is
important from multiple angles - provenance for research, enabling
collaboration, driving down costs by eliminating redundant runs of the same
jobs, etc.

We were initially calling ourselves Heroku + Github for DL, but realized that
was too confusing, haha.

------
rajsat
This looks neat! We have been doing a lot of deep learning for NLP at our
startup recently. Several “engineering bottlenecks” in the process (1)
managing multiple jobs is definitely worth solving. git for deep learning
would be neat (2) collaborating is a pain when the team is remote. I guess
this ties to (1) too.

And oh, about the time I forgot to turn off our GPU instance for a couple of
weeks… racked up a nice bill...

~~~
saip
We’ve definitely felt the bane of forgetting to turn off some really expensive
GPU instances. Efficient scheduling and spinning instances up/down as required
is one of the first things we built to cut down on the costs.

Git is an apt analogy. The search space of hyperparameters is usually fairly
large for most DL algorithms, so a good amount of experimentation is required
to tune them. Things can start to get haywire without end-to-end version
control of code, data, parameters, results, environments, etc. Definitely one
of the core problems we solve.

------
cr0sh
This sounds interesting and useful - I hope you guys make it!

A couple of years ago I worked for a local cloud server provider, as a backend
developer. Some of the work I did moved the company into deploying VPS
instances using OpenStack. Our backend code was mainly PHP-based; so we used
OpenCloud for the purpose - extending it where needed (when we started it
didn't support all we needed; I extended things in such a way so that when we
did need to upgrade OpenCloud, it would gracefully work without breaking
anything - it was a gamble that I didn't know if it would really work - just
had a hunch - 9 months in we upgraded, and it all worked perfectly).

Anyhow - at that time, seeing what we had available for servers and such (we
were competing somewhat with DO) - I suggested we add support for GPU
instances and maybe pivot toward an ML offering of some sort. Not gut our
bread-n-butter, but offer up some kind of ML package for those that needed or
wanted it.

I was shot down by management as it being too "pie in the sky" \- not even
demand or something like that. To be honest, I'm not even sure they understood
what I was trying to convey, so maybe part of the problem was mine as well.

The company was eventually sold and I moved on, but seeing now how these kinds
of services are in demand, I sometimes wonder on what "could've been". Ever
since taking my first MOOC in ML (Ng's ML Class in 2011) - I've tried to
interest employers in applying what (little) I know on the subject. I'm not an
expert, but I'd love to apply my learning (on top of the 25+ years of software
dev experience I already have). Today, I'm in the middle of the Udacity
Nanodegree MOOC - I doubt much of that will transfer for my current employer,
but maybe the general knowledge I'm getting of TensorFlow and Keras, among
other bits, might help in the future.

I think, though, that FloydHub might be fun to play around with for future
personal ML/DL projects as time goes on; I look forward to trying it out
someday soon!

~~~
saip
That’s awesome! ML to some extent, and DL to a large extent are still not
considered mainstream enough by many. The big players (Google, Facebook,
Baidu) seem to recognize this and invest appropriately, but the others not so
much yet. That’s hopefully a sign of a burgeoning market!

Good luck with your Udacity class! If you want to play around, take a look at
our guide for Neural Style Transfer:
[http://docs.floydhub.com/guides/style_transfer/](http://docs.floydhub.com/guides/style_transfer/)

------
narenst
Hi! I'm Naren, the other co-founder of FloydHub. I'll be happy to answer any
questions and really appreciate any feedback you can provide. Thanks!

~~~
insomniacity
Features page says "Develop interactively on the cloud using Jupyter Notebook.
Your code, results and outputs are always preserved."

But
[http://docs.floydhub.com/guides/jupyter/](http://docs.floydhub.com/guides/jupyter/)
says "IMPORTANT: Floyd does not save your Jupyter notebooks after you stop the
floyd job. So you need to download any relevant notebooks by selecting File >
Download As menu from the Jupyter notebook."

Which is it?

~~~
narenst
The warning in the documentation page is no longer valid. We _DO_ save the
notebook files and keep them after the session is terminated. They will be
part of the run output.

The docs have been updated to reflect this. Thanks for pointing this out.

------
mindcrime
So I guess we're competitors in a sense, but congrats nonetheless! We hope to
launch at least a private beta of NeuralObjects like "Real Soon Now"™.

It'll be interesting to see where we decide to go down different paths, or how
we take different approaches to things.

~~~
saip
Thanks for the comment! There's lots of challenges to be solved in this space,
and I'm sure there's room for all of us. Excited to see what you guys are up
to. I will look forward to your beta release "real soon" :)

~~~
mindcrime
What was it Steve Blank said? "Startups don't die from competition with other
startups, they die because they built a product nobody wants" (or something
like that).

As you say, there's plenty of room out there. And we're all competing with
Amazon, Google, Microsoft, etc. anyway. :-)

------
sarthakjain
Running DL and sharing results is a huge pain. I feel versioning is a big
challenge, also experiments with multiple architectures are a pain since large
parts of the same calculations are repeated. Do you also solve this problem?
(Feature request if not)

~~~
narenst
Yes, our Enterprise offering allows for building more complex workflows.
Everything (code, parameters, environment and data) is versioned for
reproducibility. One of the big value-adds is efficient caching of parts of
the pipeline to avoid repetitions and save resources. We have noticed that
this usually results in 10x increase in the number of experiments run by
teams.

------
vosper
You said you worked at MS, so you're presumably pretty familiar with Azure's
offering. I'm no expert (at all) but I played with it briefly and it was all
pretty slick and easy to get up and running. How does FloydHub compare to
that?

~~~
saip
I assume you're talking about AzureML Studio. It's a pretty neat UI-centric
tool for building machine learning workflows! It's great if you're starting
out with ML, but offers little in terms of customizability. For example, it
only supports R and Python, has no GPUs, no CLI, no container support for
managing reproducible environments, etc. I think these are kind of deal
breakers for doing deep learning :)

FWIW, I worked as a data scientist in Bing for 6 years and haven't seen/heard
any other data scientist use it internally. We ended up building our own GPU
clusters and going through the regular drill.

~~~
brudgers
Curious how Microsoft's approach compares to what you have heard about
Google's approach to ML as a service. My impression is that it looks
kinda/sorta like it's internal approach/infrastructure...if you squint.

Which also makes me curious about FloydHub's infrastructure. Any gory details?

~~~
narenst
Floyd’s Infrastructure runs entirely on Docker. That makes it backend agnostic
(our cloud offering currently runs on AWS). Floydhub uses nvidia-docker for
the deep learning jobs that require GPU. We also version the entire pipeline
(code, data, params and environment) for exact reproducibility.

GPUs instances are really expensive. One of the biggest challenges at the
moment is around reducing this cost. Eg. Spot Instances and Spot Blocks. Still
some challenges to be solved there.

We also want Floyd to be an end-to-end solution for building, training and
deploying deep learning models. In that vein, we are also investing in adding
support for Tensorflow serving but it has been a rough ride so far. Getting a
generic solution that can host any Tensorflow model has not been
straightforward.

~~~
pilooch
Try deepdetect, it s easier than serving, though probably still less flexible
when it comes to the TF backend.

------
Eridrus
It seems like everyone and their dog wants to solve this problem; why is it
going to be you?

~~~
nzjrs
YC Magic?

~~~
narenst
YC definitely helps in giving us a lot of credibility when talking to
customers especially enterprises. We have also been learning a ton about sales
during the last couple of months - for a couple of engineer / data scientist
it has been a humbling experience. Other than that there is not really any
magic here!

------
koolba
Skimming through the site this looks very polished for a startup offering.

~~~
saip
Thanks! We've been iterating on it for a while. My sister is a UX designer and
helped a ton. I still think there's room for improvement, but it's great to
hear a positive comment about it!

~~~
robotnoises
Who did the artwork for your landing page? Looks great.

~~~
saip
Nothing fancy - bought some cheap stock art and my sister, who's a UX
designer, modified and brushed them up :)

------
bfortuner
Nice! There is a real need for this in the market right now--especially among
students, who struggle to setup working environments. I really benefited from
your early Github project aimed to make setting up DL machines easier. Curious
how defensible your product will be in the event Heroku/AWS come in with a
competitor?

~~~
saip
Defending against big players with almost infinite resources is always an
interesting problem. Kind of have to hope they don’t come at you head on :)
That said, there might be other aspects that come into play. For example, the
market is fragmented with big providers each having their own frameworks and
infra that they prioritize (Google - Tensorflow/GCP, Facebook - Torch,
Microsoft - CNTK/Azure, Amazon - MXNet/AWS, Baidu - PaddlePaddle, etc.) Since
there is yet no clear winner, which will likely be the case for the near
future, vendor lock-in is not really desirable. We intend to be backend /
framework / language agnostic.

------
kevinyun
Thanks Sai, this looks great. I found myself dreading the clunky workflow with
AWS GPUs, even using CLI, so this is nice to see a Heroku-like product that's
a bit easier to work with.

Also, awesome 100 hours offering -- Looking forward to using FloydHub for deep
learning!

~~~
saip
Sounds good! Give it a spin and let us know what you think. If you want a
reference - here's a guide we wrote to run Neural Style Transfer
([http://docs.floydhub.com/guides/style_transfer/](http://docs.floydhub.com/guides/style_transfer/))!

------
ravibala1
You guys seem to make setting up Deep Learning infrastructure easy for data
scientists which is awesome! What about situations where the end user/business
isn't sure exactly how to use Deep Learning (which algorithm, how to partition
the data into training and result sets, stability of results etc.) for the
problem/data set at hand? Would it be possible to use FloydHub as a
marketplace of sorts where I could hire a deep learning enthusiast to
appropriately construct the experiment for me and then explain the results?

~~~
narenst
Yes, we are planning to build a FloydHub marketplace for talented data
scientists to find gigs. We believe Floydhub can showcase their work and
expertise easily (similar to StackOverflow) and help find a suitable partners
to work with. A wide range of industries like Medical, Oil and Finance have
been collecting huge amounts of data and now with Deep Learning, they can
effectively make use of that. So I definitely see a rise in demand for deep
learning practitioners and we want to support them on Floydhub.

------
svdad
Congrats on your launch! Seems like a really slick product.

Have you taken a look at
[https://cloud.google.com/ml/](https://cloud.google.com/ml/)? I assumed you
would have but didn't see anyone mention it in the comments.

Do you see your product as complementary to a service like that? e.g. could
you use the Google service as the execution engine for yours? or do you see
that as competition?

------
eudoxus
Is this currently running on a one of the available GPU cloud services (IE
AWS, Azure, Nimbix, etc...) or some self hosted hardware?

~~~
eudoxus
Nevermind, another comment mentions AWS. You seem to pretty much run at cost
then (a little less), if you GPU hours are $0.432 @ 2 cores, 32G, and a k80.
Then packing a p2 instance would run you $7.2 (or $14.4, etc..) and back of
the napkin math says you can pack about 15 gpu jobs in a p2.8x. So you hour
cost on a job is $0.48.

Just an observation :).

~~~
reubensutton
I guess the profit margin comes from the $99/user fee

~~~
eudoxus
ahhh, thats true.

~~~
saip
Our Individual plan has no monthly fees, and is purely pay-per-use @ <50% of
AWS pricing.

------
ghego
Have been using this for a couple of weeks and it looks really promising! It
definitely feels like Heroku for deep learning!

------
ryanchan001
The github page for floydhub has a lot of open source projects. What parts of
floyd is not open sourced?

~~~
narenst
Deep Learning is a very open source friendly community - most of the popular
frameworks/algorithms are in fact open source. In fact, Floydhub is built on
top of a lot of open source projects. We definitely considered open sourcing
the core components of Floyd but the Rethinkdb fiasco made us "rethink" that.
Instead, we are supporting the open source community by hosting Deep Learning
Docker images, popular datasets, and projects. In the future, we are planning
to open source more parts of Floyd but not all (similar to Github).

------
ccarnino
It feels like a good product. I will try it soon!

One small piece of feedback. If you open your landing page from mobile, e.g.
My iPhone 7, the header height is changing all the time due to the text
changing dynamically. It get hard to scroll past it and keep reading.

Cheers

~~~
saip
Thanks - fixed now! :)

------
siliconc0w
Per second pricing and jupyter mode is pretty nice. Like heroku you're trying
to resell AWS and stay competitive which is a challenge. Anyway, this is
indeed a common problem and this looks like a solid approach.

~~~
saip
Fully agree! In the longer run, I believe there might be some great
opportunities wrt infrastructure. GPUs instances are super expensive (now
combine with long runtimes). Self hosting infra at scale can drastically drive
down prices 10x+ (we're already 2x cheaper than AWS). Till we get there,
there’s a few lower hanging fruits to pick, like streamlining development
processes and making things easier. Glad you like the Jupyter Notebooks!

------
fnbr
This seems pretty awesome, but I can't get it working with my code. Do you
have any plans to release extra documentation, or examples of how to adjust
existing models to run on FloydHub?

~~~
narenst
Sorry to hear that, we are constantly improving the docs. You can find the
latest version here: [http://docs.floydhub.com/](http://docs.floydhub.com/)

If something is not clear, contact us in the communication app on the website.
We will be happy to assist you in any way possible.

~~~
fnbr
I was able to get a different model working, and FloydHub is great- I'm really
finding it useful. I'm looking forward to see where you take it!

------
reubensutton
This looks really cool, I'm slightly confused why the type of CPU intensive
workloads only has a single core though?

~~~
saip
We picked this as a starting point to assess demands, e.g. CPU vs. GPU, before
scaling out. We'll be adding more tiers very soon. What kind of instance would
suit your needs?

------
michwill
Congrats with the launch!

------
omarish
Congrats on the launch!

