
Kubeflow – Machine Learning Toolkit for Kubernetes - nikolay
https://github.com/google/kubeflow
======
TheIronYuppie
Hi! I’m David Aronchick, PM on Kubeflow, I’m happy to answer any questions! I
was one of the early PMs on Kubernetes, and we very much want to make this a
community project, so please join us in thinking about what’s next!

\- GH:
[https://GitHub.com/Google/Kubeflow](https://GitHub.com/Google/Kubeflow)

\- kubeflow-Discuss: [https://groups.google.com/forum/m/#!forum/kubeflow-
discuss](https://groups.google.com/forum/m/#!forum/kubeflow-discuss)

NOTE: The name "flow" does not refer directly to TensorFlow; if anything it's
a nod at all the river themes that pop up in the ML community (eg FBLearner
Flow)

Disclosure: I work at Google on Kubeflow

~~~
ethbro
Kudos on the release, and thanks for your and the team's work!

Any thoughts on this vs managed ml-engine? Cost-aside, seems like this nibbles
on the smaller scale "but ml tooling is too hard" use cases?

~~~
TheIronYuppie
Thank you! We still love Google Cloud ML engine - it's perfect for those who
want to run in the cloud and want a layer of abstraction. This is for people
who want portable stacks and a bit more control; and/or want to use their
Kubernetes deployments (particularly on-premise or multi-purpose).

Does that help?

Disclosure: I work at Googlr on Kubeflow

~~~
ecnahc515
Do you foresee some kind of integration between Google Cloud ML engine and
Tensorflow on k8s in the future?

~~~
TheIronYuppie
We're always ready to talk roadmap - anything in particular you'd like to see
integration-wise?

Disclosure: I work at Google on Kubeflow

~~~
ethbro
Off the top of my head, maybe a maintained "ml-engine aligned" kubeflow setup,
to the extent that's possible.

The use case I'm think of is an ml dev team building on kubeflow and proving a
system. Then wanting to transfer it to a non-engineering team, yet wash their
hands of any ongoing infrastructure ops responsibility.

Knowing that a "ml-engine aligned" kubeflow config would transfer cleanly
(including associated bells and whistles) would make that a much more
attractive option.

Caveat: I'll admit I'm not keeping up on what's in the managed offering, but
I'm assuming there are a number of value-adds of the type that end users like
(visualizations, etc).

~~~
TheIronYuppie
Yes, this is EXACTLY what we're trying to do! However, it's a bit early, so I
can't say when or where we'll be able to get to it. Also, I should be clear,
though I'm from Google, we would really like the same story to work with other
cloud's hosted offerings as well, but we'll need their support to do so!

Disclosure: I work at Google on Kubeflow

------
ericand
Looks like different components could be added in the future, but not clear
how.

The following are included:

\- A JupyterHub to create & manage interactive Jupyter notebooks.

\- A Tensorflow Training Controller that can be configured to use CPUs or
GPUs, and adjusted to the size of a cluster with a single setting.

\- A TF Serving container.

~~~
TheIronYuppie
Correct! We're currently thinking a lot about orchestration of the various
components but for now, our goal is to use the native loose coupling between
services available in K8s. So if you wanted Spark for data processing, for
example, you could start a service, and the deployment, and feed that into the
TF CRD.

Disclosure: I work at Google

------
humanfromearth
Any chance to make this a helm chart?

~~~
yuvipanda
Chime in at
[https://github.com/google/kubeflow/issues/23](https://github.com/google/kubeflow/issues/23)
:)

------
mountainerd
Pretty interesting. I'm guessing this is something that Google uses internally
for their Kubernetes workflows.

~~~
TheIronYuppie
Its very close to how we think about ML internally, but not what we use. Your
best bet to read that is look at the TFX paper[1] which describes our internal
thoughts in great detail. (Though Kubeflow is not designed to be an
externalization of TFX, we're very much working in collaboration with that
team)

[1] [http://www.kdd.org/kdd2017/papers/view/tfx-a-tensorflow-
base...](http://www.kdd.org/kdd2017/papers/view/tfx-a-tensorflow-based-
production-scale-machine-learning-platform)

Disclosure: I work at Google on Kubeflow

~~~
quadrature
Does google intend on open sourcing TFX ?. I only ask because we're building a
lot of the same infrastructure.

~~~
TheIronYuppie
We're absolutely looking at it! Please join our discussion, we'd love to talk
about what you're building and if we can help and/or what you'd like us to
OSS.

Disclosure: I work at Google on Kubeflow

~~~
quadrature
Sure, is there an issue/doc/pr to comment on ?.

~~~
TheIronYuppie
No, would you mind adding one?

------
smooc
This looks pretty cool. Is it dependent on Google’s kubernetes or can it be
run on Openshift or DC/OS as well?

~~~
ericand
It says it runs in "in any environment in which Kubernetes runs." So as long
as you are asking if it runs on Openshift's Kubernetes, than yes.

~~~
TheIronYuppie
Absolutely! Redhat are already contributing :)

Disclosure: I work at Google on Kubeflow

