
Ray: A Distributed Framework for Emerging AI Applications - rshin
https://arxiv.org/abs/1712.05889
======
t3io
Interesting Framework. Seems to hail from : >
[https://rise.cs.berkeley.edu/](https://rise.cs.berkeley.edu/) Quite the bee-
line from the authors' previous papers. It only seems to have been preceded by
: [https://arxiv.org/abs/1703.03924](https://arxiv.org/abs/1703.03924)
However, the language in the [Ray] paper is completely different. Different
from the preceding paper but familiarly distinct. Any specification of outside
groups/companies/individuals who were consulted/collaborated with on this
project? And who more or less lead the development and design? Maybe I am
jumping the gun, but it clearly wasn't of the listed names.

Of all of the authors listed, none of their previous papers read like the Ray
paper nor does the proposal paper (Real-Time Machine Learning : The missing
pieces). [http://arxiv.org/abs/1703.03924](http://arxiv.org/abs/1703.03924)
reads like a corporate/industry grade requirements/proposal doc which is a
huge departure from all of the author's prior papers... So, out of the blue,
corporate level infrastructure project proposal and completion within the span
of a year?

Can any of the paper's authors speak more clearly on who led this project over
what span of time, under what direction, and with which industry groups? I see
no background or papers from the individuals priority listed in the paper
reflective of the sort that creates a formalized and industry grade
Distributed Computational Framework such as this. > Robert Nishihara > Philipp
Moritz Are priority listed yet have no prior papers leading to such a
development. To what degree did :
[https://rise.cs.berkeley.edu/sponsors/](https://rise.cs.berkeley.edu/sponsors/)
Drive this?

~~~
pcmoritz
One of the authors here! We are glad you like it!

The project is indeed driven by the authors listed on the paper and also the
knowledge and experience that was accumulated in the AMPLab (the predecessor
of the RISELab, see
[https://amplab.cs.berkeley.edu/](https://amplab.cs.berkeley.edu/)). If you
look at the github history, we've been working on it for longer than a year
and had various prototypes before that, so it doesn't come out of "thin air"
;)

The lab's sponsors are also helpful, some of them have been experimenting with
the system internally and giving us feedback.

~~~
t3io
Thank you for responding. I indeed spent time/effort in way of my crafted
reply because it caught me eye. I haven't fully parsed the paper but covered a
number of pages that colored the nature of my inquiry. I in no way intended to
take anything away from the author of the paper but wanted to get at what you
yourself declared : "was accumulated in the AMPLab (the predecessor of the
RISELab, see
[https://amplab.cs.berkeley.edu/)"](https://amplab.cs.berkeley.edu/\)") as The
backstory behind the paper as I clearly surmised there was one its historical
nature. I also wanted to understand how long this was being worked due to the
nature of the language used in the paper and how the concepts and language
familiarly fit in with other things I've seen. And this right here : "The
lab's sponsors are also helpful, some of them have been experimenting with the
system internally and giving us feedback." Yes, I understand the nature of
this is moreso for corporate use cases than it is for academic and furthering
therein. I was in search of names but already have a number of them I can
surmise and a handful more that I will derive. I think its interesting what is
being done here but there were choice words that were stated in the paper that
limit it. At this juncture and time in the state of AI development, I will
reserve any other commentary beyond stating that there are an incredible
amount of fundamental limits in approaching things this way that fall on deaf
ears due to the shut off nature/sponsorship of such developments. I wish you
guys the best and am sure there will be traction as it relates to RL.

------
lmeyerov
Also related: [https://arrow.apache.org/blog/2017/08/08/plasma-in-memory-
ob...](https://arrow.apache.org/blog/2017/08/08/plasma-in-memory-object-
store/) . The Graphistry team has been super excited about Plasma as part of
how we're getting our NodeJS stack to talk to more GPU & fast data compute
pipelines, and suspect it'll help others as well. If you're looking for a cool
nodejs winter project, happy to share roadmaps!

------
lowglow
There is a Decentralized AI Summit[0] happening in San Francisco in Feb that
currently has a call for presenters. You should reach out and show off your
work there.

[0] [http://decentralized-ai.com/](http://decentralized-ai.com/)

~~~
elvinyung
Ray isn't decentralized.

~~~
lowglow
I would say distributed frameworks are in the wheelhouse typically when
talking about decentralization and AI applications.

~~~
elvinyung
That connection is a stretch at best.

Virtually all modern "big data" platforms are necessarily "distributed" just
by the virtue of taking up more than one computer. They also necessarily
happen to have some kind of centralized "brain" (i.e. scheduler, which is what
Ray basically is).

I know blockchain and AI are the hot buzzwords right now, but it's really not
related to the kind of decentralized computing that you're probably thinking
about (i.e. Golem-style agoric computing).

------
juanmirocks
How does this stack against flink or spark or is it trying to solve something
else ?

~~~
Eridrus
It's trying to solve different problems. AFAIK they're trying to tackle things
like distributed reinforcement learning and control among a set of robots.

------
atrilumen
[https://github.com/ray-project/ray](https://github.com/ray-project/ray)

------
wskinner
Can anyone comment on how Ray fits into the current distributed systems
landscape? How does it compare to Spark, Hadoop, Akka, distributed Tensorflow,
etc.?

It would be really helpful to have an FAQ on the github.io page that answers
the question "When should I use Ray vs. X".

------
RHSman2
I think this is amazing and hope it achieves its future goals of bringing
computational speed up to the requirements for on the fly decision making.
Bravo.

------
stuffedBelly
hmm...have they checked out ROS? I feel like this can be built on top of ROS
as a package rather than an independent framework.

[http://www.ros.org/](http://www.ros.org/)

