
Open Sourcing Peloton, Uber’s Unified Resource Scheduler - brugidou
https://eng.uber.com/open-sourcing-peloton/
======
spyspy
Is it just me or is every big release from Uber just a custom rewrite of an
existing technology? It seems their engineering department has a large not-
invented-here attitude. I could be wrong - they're certainly large enough to
have custom requirements that aren't met with what's on the market but the
pattern is just becoming suspect.

~~~
warp_factor
I think you are right.

I think that what is going there is also a bit political. They started to grow
their Engineering department so fast that they need to justify the headcounts
now. So each team is trying to invent new projects all the time. Anecdotally,
this was partially confirmed to me by a friend working there.

I said this before, but I still cannot understand why a service like Uber need
so many engineers in the backend (multiple thousands). It is a complex
distributed application, but nowhere near the scale or complexity of a
Facebook or Google.

~~~
theon144
>I said this before, but I still cannot understand why a service like Uber
need so many engineers in the backend (multiple thousands). It is a complex
distributed application, but nowhere near the scale or complexity of a
Facebook or Google.

Thank you so much, I thought I was going crazy. I understand the demands of
running a service on the level Uber has, but well, for instance I can't
imagine what kind of computational workload / infrastructure requirements
would make developing your own resource scheduler a reasonable option - for a
Taxi app? With non-essential (to the core product) machine learning?

Forgive me if I'm ignorant, but what exactly does Uber engineering team do?

edit: On their blog I was able to find that they namely "forecast rider
demand", from a relatively small [0] article - that is, comapred to the
article [1] about what essentially is "just" data visualization, which doesn't
help my confusion much.

0 - [https://eng.uber.com/neural-networks/](https://eng.uber.com/neural-
networks/) 1 - [https://eng.uber.com/maze/](https://eng.uber.com/maze/)

~~~
jvanderbot
Those cars generate a _lot_ of sensor data. (Tb per drive?). Id imagine that
data needs to be made actionable and seperated into training and simulation
sets pretty quickly. Mapping is a massive problem to automate.

~~~
theon144
Oh right, the self-driving cars. Well that's starting to make sense now.

------
a-wu
I’ve worked with Mesos pretty extensively before and when Uber first announced
Peloton last year I was intrigued. Peloton seems to be a wrapper around Mesos
that allows for running smaller, unique jobs without having to write a Mesos
framework for each. Writing a Mesos framework for every small job you have can
get annoying when you just want to define how your job should run and don’t
really care for the resources or task allocation of the job, and it seems like
Peloton solves this on Mesos. It’s similar to YARN but not limited to Hadoop.
It would have been useful for the project that I worked on because it was more
geared for our use case and shifting from Mesos to k8s would’ve been a huge
engineering project.

~~~
filereaper
Mesosphere created dcos-commons to reduce the friction of getting frameworks
up and running.

[1] [https://github.com/mesosphere/dcos-
commons](https://github.com/mesosphere/dcos-commons)

------
frostyj
Just call it (Uber) customized mesos. I find this article somehow deceiving
and boring. I am pretty sure I can run this peloton thingy with most Mesos API
calls.

~~~
a-wu
It pretty much is, but the appeal (at least to me) is abstracting away all the
framework writing. It seems like it’s easier to run small, unique jobs on it
similar to something like YARN.

------
thekhatribharat
What exactly is Uber running in its clusters?

\- Route Optimization

\- Demand Forecasting

\- Rider Hotspot Prediction

This post doesn't exactly tell us the true nature of their workloads (other
than the crude categorization - batch, stateful, stateless), nor does it talk
about the inflection points where off-the-shelf solutions don't cut it anymore
and such customization is required. I mean some _before & after numbers /
graphs_ on resource utilization would have really helped.

------
zachrose
At first glance I thought the "resources" here might be Uber drivers, which
would be interesting.

Is this thing/Apache Mesos abstract enough to allow for such a use?

~~~
roskilli
The best thing I can think of that fits what you're asking is an Actor
framework (which abstracts the compute and message passing between objects for
you).

Think Akka or New Orleans, etc.

------
anonfmrtser
This sounds a bit like Two Sigma’s Cook scheduler for Mesos.
[https://github.com/twosigma/Cook](https://github.com/twosigma/Cook)

------
r_klancer
Immediate reaction: Well, now there's Peloton, the fitness tech company,
Peloton, the self-driving truck caravan company, and Peloton, the cluster
scheduler...

~~~
jonny_eh
Where do all these names come from? Is it Latin or Greek or something?

~~~
WorldMaker
Peloton is a somewhat common French word for "ball" that became common in
English as a sports term for a grouping in a bicycle race, by way of the Tour
de France. [https://www.merriam-
webster.com/dictionary/peloton](https://www.merriam-
webster.com/dictionary/peloton)

~~~
jeromegv
Small detail, that's not what it means in French, it's basically used the same
was as in English (a group of cyclist, group of race car drivers, etc.)

It is however pelota, ball, in Spanish.

------
eropple
My first reaction was "this just sounds like Mesos?". And it's cited in their
page (which on first read I thought meant they were trying to act as a single
pane of glass for Mesos/k8s/etc.):

[https://eng.uber.com/peloton/](https://eng.uber.com/peloton/)

In the OP blog post though, they assert "to our knowledge, there is no other
open source scheduler which combines all types of workloads for web-scale
companies like Uber."

And then, when you dig...it's just Mesos. They built a framework for Mesos.
So, that's cool. But _man_ , the puff piecery borders on dishonesty. I mean--
Singularity has existed, and is implemented at very large scales, for a while.
I'm sure Peloton is a fine scheduler, but there's a lot of huffing-one's-own-
farts in the documentation here.

~~~
subway
Singularity isn't a scheduler. It's generally invoked by traditional hpc
schedulers like moab or slurm.

edit: clearly I'm thinking of a different Singularity than you.

~~~
eropple
Yeah - on a phone but I'm referring to the Mesos framework by HubSpot.

------
notyourday
You mean not a bike company?!

------
dtrailin
Peloton should make a framework called Uber in response.

~~~
platinumrad
The term "peloton" obviously predates the company of the same name...

~~~
fenwick67
The term "uber" obviously predates the company of the same name...

------
ProAm
Would people use open source stuff from a morally questionable company?
Especially when its just a re-write of existing technology posted to a
different github repo?

 _EDIT_ People get so up in arms about Google and Microsoft working with China
and the military, but Uber has done some horrendous stuff on their own. Just
curious where people think the line is OK to be.

~~~
baroffoos
Using their open source stuff doesn't fund them to do bad things.

~~~
mandelbrotwurst
Arguably it does indirectly - e.g. being able to say that you're responsible
for Popular Tool X might help your brand, make it easier to make sales, etc

