
Peloton: Uber’s Unified Resource Scheduler for Diverse Cluster Workloads - r4um
https://eng.uber.com/peloton/
======
elvinyung
>Kubernetes hasn’t been able to scale to the large clusters that Uber
requires, i.e. 10,000-plus, nor does it support elastic resource sharing. It
is also not the ideal scheduler for batch workloads due to the high-churn
nature of batch jobs.

Feels like a semi-weird justification for NIH. Isn't there pretty active work
in addressing most/all of those objections in k8s?

~~~
tapoxi
Didn't they also have a weird "Postgres can't suit our needs so we're
switching to MySQL" argument?

~~~
cphoover
yurp

------
romed
It's interesting how big of a difference it makes when you allow local disk
I/O, and you need to schedule around it. At Google they disintermediate
storage and all (not really all) I/O goes to their cluster FS (Colossus, sort
of). They don't have to schedule around I/O resources because every process
has access to the full I/O resources of the entire cluster at any time from
any node. By contrast as soon as you let some open source or off-the-shelf
commercial thing leak into your operations, it will demand ordinary POSIX disk
I/O and then you've got big problems. I propose that some companies would
actually be better off concentrating on disintermediated storage more, and I/O
workload scheduling less.

~~~
closeparen
The “Storage as a Service” needs to run somewhere. You can’t escape from
running a storage aware scheduler. Either it will be the same as your primary
scheduler, or storage services will be special snowflakes.

~~~
romed
There are always daemons on the machines that aren’t under the control of the
cluster scheduler (init or its equivalent, etc). The thing that exports the
storage assets of the box can simply be one of those things.

------
SteveNuts
I'm always bummed when I don't see Nomad listed in blog posts like these. It
has quite a few powerful features that people overlook.

------
dominotw
CMU has a "self driving" db with the same name [https://github.com/cmu-
db/peloton](https://github.com/cmu-db/peloton)

~~~
JumpCrisscross
Also, there's a B2C company called Peloton [1].

[1] [https://www.onepeloton.com](https://www.onepeloton.com)

------
crb
Nothing in the article about this being open source or available to the
public?

~~~
packetslave
why did you think it would be?

------
mistrial9
from the article -- Peloton is built on top of Mesos

~~~
detaro
They use mesos, and the article explains why they added something to it.

