
Orchestra: An open-source system to orchestrate teams of experts and machines - marcua
http://orchestra.unlimitedlabs.com/
======
jrbotros
Hi, Hacker News!

Today we’re excited to be making our first contribution back to the tech
community by open sourcing Orchestra, a system for coordinating project teams
of experts and machines. It can be used to reimagine and implement all sorts
of workflows, from professional services (like legal, design, and data
analysis) to new product experiences (like M from Facebook). Orchestra is
based upon years of research into machine-mediated expert teams. We hope
you’ll come up with new and creative ways of using it!

At Unlimited Labs, we’re excited about building a brighter future of work and
are currently building our first product on top of Orchestra. We’d love to
answer any questions you have on here, or feel free to email the team at
hello@unlimitedlabs.com!

------
marcua
(Orchestra dev here)

If you're curious how it can be used, Daniel Haas put together a wonderful
example of how Orchestra could be used in a newsroom:

[http://orchestra.readthedocs.org/en/latest/example_use.html](http://orchestra.readthedocs.org/en/latest/example_use.html)

Happy to take questions!

------
IanCal
This looks absolutely wonderful. A few random thoughts:

1\. I'm going to play with this as a way of documenting processes. Being able
to answer "how do you develop features" or "what's best practice for X" with a
usable process is great.

1 a) I'm particularly interested in using this for documenting how to perform
experiments, and tracking experiments as they go.

2\. I think this is amazing for progressively automating things. Split the
task up into manageable, clear chunks to be performed by people. Each step can
then (if it seems worth it) be replaced by code.

3\. I don't think this is present at the moment (and it probably shouldn't be
for a first release) but I'd like to see process hierarchies and importing
others. I'd like to say 'Use github's "GITFLOW V2" process for managing
feature additions and use "MY_COMPANY_QA V3" for reviewing features before
full release'.

Anyway, this is great, and I look forward to playing around with it.

edit -

Even without these, I can already see myself using it just for me.

edit 2 - It'd be good to work on the getting started, so that there's just a
"run this script" or "clone this repo". There's a lot of "download this file
and copy it here" that feels like it should be scripted.

~~~
dhaas
Thanks for the feedback!

Your comment about embedding external QA workflows is super interesting. Right
now Orchestra does support what we call 'Review Policies', which allow you to
specify that a given workflow step be reviewed k times, or with probability X,
or by specific expert Z. But it would be interesting to think of whole
workflows for QA.

Re: getting started, yeah, we're working on a script--turns out cross-platform
support is hard!

~~~
IanCal
> Right now Orchestra does support what we call 'Review Policies', which allow
> you to specify that a given workflow step be reviewed k times, or with
> probability X, or by specific expert Z.

That's handy, I can think of a bunch of things that fit well with that
(particularly review a subset of x%)

> Re: getting started, yeah, we're working on a script--turns out cross-
> platform support is hard!

Hah, yeah it is!

I've got a version setup now, but am struggling a bit to setup workflows &
figure out quite what goes where, but I think I've found the right bit of
documentation:
[http://orchestra.readthedocs.org/en/latest/example_use.html](http://orchestra.readthedocs.org/en/latest/example_use.html)

As a suggestion, if you add a new example, try doing it step by step of what
you need to add & do. It's more work, but I find it really helps show what
someone will actually experience while creating / updating workflows.

~~~
dhaas
We have an issue open to improve the journalism exaple:
[https://github.com/unlimitedlabs/orchestra/issues/32](https://github.com/unlimitedlabs/orchestra/issues/32).
If you have concrete suggestions for how to make it better, we'd love to hear
them--especially for documentation, which can always use work!

Also feel free to reach out to us on gitter
([https://gitter.im/unlimitedlabs/orchestra](https://gitter.im/unlimitedlabs/orchestra))
if you're having trouble getting your local setup to run the workflow and we
can talk through the details.

------
usman-m
Great stuff! For now it seems like this leaves out the job of finding crowd
workers. Could it be possible to do something like designing microtasks in
MTurk and "stitching" the workflow together using Orchestra?

~~~
dhaas
(Orchestra dev here too!) That's a great question. We're hoping that emerging
online expert communities like Upwork or Dribbble will make it easier and
easier to find experts suited to the work, but it's definitely not a solved
problem (or tech recruiting would be so much easier!).

Re: MTurk, there's nothing stopping you from stitching microtasks together
with Orchestra, but I think that undersells Orchestra's strengths (e.g.,
collaboration among workers) a bit.

------
CoreSet
This might seem a dumb question, but can you talk about how this improves on
existing productivity workflows? Does it integrate with tasking systems like
Jira or is this meant to be a complete a replacement?

It seems high-level (and great) I'm just curious what implementing it "in the
field" would look like.

~~~
dhaas
Products like Jira help organize tasks within workflows, but don't automate
any that organization. Orchestra takes care of things like "which reporter
should I assign to a new story?" without requiring humans to make those
decisions.

Research from the Flash teams project that I mentioned in another comment
([http://stanfordhci.github.io/flash-
teams/index.html](http://stanfordhci.github.io/flash-teams/index.html)) has
shown that this type of automation can decrease end-to-end workflow time by as
much as 2x!

------
SamBam
Does this tie into products like PivotalTracker or JIRA to make it useful for
agile software development?

It would be nice to be able to create tasks from stories in Pivotal, assign a
reviewer to them, and, when a reviewer thumbs-up, have it automatically built
into a staging server. Anything like that in the works?

~~~
dhaas
We haven't built that integration yet, but it's totally possible! Really all
it should take is a Machine Step that calls out from Python to, say, Pivotal's
APIs
([https://pypi.python.org/pypi/pivotaltracker](https://pypi.python.org/pypi/pivotaltracker)).

------
zallarak
This seems very cool - are there case studies of adopting this type of
workflow for complex projects?

~~~
dhaas
I don't know of any formal case studies, but Adam Marcus and myself (two devs
on the project) worked on a large-scale workflow at GoDaddy where we
automatically organized data extraction experts into hierarchies of review.
That work actually got published at a Databases conference: see Adam's nice
summary here:
[http://blog.marcua.net/post/128120315677](http://blog.marcua.net/post/128120315677)

On the research side of things, the flash teams work by Daniela Retelny at
Stanford ([http://stanfordhci.github.io/flash-
teams/](http://stanfordhci.github.io/flash-teams/)) has done great proof-of-
concept things like build an app in a day with distributed teams.

------
keredhaas
This looks really exciting, and I can't wait to try it. I think it will be
very helpful for my business.

------
dharma1
i think this looks potentially great - would definitely benefit from more real
world examples of workflows

------
ggiacaglia
This seems very cool!!!

------
johndoe7
Are musical instruments provided?

~~~
dhaas
They don't come in the box, but you're free to hire musicians for your
workflows!

------
tilek
This is very exciting!

