
Argo: Open source Kubernetes native workflows, events, CI and CD - jcamou
https://argoproj.github.io/
======
bwarminski
We're using Argo as a replacement for our GoCD based continuous delivery
system and it's been fantastic. We were about to move onto Concourse and
realized we would need to set up the and manage the equivalent of a Kubernetes
cluster just to support it. We took a few extra weeks to prototype Argo first
and were glad we did.

In our use case, the primary deployment model is blue/green deploys of VM
based microservices on AWS with a bunch of Terraform managed infrastructure
gluing them together. The newer class of service we're developing right now is
container based and has even more infrastructure complexity. Teams manage
their own service releases and each group has slight variations in their
deploy steps. We need a deploy system that can support the old and new style
and remain flexible while providing a migration path.

Teams love the flexibility and expressiveness of the workflow definitions.
They've begun to move cron jobs onto it as well. Argo is much more lightweight
and easy to operate than any other system we had worked with because it leans
so heavily on Kubernetes primitives that we already needed to understand. The
codebase is also relatively easy to understand, so we've been able to
contribute things back to the project while working through the migration.

We've taken a cue from the project and begun to consolidate our control plane
on CRDs so they can seamlessly integrate with Argo. CRD Operators + Argo are
allowing us to consolidate all of the custom deploy/config tooling we built
over the years onto a common system that is testable and well integrated.

It's a little early in the project to measure the full effect, but the
internal project has a lot of momentum.

------
mattashii
The idea is nice, but if security is one of your concerns, you'll probably not
want to run this if the docs are even close to representative of the actual
application:

> 3\. Configure the service account to run workflows

> For clusters with RBAC enabled, the 'default' service account is too limited
> to support features like artifacts, outputs, access to secrets, etc... Run
> the following command to grant admin privileges to the 'default' service
> account in the namespace 'default'

This raises so many red flags. Never give admin rights 'as a default'.

~~~
nrmitchi
While I agree with your position that admin rights should not be the default,
that doesn't seem to be what Argo is actually doing.

The text snippet you've included is from the quick start/demo page, which
seems to be targeted at getting the examples running quickly so that a
potential user can see what it is capable of. Other issues for this in a
production setup would be the use of the 'default' role/namespace at all. In
this use case (running the examples in a new cluster), giving the default user
full access is the quickest way to this result.

Clearly it would be better if Argo included a more precise (permission wise)
role by default, but from my understanding Argo does not require cluster admin
permissions in order to function.

~~~
TheDong
The parent post did quality it appropriately with "if the docs are even close
to representative of the actual application".

I hope the docs are not representative, but the parent post is right that it's
a bad sign. At least put up a warning sign that it's insecure and only
suitable for demo purposes to do that.

~~~
jessesuen
Thanks. We've updated the docs to outline the bare minimum set of privileges.

------
drewda
We've had much success with Argo workflows over the past months. Among other
projects, we use it to generate city-sized extracts of mapping data from the
OpenStreetMap planet. Blog post at [https://www.interline.io/blog/scaling-
openstreetmap-data-wor...](https://www.interline.io/blog/scaling-
openstreetmap-data-workflows/)

Using Argo on Google Kubernetes Engine makes it easy to auto-scale for our
workflow steps that parallelize. We're also able to reuse Dockerfiles and
resources between both our batch workflows and our running services (also on
Kubernetes).

------
austinshea
What does this project simplify?

It seems to be equivalently complex to the process it would have you replace.

~~~
jessesuen
I always explain that Workflows should be thought of as a fancier version of
Kubernetes Jobs. Some of the killer features, IMO, are with artifact
management and control flow (e.g. loops, conditionals, parallelism).

So far, our most common use case has been in the machine learning space. One
of the biggest problems it can solve is the ability to leverage a Kubernetes
cluster as an auto-scaling compute grid for all forms batch processing. We've
been working closely with ML teams at various companies, both large and small,
and many of our features were implemented to address the ML use cases (e.g.
DAG support, loops/step expansion, step aggregation).

With that said, Argo got its start trying to address the CI use case, so we
also feel like we handle that one pretty well.

The workflow controller has always been intended to be a lego block to enable
higher level applications to be built on top of it.

------
devedlee
FYI, Google's new Kubeflow Pipelines service uses Argo.

[https://cloud.google.com/blog/products/ai-machine-
learning/i...](https://cloud.google.com/blog/products/ai-machine-
learning/introducing-ai-hub-and-kubeflow-pipelines-making-ai-simpler-faster-
and-more-useful-for-businesses)

[https://github.com/kubeflow/pipelines#acknowledgments](https://github.com/kubeflow/pipelines#acknowledgments)

------
wepower_ico
Is there easy way to build containers ?

I see many resources for running stuff with already built container but could
not any simple how to do it. Currently just mounting docker host and building
on it.

~~~
meddlepal
Dockerfiles?

~~~
meddlepal
Edit: I misunderstood the parents question.

------
pm90
Very very small suggestion: its not very expensive to register a domain and
have it redirect to the github pages; might make it appear somewhat more
polished.

~~~
gcb0
"that's like yout opinion, man"

registering vanity domains is a shure way to make this information harder to
find in a few years

