
PAAS Comparison – Dokku vs. Flynn vs. Deis vs. Kubernetes vs. Docker Swarm - wilsonfiifi
http://www.jancarloviray.com/blog/paas-comparison-2017-dokku-flynn-deis-kubernetes-docker-swarm/
======
likelynew
Why is upvoted so much. This just touches the surface. Although, I am not very
related to PaaS, I know much more for the three of the methods listed. This
even oversimplifies so many things, to the point of getting it wrong.

> WHEN TO CHOOSE DOKKU: for hobby and side projects that do not require high
> availability

Surely, not correct. Don't make your decision on this line. It's not made for
fault tolerance, but anyone who has doubts regarding which to use is very
likely to use a single region of single cloud provider. I think most
companies(even large one) do that.

> Big caveat (for Kubernetes): it is very hard to setup

I really don't think he did that. It's really easy for AWS at least. See
[https://kubernetes.io/docs/getting-started-
guides/aws/](https://kubernetes.io/docs/getting-started-guides/aws/)

> WHEN TO CHOOSE DOCKER SWARM: in my opinion, this has a very strong potential
> to be a contender against current solutions mentioned above but as of right
> now, it is still evolving.

AFAIK, it has been extensively tested and is being used live. I am not sure,
but the amount of presence it has, it's hard to believe that it is just
evolving and not to be used.

~~~
deforciant
Swarm is used but I have heard many companies migrating from swarm to k8s and
openshift. Swarm is just unstable, network connectivity is flaky, not sure how
can you run it in prod (maybe you can't?).

As for k8s - would recommend it for anything from hobby projects to large prod
deployment. Easy to set up anywhere.

This article is indeed very shallow and clearly author hasn't bothered to do
some research.

~~~
likelynew
> This article is indeed very shallow and clearly author hasn't bothered to do
> some research.

Well, yeah, but the thing concerning here is HN upvoted it to the top. I have
nothing against the author.

> As for k8s - would recommend it for anything from hobby projects to large
> prod deployment

That's what I am saying. IMO, the reputation that it has to be used only if
somewhat wanted to manage very large number of servers is a little misleading.
I think Kubernetes can feel very intimidating at the start but it's not that
hard.

~~~
bryanlarsen
Just today I spun up a single node Kubernetes production cluster. Obviously
not its sweet spot, but the ecosystem, flexibility and tooling made it an
appropriate choice, given that I already know Kubernetes and that I'll be
deploying the same software elsewhere on "real" clusters.

------
CrankyBear
So much is wrong here it's hard to know where to begin. First, PaaS is not
container orchestration. Second, this is extremely superficial. Third, some
things are flat out wrong. For example, Docker Swarm has been depreciated.
Swarm's capability is baked into Docker as of 1.12. For a better overview see:
[https://insights.hpe.com/articles/the-basics-explaining-
kube...](https://insights.hpe.com/articles/the-basics-explaining-kubernetes-
mesosphere-and-docker-swarm-1702.html)

------
eliq
So, the way I deploy code is to SSH into my server, clone my repo from GitHub,
setup the db and start the relevant systemd services. If I need to install
updates it's a simple git pull and restart the services. I can use something
like ansible for configuration management to reduce the manual SSH part.

For someone like me, what benefit do these PaaS offerings provide?

~~~
PaulRobinson
Automated repeatability, automated rollbacks, self-documenting infrastructure
and the ability to deploy with a single command or automatically on a
successful test build.

That means you can merge to master and know the build will just get released
when the tests pass without having to do any of that work. Further, as the
needs of your infrastructure or DB develop (you are hoping for growth,
right?), your deployment stays automated, you can add more complexity to your
setup with little additional workload, and you can hire other developers who
can safely ship to production without having to be shown any of your method.

~~~
Silhouette
The question always seems to be how much extra overhead you incur setting
these things up in the first place for a relatively small business or side
project. I've been curious about these kinds of automation tools for as long
as they've been around, but they always look like more trouble than they're
worth for any of the projects and deployment strategies I'm involved with at
that smaller end of the scale.

Maybe they work better if you're doing a relatively standard DB + API + front-
end kind of setup, which as it happens nothing I work on actually is. Or maybe
they just don't really pay for themselves until you're working at a scale
where a simple copy/clone step and everyday scripting tools aren't sufficient
to run everything routine anyway?

~~~
WrtCdEvrydy
I swear by Flynn because it can mount Web Apps, Docker Containers and just
anything that runs inside of Docker under Linux.

I often find new sexy stuff on the Hacker News frontpage and just mount it
right on my Flynn cluster to fuck with it.

~~~
Silhouette
I suppose my question would be why a lot of small systems would even need
something like Docker. The theoretical benefits certainly would apply to
various projects I work on, but the reality is that businesses at this scale
often set up a standard system image and deploy it on a static set of real
servers. Once that's done, the foundation might not change for months or years
at a time.

There are usually two significant exceptions. One is security updates, which
are typically monitored and tested/deployed via a separate process anyway. The
rest is the day-to-day development work and deploying new assets, which
typically needs no more than some sort of copy/clone job from the VCS to get
the data to the servers and then running a single script to deploy/activate
everything.

If you're working with hundreds of servers or dynamically reconfiguring things
all the time in a Cloud environment, obviously the situation may be different.
However, for the simpler cases -- and you can get a long way with them if
you're just running a normal business and not trying to be the next unicorn --
the whole process already seems reasonably reliable and efficient anyway.

I'm a software guy rather than an operations specialist, as are the people I
work with in almost all cases, so I always have some nagging doubt that we
just have a total collective blind spot here. But so far, speaking only about
the smaller scale and more static deployments I tend to work with, these kinds
of tools seem like solutions to problems we don't often have.

~~~
WrtCdEvrydy
I'm a software guy too
([https://github.com/WriteCodeEveryday](https://github.com/WriteCodeEveryday)),
but I have spent an entire 3 years of my life building prototypes and software
for small businesses, that just aren't technically savvy and don't have large
IT departments.

Once I deployed / configured my fifth Postgres database with Rails, I decided
enough was enough, I needed to move faster in my Operations setup since I'm
the "ops guy". If you're in a company that has dedicated capable ops guys, you
don't need a platform, but if you don't have those guys, you will need one
desperately.

Flynn's decent for my use cases because those very things you say you need
done -> "deploying new assets", "copy/clone job from the VCS", and "running a
single script to deploy/activate everything" are wicked simple and supported
right out of the box securely, so instead of handing over the SSH keys to a
developer, I can give them the Flynn key and they don't get access to the
underlying infrastructure, just a basic API for doing development tasks.

Additionally, the backup functionality allows me to backup a single app or a
full cluster and upgrade our infrastructure as it's running.

If you've never looked into Docker, or Flynn, take this post, set aside a $60
budget for DigitalOcean and try it out, I can't guarantee it will be perfect
for you, but it's perfect for me. All hail /u/_lmars and /u/titanous.

~~~
Silhouette
I've looked into Docker, and various other devops-style tools, in recent
years. I guess I just don't see what they usefully achieve _in the context I
'm talking about_.

I mean, for all of the projects I'm thinking of here, we already have full
version control of all of our assets. We can deploy an update in about upload
time plus a few seconds with a single command, and roll it back with a single
command as well. Security updates are typically deployed directly from their
own upstream repos, again normally with a single command after any due
diligence we feel is necessary. Installing the OS and other foundation
packages was more substantial in some cases, but mostly because of the various
non-standard things any given project might be using in addition to the
routine OS + web + DB stack, and we'd surely have to set that up regardless of
whether we were making some sort of master filesystem image to clone directly
or some sort of image for use in a container system. And we typically run test
infrastructure identical to production in both hardware and software, with
deployment to that rather than a live system requiring just a couple of simple
changes to configuration.

So given the above, how would introducing an extra layer of infrastructure and
a bunch more tools really help us? I'd be happy to look into these things in
more detail if someone can tell me what I should be looking for and what
problem it solves that we don't even realise we have, but none of the (quite a
few) resources I've read/watched about them over the past several years has
suggested a compelling advantage for smaller, statically hosted
infrastructure. Again, the advantages seem pretty clear for people managing,
say, hundreds of systems of various types that are spun up on demand and
discarded when no longer useful on some sort of Cloud platform, but eliq's
original question and my follow-up relate to a different sort of environment.

~~~
WrtCdEvrydy
Flynn's key selling point is easy setup and load-balancing.

You should try it out yourself, and see if it eases some of your pain points.

It also includes rollback of code (right from the UI, and every ENV variable
change / worker change is a new release) and continuous deployment options
(ensuring your code isn't broken when deploying a new version and deploying
while ensuring your service never goes offline)

~~~
Silhouette
_You should try it out yourself, and see if it eases some of your pain
points._

Could you suggest any good places to start, please? I haven't looked at Flynn
specifically before, but I did take a look at its website on your
recommendation. Unfortunately, the videos and explanatory text I've found so
far don't seem to be any better than others I've seen before in this area:
plenty of buzzwords, but little real substance or context to help someone
understand what the tool offers or how it works. Are there some better sources
I could look into?

~~~
WrtCdEvrydy
Go through the basics, and then install the flynn-cli and create a cluster.

You may additionally want to purchase a cheap domain ($0.99 special is enough,
doesn't need to be fancy).

Docs: [https://flynn.io/docs/basics](https://flynn.io/docs/basics)
[https://flynn.io/docs/installation](https://flynn.io/docs/installation)

~~~
Silhouette
Thanks, but those are exactly the pages I was looking at before and didn't
find very helpful.

Some examples of what got in the way for me, as someone totally new to Flynn:

I don't know what a "Flynn cluster" is, but it's obviously a central concept
and mentioned almost immediately on both of those pages.

There are lots of impressive-looking commands doing impressive-looking things
quickly in the videos, but there's not a single mention of where you're
actually running them, where any other systems they are affecting came from,
or what they're actually doing for that matter.

In fact, _nothing on those pages tells me what Flynn does_. I spent probably
an hour or so exploring their site after we swapped posts the other day, but
it was an exercise in frustration and I'm still none the wiser about what its
scope or feature set actually is. (For comparison, that's longer than the
total time it took me to write all the scripts we use to do the automation for
one of those projects I mentioned before!)

Anyway, I don't want to take any more of your time on this, so I'll stop
there. Thanks for trying to help, even if it didn't work out in the end.

------
meddlepal
Seriously why is this anywhere near the top of the page? It's basically two to
three sentences about each and barely scratches the surface of each
technology.

------
dabeeeenster
We used Flynn over the last 12 months (tech agency, lots of fairly small
projects) but gave it up in favour of Dokku. Every single one of our nodes
eventually died, and we were unable to recover them. They chewed up disk space
continuously and eventually ran out of disk and then were not recoverable.
Other times they just died and would not restart all the required services.
This was not even as part of a cluster; just single nodes. We raised issues
and messaged on IRC but nothing happened. We really wanted to like it, but
that was our experience. There's a lot of magic going on behind the scenes,
which makes it really hard to even start to try and resuscitate a dead node.
Sorry Flynn!

We moved to Dokku and are really happy with it. It's not H/A but it's mega
simple and it works really well. It's basically just Docker, buildpacks and
the networking to wire everything together. The plugins also mean it's more
useful than Flynn - if you need something like ElasticSearch you just install
the plugin. Ditto LetsEncrypt etc etc.

------
jimaek
I am personally using Rancher with Docker to deploy my apps. I like the web-ui
and 1.6.1 is stable to use in production.

~~~
drdaeman
Is it? 1.1.x looked nice when I had evaluated it (especially the Web UI), but
was a real bugs nest in the long run. Had issues with almost every component.
Usually found matching bugs that were already filed and had the workarounds
described, but it left the bad aftertaste.

Finally, DNS component had nearly malfunctioned and I've switched to Compose
with some WeaveNet and semi-manual (re)scheduling. Lost some useful
functionality (and the UI) but at least the switch fixed things and it works.
Wanted to use Swarm for fully automated scheduling without a DIY external
watchdog system, but its networking is extremely limited at the moment (and I
haven't groked how to use it with Weave).

Kubernetes looks nice, but the amount of YAML I'll have to write scares me
away. And I'm not sure I trust all that "we do magic" stuff anymore. The more
complicated system is, the harder it's to figure out what to do when it starts
to fail.

~~~
daniel_levine
There's now [http://ksonnet.heptio.com](http://ksonnet.heptio.com) to make it
much easier to get started with k8s and eliminate the need to write YAML

------
notamy
Somewhat-related question:

I have a cluster of ~6 machines that I'm using to play with containers and
container orchestration tools and so on. I specifically want to be able to

1\. Have a non-cli interface for management, and especially not have to write
a bunch of YML files myself, and

2\. Be​ able to specify exactly which machine in the cluster a specific
container ends up on.

Something like Openshift Origin seems to be the way to go, but is there
another option worth thinking about?

~~~
tyingq
#2 would probably be a combination of "stateful sets" and "label selectors" in
kubernetes.

Generally, though, you're fighting the high level idea of these tools by
trying to tie to specific hardware.

~~~
jacques_chester
I guess it should've occurred to me to ask why notamy wants that. HA?
Hardware-specific differences maybe? Those are the uses I know of.

~~~
notamy
It's an issue of hardware differences + some of the software not being
designed to be clustered Kubernetes/etc.-style (and it's not mine so I can't
just rewrite it).

~~~
jacques_chester
OK, that makes sense. But why use a platform that abstracts physical layout
when you already have a physical layout in mind?

~~~
notamy
This isn't really something that I have experience with, hence asking; my
issue is mainly not really knowing the tools available, therefore I'm looking
in the wrong place.

~~~
jacques_chester
Gotcha! I hope I was helpful.

Normally in these circumstances I'd recommend BOSH but, off the top of my
head, I don't know what kind of affinity can be defined. So it might be
another rabbit hole.

------
sandGorgon
Docker Swarm is the simplest setup for a fairly sophisticated PAAS. It works
of a docker compose file that will carry port binding, network
configuration,etc. You just run "docker stack deploy". Adding multiple nodes
to a cluster is a one line command.

~~~
pryelluw
Do you mind sharing more from the standpoint of security? What can a user
expect to maintain, do, or setup to be secure (as possible)?

~~~
sandGorgon
docker swarm has TLS overlay network support built in. So it is secure by
construction. Docker Secrets is built-in
([https://docs.docker.com/engine/swarm/secrets/](https://docs.docker.com/engine/swarm/secrets/))
and is pretty cool.

Not sure what else are you looking at

~~~
pryelluw
Thanks. Thats a solid start. :)

------
technologyvault
One of the engineers from my company, Nanobox (a micro-PaaS), compares a few
of these tools and others in context of how well they cover the development to
production workflow: [https://content.nanobox.io/development-to-production-
workflo...](https://content.nanobox.io/development-to-production-workflow-
coverage-solution-comparison/)

Tools included on this chart include: \- Heroku \- Kubernetes \- Flynn \-
Nanobox \- Openshift \- Dokku \- Rancher \- Convox \- Mesosphere \- Hyper.sh

------
jdwyah
Surprised not to see an ECS shoutout yet. I'm curious what the rap is on that
that makes people walk right on past the AWS default option.

I've been really happy with ECS. Tons works out of the box. Last piece for me
was figuring out a good way to do cron, but I just figured that out.
[http://blog.ratelim.it/blog/cron-jobs-on-amazon-ecs-with-
dat...](http://blog.ratelim.it/blog/cron-jobs-on-amazon-ecs-with-datapipeline)

~~~
mwarkentin
Glad you figured it out, but they just announced native scheduled tasks:
[http://docs.aws.amazon.com/AmazonECS/latest/developerguide/s...](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduled_tasks.html)

~~~
jdwyah
Hah! Le sigh. That's hilarious. Thanks for the pointer, can't believe I missed
that.

------
richardknop
Couple additional PaaS that could be on the list as well: cloudfoundry, deus,
tsuru, openshift (although that's mostly covered by kubernetes).

------
alixaxel
Flynn looks very interesting nowadays (it's been a while since I last checked
it), I specially like the fact that they are trying to handle stateful
(databases for instance) environments too. But after reading through the
website and the Go code, I'm still a bit lost regarding what kind of
guarantees/architecture this implies. Has anyone had any experience deploying
database appliances with this?

------
jacques_chester
A first note, this is not like-for-like. Dokku, Flynn and Deis fit the PaaS
spot more meaningfully. Kubernetes and Swarm are sometimes called "Container
as a Service" \-- you still need a bunch of work to select, install and
configure features that PaaS users take for granted.

Missing from the list: Cloud Foundry. And OpenShift! Give Red Hat some respect
here for making Kubernetes into a platform. And DCOS, which I think got its
thunder stolen by Kubernetes.

But I mostly know Cloud Foundry, because it's my day job to keep making it
better.

It's the most mature out of all of these, has been soak tested to 250,000
running applications[0], can be deployed to any major IaaS or bare metal,
comes with routing, logging, service injection, healing, no-downtime upgrading
and I forget what other headlines I usually pick out of the several hundred
features it now includes.

The Cloud Foundry Foundation includes Pivotal (my employer), IBM, SAP, Google,
DellEMC, VMWare, Cisco, Suse and those are just the fancy tech names.

The reason you don't hear about it is that we and our partners have focused on
competing for enterprise sales. It's not how you get publicity on HN, but it
does mean that PivotalCF -- our commercial distribution -- has the fastest-
growing sales of an opensource-based product _in history_. And sales are
_still_ zooming up and to the right.

The nice thing is that we and other partners get very broad, specific feedback
from customers who are _already_ at massive scale and who expect utter, non-
negotiable reliability. We run a public service ourselves (Pivotal Web
Services), SAP have HANA Cloud, IBM runs BlueMix.

We and other Cloud Foundry contributors have had the benefit of that
dogfooding and feedback for longer than any other container-based platform in
existence. And it turns out, there are so many things that can go wrong. _So_
many. It's crazy.

Lastly, Cloud Foundry teams have massive investments in project automation.
This gives us two capabilities: one is that we can roll out complete rebuilds
of the whole platform within hours of an upstream CVE patch. Users can then
apply these fixes to their platform live, in-place, without any running apps
noticing that it has occurred. BOSH[1] will roll out the deployment with
canaries, and Diego[2] will relaunch containers as they are reaped during
upgrade.

The second capability is that we are confident in making very deep, very
aggressive changes if it proves necessary, because we have tests upon tests
upon mountains of more tests. And again: nobody notices, except that their
platform gets faster or becomes reliable under even more extreme
circumstances.

If I sound like an utterly biased fan, it's because I am an utterly biased
fan. I've watched this thing evolve up-close for years. It is _amazing_.

We build installable superpowers.

Disclosure: If it's not obvious, I work for Pivotal on Cloud Foundry.

[0] [https://content.pivotal.io/blog/250k-containers-in-
productio...](https://content.pivotal.io/blog/250k-containers-in-production-a-
real-test-for-the-real-world)

[1] [http://bosh.io/](http://bosh.io/)

[2] [https://github.com/cloudfoundry/diego-design-
notes](https://github.com/cloudfoundry/diego-design-notes)

Minor edits for style.

~~~
Veratyr
Is it still the case that CF requires a ton of overhead to get running? Last I
checked there was no easy way to set up CF without:

\- Running on a cloud

\- Spinning up ~8 different servers, each with ~2GB of memory iirc

Kinda a lot of overhead if you're just running a few dev environments like I
was.

~~~
jacques_chester
I get this question a lot.

PCFDev[0] if you just want a dev environment to tinker with. You can also
tinker with cflocal[1] (unsupported) or BOSH-lite[2] (somewhat-supported) if
you're looking to practice ops as well.

A lot of Cloud Foundry teams do integration testing with BOSH-lite on large
single VMs, because it essentially presents BOSH with a "VM" interface that
spins up nested containers instead.

You can, in theory, deploy however few or however many real VMs, or bare metal
servers, as you wish, by editing a BOSH manifest.

By default the cf-deployment[3] manifest uses something like 23 VMs of varying
sizes. That's because the default manifest is intended for production use with
multi-AZ HA as the minimum level of reliability. If you don't need that, one
of the alternatives I noted will get you what you want.

[0] [https://pivotal.io/pcf-dev](https://pivotal.io/pcf-dev)

[1] [https://github.com/sclevine/cflocal](https://github.com/sclevine/cflocal)

[2] [https://github.com/cloudfoundry/bosh-
lite](https://github.com/cloudfoundry/bosh-lite)

[3] [https://github.com/cloudfoundry/cf-
deployment](https://github.com/cloudfoundry/cf-deployment)

------
tylerflint
Some of those aren't PaaS, but container orchestration frameworks.

If you're reading this list of comparisons out of genuine interest, I would
suggest also looking into Nanobox: [https://nanobox.io](https://nanobox.io)

------
neocodesoftware
help me imagine using flynn in production - these seeem like deal breakers

upgrade flynn requires back up and restore

backups are full backups only no incremental

recommended filesystem is s3 azure or google since

postgres is not reliable

~~~
aisofteng
>postgres is not reliable

That will require a pretty strong argument.

------
johnsmith21006
You want to invest into what will win including ones own skills. K8 is the
winner, IMO. Writing is already on the wall.

