
Defence Against the Docker Arts - codefinger
https://blog.heroku.com/defence-against-the-docker-arts
======
zapita
Here’s a little known fact: “docker build” can trivially be extended to build
buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles
are just the default frontend. There’s already a buildpack frontend in the
community repo, and it works great. Writing your own frontend is real
straightforward.

Honestly after years of stagnation, the most exciting work on container
building is now coming out of Docker. Buildkit is amazing, a real hidden gem.

See [https://github.com/moby/buildkit](https://github.com/moby/buildkit)

~~~
meowface
My favorite Docker BuildKit feature is SSH agent forwarding. Add
"\--mount=type=ssh" after RUN commands in Dockerfiles and the command will use
your host machine's SSH agent. I've been able to greatly simplify a lot of
Dockerfiles and CI build processes using it.

There's a good introduction here: [https://medium.com/@tonistiigi/build-
secrets-and-ssh-forward...](https://medium.com/@tonistiigi/build-secrets-and-
ssh-forwarding-in-docker-18-09-ae8161d066)

~~~
bsilvereagle
This feature currently does not work with the OS X ssh agent:

[https://github.com/docker/for-mac/issues/410](https://github.com/docker/for-
mac/issues/410)

~~~
auvrw
ssh access at build time via buildkit works on os x

[https://medium.com/@tonistiigi/build-secrets-and-ssh-
forward...](https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-
in-docker-18-09-ae8161d066)

the link you've supplied regards ssh access when running docker images, not
when building them.

... i did notice that for lots of downloads, it is somewhat slower, or perhaps
more prone to lag, than authentication from inside the running image.

------
jrockway
I've been very close to creating something like this internally. It is easy to
write a Dockerfile that produces a compact and optimal container. But it's the
same lines of code over and over again. Anything we have that's a static site
looks like:

    
    
        FROM node:10 AS build
        WORKDIR /foo
        COPY . .
        RUN npm i
        RUN npx webpack
    
        FROM nginx:whatever
        COPY nginx.conf /etc/nginx/config.d/
        COPY --from=build /foo/dist /srv
        ...
    

It's fine when you have one. Annoying when you have a couple. This isn't code
that needs to be checked into the repository and updated. It needs to just
work.

The other thing I'd like to see is the ability to output multiple containers
from one Dockerfile. There is so much wasted work where I have webpack stuff
and a go application that run in separate containers but are built together.
There is one Dockerfile like the above to build the static part. There is
another to build the go binary and copy it to an almost-pristine alpine
container (including dumb-init, cacerts, tzdata, and grpc_health_probe). I
don't understand why I have to have two Docker files to do that.

~~~
chusk3
You don't have to have separate dockerfiles. If you use multistage builds, you
can name the specific terminal stages and then invoke them with 'docker build
-t stagename', which will reuse the build cache as you'd expect. I've done
this to export multiple app containers from a monorepo.

~~~
krainboltgreene
Yo, their example has multiple stages.

~~~
rhizome
I think they're talking about intermediates, which wouldn't have to be rebuilt
every time.

------
m463
I'm a novice docker user, but I found Dockerfiles to be probably the most
direct, graspable, important part of docker.

It's one readable text file used to recreate an entire environment. It's sort
of a picture worth a thousand command lines.

That said, I wish there was a way to get rid of all the && stuff, which is
used to avoild writing a layer of the filesytem.

Why not have something like:

    
    
        RUN foo
        RUN bar
        RUN bletch
        LAYER

~~~
nhumrich
The && is just layer squashing and not really needed. You can just use run on
every line. Docker even supports squashing images now, so the && doesn't
matter if you squash.

~~~
jacques_chester
&& doesn't layer squash, it's a trick to avoid the layer commit that RUN
implies.

And squashing comes with a major drawback: you lose layer caching and any hope
of a vaguely-efficient rebuilding process.

------
choeger
You are setting up an entire operating system to install a single Microservice
and just now noticed that you have redundancies?

You could have the same issue by simply trying to rpmbuild your app. No
really, you are just doing packaging. If you want more comfort, look into how
redhat or Debian maintain their packages. They have similar problems and most
likely they have mature solutions.

~~~
viraptor
If you have a single service and will never want more - sure. Once you have
multiple, you actually get benefits if you depend on a lot of native libraries
and external binaries. If you're in that situation, not having to upgrade
every service at the same time when moving to a new base os release is really
convenient.

~~~
Apaec
NixOS offers the same benefits(more actually). Without containers.

------
jtwaleson
I've always felt that buildpacks in Heroku / Cloud Foundry are the way to go
as they offer a higher level of abstraction than Docker files. The resulting
containers are often production ready with good default settings. In docker
you are re-inventing the wheel more often than not.

~~~
imhoguy
With Dockerfiles you can achieve high level abstractions by using proven
images and composing multi-stage build assets. And then you can customize them
with some lower level abstraction by in-line bash or independent scripts.

E.g. look at phusion/baseimage or phusion/passenger - they are production
ready and with good defaults. But you also have an easy way to augument them
with latest ffmepg compiled from sources to support some exotic format for
video conversion worker.

~~~
rhizome
I mean, I'm essentially a beginner (with lots of system experience), and _I
'm_ noticing a lack of intermediates in these comments.

------
Perceptes
I wish this had gone into some more technical detail about what "CNB" does
that is actually better. Most of the article was just rehashing some problems
with Dockerfiles, but the conclusion is just "CNB fixes it!" The one specific
improvement they mention is being able to "rebase" an image without rebuilding
the whole thing, which certainly sounds interesting, but is not explained. How
does it work? What else is CNB other than a wrapper around `docker build`?

~~~
sclevine
The presentation to the CNCF TOC covers some of the technical details:
[https://www.youtube.com/watch?v=uDLa5cc-B0E&feature=youtu.be](https://www.youtube.com/watch?v=uDLa5cc-B0E&feature=youtu.be)

Some key points:

\- CNBs can manipulate images directly on Docker registries without re-
downloading layers from previous builds. The CNB tooling does this by remotely
re-writing image manifests and re-uploading only layers that need to change
(regardless of their order).

\- CNB doesn't require a Docker daemon or `docker build` if it runs on a
container platform like k8s or k8s+knative. The local-workstation CLI (pack)
just uses Docker because it needs local Linux containers on macos/windows.

------
kevinsimper
I like they way they highlight that combining Ruby and nodejs makes for a
complicated Dockerfile, while their example after only includes Ruby and not
nodejs. And do they propose a buildpack called ruby-nodejs, because in many
cases you don't need nodejs in your Ruby app. OMG now buildpack's are a leaky
abstraction!

~~~
codefinger
The Ruby buildpack shown in the example installs Node.js for you. A developer
doesn’t even need to be aware that it’s required to precompile assets.

------
benbristow
I love the fact Heroku have open-sourced their buildpacks. Dokku takes great
use of these and provides a very similar platform to themselves that you can
host yourself (DigitalOcean even provide a base-image that will pre-configure
Dokku for you). Great for personal websites and the like.

If you want to scale in a pinch then it's a case of making some tiny tweaks
and pushing to Heroku instead.

~~~
justinsaccount
I run some things at home using dokku and I switched them all to use
Dockerfiles since the buildpacks are crazy slow.

Not 100% sure if that is the fault of the dokku implementation or buildpacks
in general though. It's all IO related, so I might not even have noticed it
had I had faster disks.

~~~
jacques_chester
Cloud Native Buildpacks are substantially faster in a number of scenarios.
Some of the preliminary Java buildpack changes on the Cloud Foundry side have
updates dropping from minutes to milliseconds.

------
markbnj
It's nice that they've built a tool that understands specific application
contexts and can do the right things to build an efficient image. But imo that
does not make the dockerfile a leaky abstraction.

------
wmf
I just saw a submission on the "demise" of Cloud Foundry (a Heroku-like PaaS)
that's relevant to this discussion: [https://medium.com/@krishnan/lessons-
from-the-demise-of-clou...](https://medium.com/@krishnan/lessons-from-the-
demise-of-cloudfoundry-ed4f77a08a73)

 _Opinionated Platforms Are Risky: The CloudFoundry platform was more
opinionated than some competing platforms in the market. In fact, the biggest
debate between CloudFoundry and its direct competitors was about whether
customers need opinionated platforms or not. CloudFoundry only supported 12
factor applications whereas platforms built on top of Kubernetes could support
both stateful and stateless (12 factor) applications._

If you're building a stateless 12-factor app _and_ there's a buildpack that
does what you want, buildpacks are clearly better than lower-level
Dockerfiles. But there's no buildpack for something like a database and there
probably never will be, so the flexibility of directly building containers
needs to exist.

~~~
ekcasey
Now that Cloud Native Buildpacks build OCI images instead of platform specific
artifacts (slugs or droplets), developers can choose the best build solution
for their problem (buildpacks for 12-factor apps, Dockerfiles for a data
store)and deploy them to the same container orchestrator.

------
bryanrasmussen
It took me a while to get the pun, it doesn't show up in the article anywhere
from what I could fine and I don't actually see how you're defending against
anything here so I wonder - Did you just have this pun sitting around and were
itching to use it somewhere, anywhere?

~~~
akavel
Personally, I still don't get it; would you please care to explain? I'm really
confused with what does the title really try to convey... :/

 _edit:_ Also, clicking to the article, the actual title seems (now?) to be:
_" Turn Your Code into Docker Images with Cloud Native Buildpacks"_, so I'm
even more confused now... o_O

~~~
bryanrasmussen
Defence against the dark arts is a professorial position at Hogwarts School in
the Harry Potter universe - why the pun was used is what I'm confused about,
since without a matching context it loses capacity as a pun.

------
yRetsyM
Gitlab would benefit from this in the context of their autodevops featureset

~~~
sytse
For sure! I was wondering about the same question and asked the team in
[https://www.dropbox.com/s/lcdlpz2l46e1uu1/Screenshot%202019-...](https://www.dropbox.com/s/lcdlpz2l46e1uu1/Screenshot%202019-04-03%2015.15.54.png?dl=0)

~~~
jacques_chester
Please come visit us! We'd love to help.
[https://buildpacks.slack.com](https://buildpacks.slack.com)

------
amai
So instead of a Dockerfile we now need a builder.toml file. In addition you
need a detect and build script: [https://buildpacks.io/docs/create-
buildpack/building-blocks-...](https://buildpacks.io/docs/create-
buildpack/building-blocks-cnb/) Is this really a simplification?

~~~
jacques_chester
> _Is this really a simplification?_

Yes, because for end-user developers, you don't need any files. The files you
mentioned are used by buildpack authors.

------
caust1c
It's not a case of buildpacks vs dockerfiles. They both solve different
problems with different solutions.

Buildpacks fit nicely into the heroku way of doing things. But at any medium
to large sized engineering organization, there's no way they could satisfy the
requirements for even a simple majority of services that using a Dockerfile
provides.

~~~
jacques_chester
Could you give an example of such requirements?

Disclosure: I've worked on Cloud Foundry Buildpacks twice and worked on this
latest effort until recently.

------
onefuncman
This seems written from an outdated perspective of Dockerfiles -- multi-stage
builds landed in Docker 17.05 which are almost a year old and address most of
the concerns in the article...

~~~
jacques_chester
Multi-stage builds address some of the problems, but they still have
drawbacks. For example, at the moment they need a Docker daemon, which is a
non-starter for lots of environments. They also don't help you with fast
updates across a fleet of many applications unless you standardise all your
Dockerfiles. At which point you are, essentially, recreating buildpacks.

------
pugworthy
Vote me down, but oh god that title...

~~~
haditab
I really wanted to hate the article because of the title, but it actually
taught me some neat tricks...

~~~
maimeowmeow
They gave one docker tradeoff/alternative. Didnt come to the article to learn
about heroku, came for docker tips, and tricks.

Assuming most people arnt going to change thier whole platform because they
didnt bother rtfm, or googling.

Yes docker doesnt handle this well, but there is a lot more nuance to that
example.

~~~
hateful
I liked the title, and even though I don't do Ruby, I saved it because of the
tips and tricks.

Flick and swoosh.

------
Polyisoprene
”Mixing operational concerns with application concerns like this results in a
poor tool for developers who just want to write code and ship it as painlessly
as possible.”

Yeah, throw that code over to the ops team. Let them figure stuff out
themselves.

~~~
parasubvert
Docker’s vision was to throw images over to the ops team so they can figure it
out themselves.

Generally speaking I don’t know of many dev teams that enjoy OS and middleware
patching or the nuances of Linux security. Some do of course, but many just
want to code.

Buildpacks’ vision is that the ops team already knows how to install run
production systems and that knowledge is encoded and tweaked in a buildpack.

------
mychael
> creating Cloud Native Buildpacks (CNB), a standard for turning source code
> into Docker images without the need for Dockerfile

This is a solution in search of a problem. Please stop.

~~~
jkaplowitz
I understand and accept that you're fine with Dockerfiles, but your statement
is overbroad: Heroku and Pivotal aren't the only ones in the industry
frustrated at the shortcomings of Dockerfiles. I'm glad alternatives are
growing in number. (Others include Buildah, to be included in RHEL 8; and the
Docker image support for Bazel.)

~~~
jacques_chester
As well as Kaniko, Makisu, Orca and I've genuinely lost track.

Though buildpacks differ from most of these by skipping Dockerfiles
altogether.

~~~
jkaplowitz
Thanks for that list. If I'm not mistaken, Buildah and Bazel skip Dockerfiles
as well; not sure of the three you just named.

~~~
jacques_chester
I forgot Jib! It also skips Dockerfiles.

------
npstr
Lol no. Heroku is way too much magic for my taste.

