
Dive – A tool for exploring each layer in a Docker image - boyter
https://github.com/wagoodman/dive
======
cfontes
This is amazing, thanks for sharing it! A very useful feature that I always
missed but never actually tough about.

I've had a hard time understanding the slices on the beginning will be very
useful for troubleshooting bad images also while optimizing image sizes.

Can probably help to understand problems like the one bug where the image size
got almost a 2x increase after a CHOWN, because the slice is added again with
different owner ( I think ;) ) would be so much easier to see with this.

Ref for the old bug
[https://github.com/moby/moby/issues/5505](https://github.com/moby/moby/issues/5505)

~~~
ravedave5
We added a lib to a java image and the size went up by about 4x the library
size, I spent a few hours trying to track it down to no avail. Hopefully this
will help pin down the issue.

~~~
dvdbloc
Let us know if it’s a relatable issue. Might save someone else a headache in
the future.

------
etaioinshrdlu
Project idea: a Google for docker images.

Search for files by hash within all public docker images.

Find images that contain a certain piece of code.

Reverse engineering Dockerfiles for those that were built without.

~~~
bacongobbler
The latter can be easily done in a few hours by inspecting the image layer
metadata.

Every layer understands the command it was run in the Dockerfile to create
itself. Just look at `docker history` and have a look at the "CREATED BY"
field for human-readable output of the layer metadata, or depending on your
graph driver have a look in
/var/lib/docker/image/overlay2/imagedb/content/sha256. From there you can
reverse-engineer a Dockerfile.

For layers that were not built using `docker build` (e.g. `docker commit`,
OCI-compatible image builders), re-creating the exact command that generated
that layer is much harder to do. The only information most tools will give you
might just be the diff itself.

~~~
edraferi
> Every layer understands the command it was run in the Dockerfile to create
> itself

How reliable is this? Can it be modified after creation by a malicious party?

That is, if I get a wild docker image, can I trust the results of `docker
history`?

------
orliesaurus
Can someone well versed in Docker explain in their own words what this does
and why it is useful? Thanks in advance! I'm definitely not a docker
poweruser...other than build start and stopping..

~~~
IMTDb
A docker image is built in layers that stack on top of each other. Think of
each layer as a commit in a git repository.

This project takes an image and shows the list of layers (=commits) and for
each layer (=commit) allow you to see what was changed (=the diff).

A bit like a git log for a docker image.

~~~
djsumdog
If OP looks up some Dockerfiles on docker hub, this will make more sense. Look
at this mysql file for instance:

[https://github.com/docker-
library/mysql/blob/696fc899126ae00...](https://github.com/docker-
library/mysql/blob/696fc899126ae00771b5d87bdadae836e704ae7d/8.0/Dockerfile)

If two Dockerfiles use debain:stretch-slim as their base, docker won't
download both base images twice. Each command in the Dockerfile creates an
image. If you copy a Dockerfile and just change the last line and build them
in order, the 2nd build will run a lot faster and you'll see in the output it
pulls out "cached images" since it's already run those commands (docker build
--no-cache always forced a full build).

If you start making Dockerfiles and playing around with it, it will start to
make more sense.

------
vachi
As others have noted here, this is why we use Nix
[https://nixos.org/](https://nixos.org/)

------
maxbendick
This is great! Docker feels imperative to me, and using this could help
mitigate that.

Also looking into Nix to help my container workflow more declarative.

~~~
avip
But a Dockerfile _is_ imperative. It just lists, in order, the steps needed to
build an image.

~~~
maxbendick
It's declarative in the sense that the instructions are set in text. But what
goes inside a Dockerfile might be imperative, like running apt-get to install
packages (i.e., running steps in order).

~~~
iamdave
_But what goes inside a Dockerfile might be imperative, like running apt-get
to install packages (i.e., running steps in order)_

But...that too is done via the Dockerfile, via the RUN command to execute
commands inside the container at run time. Or alternatively using the
ENTRYPOINT command.

------
jugg1es
I have been looking for something like this for like 2 years. This will be
most useful for experienced devs new to Docker that don't quite understand
what's going on and haven't had time to read the specs on how docker works.

------
bepvte
Is there a smart way to make more layers than splitting everything up into
modules and ADDing each one? Why doesnt docker have delta transfers yet?

~~~
orf
Layers _are_ delta transfers. Maybe not as smart as they could be, but there
is a lot of under-appreciated complexity hiding here.

~~~
bepvte
layers are not delta transfers in my opinion, they are just new copies or
removed files. its like saying sftp is delta transfer because you don't have
to send your entire disk image every time. I love layers and dockers very nice
reusing of base layers and such but i feel like delta transfers would
dramatically improve my workflow and upload times.

~~~
Confiks
> its like saying sftp is delta transfer because you don't have to send your
> entire disk image every time

I see your point, but the comparison to sftp seems really off. Overlay2 [1]
indeed works on whole file differences, just as sftp would do.

Additionally, I don't think making actual byte-level deltas would yield you
much improvement on container size. It would also increase access time unless
you keep a cache that doubles storage requirements per image. Dockerfiles
primarily add files rather than change them, and actual changed files are
often small.

[1] [https://docs.docker.com/storage/storagedriver/overlayfs-
driv...](https://docs.docker.com/storage/storagedriver/overlayfs-driver/)

------
tmkc13
Nice!! have been waiting for something like this for a while. Gonna try this
for sure.

~~~
manneshiva
just curious, how do you intend to use this? I mean, what problem does
exploring each layer in a Docker solve for you?

~~~
orf
"why does this small change increase the size of my image by 500mb"

~~~
simonw
A few people pointed me to this dive tool after I wrote up these notes on
shrinking Docker images: [https://simonwillison.net/2018/Nov/19/smaller-
python-docker-...](https://simonwillison.net/2018/Nov/19/smaller-python-
docker-images/)

------
king_magic
This looks spectacular. This could solve a large chunk of my challenges with
Docker in one swoop. Really excited to use this!

------
brunoqc
Is there a tool to compare 2 or more images to check which layers they have in
common?

~~~
bacongobbler
There's no tool I'm aware of, but with experimental features enabled in the
Docker CLI you can use `docker manifest inspect` on each image manifest and
diff the content. e.g. `docker manifest inspect ubuntu:latest`.

This requires that you have logged in to the registry, pushed the image to the
registry and have `docker pull` rights for the image. You could also run a
registry locally, push your image there and inspect your registry's storage
db. There's just no CLI command to do that.

~~~
brunoqc
Thanks!

------
bloopernova
Very, very useful. Thank you for sharing this!

------
Exuma
dude this is sick

------
wayoverthecloud
This is amazing!

------
vkaku
Good tool!

------
kureikain
Off topic but anyone notices lot of issue with Dockerhub recently? Such as
image tag disappear :( randomly fail to push or take a very long time to pull

