I've had a hard time understanding the slices on the beginning will be very useful for troubleshooting bad images also while optimizing image sizes.
Can probably help to understand problems like the one bug where the image size got almost a 2x increase after a CHOWN, because the slice is added again with different owner ( I think ;) ) would be so much easier to see with this.
Ref for the old bug
Search for files by hash within all public docker images.
Find images that contain a certain piece of code.
Reverse engineering Dockerfiles for those that were built without.
Every layer understands the command it was run in the Dockerfile to create itself. Just look at `docker history` and have a look at the "CREATED BY" field for human-readable output of the layer metadata, or depending on your graph driver have a look in /var/lib/docker/image/overlay2/imagedb/content/sha256. From there you can reverse-engineer a Dockerfile.
For layers that were not built using `docker build` (e.g. `docker commit`, OCI-compatible image builders), re-creating the exact command that generated that layer is much harder to do. The only information most tools will give you might just be the diff itself.
How reliable is this? Can it be modified after creation by a malicious party?
That is, if I get a wild docker image, can I trust the results of `docker history`?
New project idea ... Crawl a large set of popular docker images lacking Dockerfiles and attempt to recreate the dockerfile with this technique.
Docker the company could, I imagine, add deep search and browsing as enhancements to their existing Docker Hub.
This project takes an image and shows the list of layers (=commits) and for each layer (=commit) allow you to see what was changed (=the diff).
A bit like a git log for a docker image.
If two Dockerfiles use debain:stretch-slim as their base, docker won't download both base images twice. Each command in the Dockerfile creates an image. If you copy a Dockerfile and just change the last line and build them in order, the 2nd build will run a lot faster and you'll see in the output it pulls out "cached images" since it's already run those commands (docker build --no-cache always forced a full build).
If you start making Dockerfiles and playing around with it, it will start to make more sense.
Also looking into Nix to help my container workflow more declarative.
But...that too is done via the Dockerfile, via the RUN command to execute commands inside the container at run time. Or alternatively using the ENTRYPOINT command.
I see your point, but the comparison to sftp seems really off. Overlay2  indeed works on whole file differences, just as sftp would do.
Additionally, I don't think making actual byte-level deltas would yield you much improvement on container size. It would also increase access time unless you keep a cache that doubles storage requirements per image. Dockerfiles primarily add files rather than change them, and actual changed files are often small.
If you think delta transfers would help, you're saying you have a bunch of data that doesn't change between image builds, but you have to re-upload all that data every time.
This actually sounds like a perfect use-case for image layers. If you can rearrange your Dockerfile to put the steps where files change more rarely before steps that change frequently, docker will automatically cache the earlier steps and not reupload if that layer already exists.
Image layers are at their heart already an implementation of a cache + delta transfer system. (Compared to some other delta systems it trades storage efficiency for computational efficiency.) You can get small delta transfers, but you have to work with docker via your Dockerfile to make it happen.
For example, that could mean: moving steps up or down the Dockerfile so more frequently changed layers come later; splitting a step into two steps to exploit the previous; making a step deterministic so you don't have to force rebuild; use multi-stage builds to make all of these easier to implement.
This requires that you have logged in to the registry, pushed the image to the registry and have `docker pull` rights for the image. You could also run a registry locally, push your image there and inspect your registry's storage db. There's just no CLI command to do that.
In this way it's not just showing you a score... the score doesn't help you make the image any better. This is why letting the user explore the layers is good, to help discover and explain why there is an inefficiency (not just that there is one).