
Show HN: Bocker – Docker implemented in 100 lines of bash - p8952
https://github.com/p8952/bocker
======
tobbyb
Just playing with this in a VM with an attached btrfs volume, a complete
revelation. 96 lines! And it's actually pretty functional. This takes keeping
it simple to a whole new level.

The Wheezy image I use with LXC worked well enough, the minimal alpine image
not so well, apk complaining about its database.

User name spaces support would be nice, then we can play with unprivileged
containers.

And Overlayfs would be a nifty alternative to btrfs, it's in kernel 3.18, and
4.04 adds support for multiple lower layers. But this btrfs implementation is
cool too. Cgroups support will be somewhere on that list too.

Cgroups and namespaces is in the kernel. General Linux ecosystem for
networking, storage and distributed systems is already extensive. The
possibilities are endless.

So now LXC, Docker, Rkt and Nspawn have Bocker for company.

~~~
cyansmoker
Yes they do. Nice to learn about even more projects than I thought existed.

------
vezzy-fnord
Using btrfs subvolumes as the image format, that's a nice touch. On the same
road as the hypothetical systemd packaging system (not that I'm very
enthusiastic about that).

The network, PID and mount namespaces are the ones unshared, plus a private
/proc.

I like tools like this because they're reality checks on how the basics of
Linux containers are just a few essential system calls, and particularly that
they're limited.

~~~
p8952
Thank you.

One of the things I found really interesting here is how much could be done
with just basic userland tools, and how old some of those tools are.

Docker was released in 2013, but support for kernel namespacing has been
around since ~2007. That's quite a long time for such a great feature to go
mainstream.

~~~
hosh
Yes. The basic technology has been there for a while. And the Docker source
code has some eyebrow-raising parts to it.

However, I've stated this in other threads: Docker isn't about containment.
It's really about the packaging system. I don't think this technology demo
gets that.

~~~
p8952
I agree 100%. While the tech in Docker is great, it alone can't be easily
monetized.

The real value comes from things like DockerHub, and getting people to buy
into the whole ecosystem.

~~~
vezzy-fnord
Which is probably why people were concerned about Docker's expansion into the
clustering and orchestration markets, even if from a business perspective
those are their only real holdouts to avoid commodification. The base Docker
is easy to replace if the project goes out of hand, the services around it are
trickier.

~~~
hosh
There are several tech streams converging there. A bigger chunk is the space
that Mesos and Kubernetes occupy.

And to be fair, I suspect the Docker folks were thinking less about clustering
and orchestration so much as: (1) clustering and orchestration still sucks;
(2) people want as good of an experience using docker as they do when spanning
across multiple nodes; (3) let's make clustering and orchestration less sucky
and use the 'Docker Way'[1]

[1] 'Docker Way' is a pointer to the fuzzy, difficult-to-verbalize thing that
Docker enables, namely in packaging.

~~~
vezzy-fnord
_clustering and orchestration still sucks_

I don't see that changing until we get proper single-system imaging, location
transparency, process and IPC migration, process checkpointing and RPC-based
servers for representing network and local resources as objects (be they file-
based or other) in our mainstream systems.

These things only really caught on in the HPC and scientific computing spaces,
where you've had distributed task managers and workload balancers like
HTCondor and MOSIX for decades. They've also been research interests in
systems like Amoeba and Sprite, but sans that, not much.

The likes of Mesos, the Mesosphere ecosystem with Marathon and Chronos, and
Docker Swarm bring only the primitive parts of the whole picture. Some other
stuff they can half-ass by (ab)using file system features like subvolumes, but
overall I don't see them improving on all the suck.

~~~
nitrogen
I ran an OpenMOSIX cluster as a hobby. The alternative was Beowulf (the meme
of the day was "Imagine a Beowulf cluster of these things").

It seems the mainstream server industry has moved to more isolation rather
than more interconnectedness, which is probably better for most public-facing
systems.

~~~
vezzy-fnord
Isolation is orthogonal to what I listed. MOSIX has decent sandboxing. I don't
know about Linux-PMI or OpenMOSIX, though. They died off years ago anyway.

------
tlrobinson
Here's a proof-of-concept implementation of "docker pull" in bash (YMMV, I
think it has broken since I wrote it):
[https://gist.github.com/tlrobinson/c85dca269f4405ad4201](https://gist.github.com/tlrobinson/c85dca269f4405ad4201)

~~~
p8952
Looks good!

I'd marked 'docker pull' as out of scope because I thought it would be fairly
hard to interface with their API from bash, looks like I was wrong.

~~~
tlrobinson
The only hard bit is parsing the JSON, which I use jq for
([http://stedolan.github.io/jq/](http://stedolan.github.io/jq/))

I think the v2 API requires hitting an auth endpoint too.

------
anh79
When I first saw the HN title, I was so stunning. Weird, it's not my tool
[https://github.com/icy/bocker](https://github.com/icy/bocker) ;)

The author of "bocker" (not my bocker) has a great idea. I would learn from
the script. Docker is not magic anymore.!!

------
eatitraw
Now we need bokku:
[https://github.com/progrium/dokku](https://github.com/progrium/dokku)

~~~
josegonzalez
Dokku written in bash?

(I am one the maintainers of dokku, which is written in bash).

~~~
yodasan
I think he's suggesting creating a similar project to dokku, but instead of
using Docker, use Bocker.

~~~
josegonzalez
If installing via apt-get, that means creating a debian package that provides
lxc-docker-1.6.2 (docker 1.7 broke backwards compatibility in output of docker
ps). Otherwise, just patching the
[Makefile]([https://github.com/progrium/dokku/blob/master/Makefile#L85](https://github.com/progrium/dokku/blob/master/Makefile#L85))
to check for bocker (linked to `/usr/local/bin/docker` of course) would
suffice.

Though I don't think that is what he is asking for.

------
tzm
$10M per LoC

------
WD-42
Nice work. Great to see the advanced features of BTRFS put to use.

~~~
p8952
Thank you.

~~~
hndl
Very interesting work, p8952. Can you elaborate more on what features made
btrfs a good fit for the project?

------
asdfaoeu
There's also
[https://github.com/docker/dockerlite](https://github.com/docker/dockerlite) .
But not sure how current it is.

------
sandGorgon
not sure if you ever saw this -
[https://www.phoronix.com/scan.php?page=news_item&px=CoreOS-B...](https://www.phoronix.com/scan.php?page=news_item&px=CoreOS-
Btrfs-To-EXT4-OverlayFS)

it might be interesting to see a version of your script using overlayfs

~~~
p8952
Interesting, it was my understanding that Docker was going in the other
direction. Moving off AuFS towards Btrfs due to issues getting AuFS patches
into the mainline kernel.

I'll have to look info CoreOS's reasons for going with Ext4.

~~~
sandGorgon
Docker 1.6+ works with overlayfs on my debian box (in production). I thought
AUFS was already deprecated

------
arianvanp
Add support for GPG-signed `btrfs pull` and `btrfs push` and I'm totally sold!
I've been working on something similar to this but on top of systemd-nspawn,
which already does some stuff for you.

systemd-nspawn is nice because I run systemd in all my containers and thus
allows me to easily do logging etc.

I don't really dig the docker-microservices mantra that much. I just use them
as glorified VMs I guess.

(And yes, you should run an init system in your containers [0])

[0] - [https://blog.phusion.nl/2015/01/20/docker-and-the-
pid-1-zomb...](https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-
reaping-problem/)

------
Bocker
Approved.

------
joeevans1000
The emperor is wearing fewer clothes.

------
jenscow
That script is the best description of Docker I've read.

------
spydum
I'm missing something, what actually "executes" the command here echo "$2" >
"$btrfs_path/$uuid/$uuid.cmd"

Is something watching for .cmd? Is this some behavior of util-linux (for
which, my few seconds couldn't find solid documentation on)?

~~~
inopinatus
If my dry read of the code is correct, that line is for the "bocker ps"
routine to be able to print the command later.

The run command itself is executed next, inside of ip netns exec.

~~~
spydum
Ahhh, no.. I was looking at line 57 and somehow missed the actual call to $2
on 59. Clear as a bell now

------
e12e
Nice. As I just started playing with zfsonlinux, I'm tempted to "port" this
from btrfs to zfs... Should allow for migrating "images"/snapshots with zfs
send...

------
kzisme
As an entry level developer - how does someone go about re-writing "x" in 100
lines of "x"?

Is there a certain process that goes into developing something like this, and
why is this a popular thing to do? (writing an existing software in lesser
lines)

~~~
MoOmer
1) Knowledge of the space is good, but can be learned while implementing.
You'll need to know the language pretty well, in order to keep the line count
down.

2) It's fun

------
cyansmoker
Hey, I just submitted a pull request for the "exec" command :)

------
agumonkey
Persistent data structure FTW.

------
amrit_b
This is incredible.

------
logicrime
Holy crap! I've been keeping up with the hype, yet having never used Docker
and never needed it yet I can't help but become more skeptical now that I know
that it's features aren't more complex than a little bit of bash.

People give bash a hard time, but things like this really give me that warm,
fuzzy feeling.

esac4lyfe

~~~
welly
Why would it need to be more complex than "a little bit of bash" if it fulfils
the task intended?

Surely that is better than a convoluted, 1,000,000,000 LoC application that no
one outside a handful of core developers understands? Right?

~~~
antsar
I think you and logicrime are saying the same thing.

------
sklogic
I honestly thought docker was just a little script when I saw it first,
judging on the functionality.

------
BonoboBoner
You published it? You should have gotten a "micro-docker"-like hashtag trend
going and then pitch your idea to VCs. The main "Lighter than Docker" startup
would be valued at around 5-7 billion right now.

~~~
tritium
Just a quick interjection:

Something that one smart developer can do in 100 lines on any interpretter is
never worth billions of dollars.

Someone who can do it, on the other hand, is certainly worth hundreds of
thousands of dollars, annually.

Meanwhile, so many people continue to marvel at what can be done with an
interpretter and a turing-complete language. Yet, the last thing we need is
yet another turing complete language.

Unfortunately, the problem with turing machines, virtual or otherwise, is that
they're so good at faihfully reduplicating themselves...

~~~
mpdehaan2
"Something that one smart developer can do in 100 lines on any interpretter is
never worth billions of dollars."

100 lines maybe not, but docker is pretty lightweight glue riding on existing
technology that did most of the heavy lifting. The valuation IS lopsided
because it sort of did a "name grab" around the underpinning technology (sort
of like "AJAX" was "XMLHttpRequest") and packaged it in a way that made it a
more useful (some) and more importantly, talked about in a way people could
understand, mostly giving it a name and describing some common practices was
what happened.

The original idea was definitely clever though, and it is getting people who
didn't adopt immutable systems before to start to understand immutable
systems, even if the future is not actually going to be Docker. Yet, Docker is
getting the press versus the higher level software that needs to exist to
_drive_ Docker.

While this makes it very hard for other projects to get VC attention, I think
that's maybe a good thing for them if they don't know it - you want to
bootstrap if you can anyway, and I hope many of them do.

While this isn't a super robust implementation or anything, I think it's
important to show that Docker is more or less glue around existing concepts,
and that there's still room to make better things.

Don't get me wrong, immutable systems are GREAT. Docker deserves points for
getting people to understand them, and the ideas of private registries and
Dockerfiles (though also not original) are good parts. Microservices? Not
really neccessary for most people, that's more of a human problem. It sort of
conflates immutable with microservices and makes you hop out of the norm to do
the basics, but ok, sure, that's somewhat like a celebrity actor advocating
for environmental causes. Still a good thing.

But is it a giant leap forward? Not as much as people think, compared to AMI
builds, and you see folks running Docker on top of EC2 in cases (yes, they
boot faster - but AMIs gave you immutable and things like Packer allowed
redistributable images; stacking images is kind of sketchy if you don't
control them all). But it's enough to make people use them, and that's a win,
and someday the management software for it may be smart enough to make it feel
as easy and powerful as that (fingers crossed for ECS?).

The 100 liner at least has the advantage of reminding people when people say
"Docker is great" they mostly mean "I like this immutable systems thing and
describing systems in text", and the other properties of Docker, and reminds
people that if they can do better and try to make a better thing, they should
also still try.

------
vacri
Interesting work. And 10% of those lines are simply closing braces which can
be collapsed to the previous line, and half a dozen lines can be reclaimed
from the help function...

------
SFjulie1
I thought puppet/chef were the pit of the devops ridicule. Then I not only saw
this, but also positive reactions to a readable code in which you have :

    
    
        echo 'nameserver 8.8.8.8' > "$btrfs_path/$uuid"/etc/resolv.conf
    

This is wrong on so many level that I don't know where to begin with.

~~~
Gigablah
Can you share why you think puppet/chef are the "pit of the devops ridicule"?

~~~
dsmithatx
Having implemented it across 10k+ servers in 12 datacenters I'd say he means
Puppet is overly complex for what provides. Kind of like Docker. I think
people equate Chef to the same over-complexity.

Having now used Salt and starting to play with Ansible I'm growing an extreme
dislike for Puppet and the weeks of my life I can never recover dealing with
things that Salt has made so much easier.

------
pekk
Wow, Docker in 100 lines! It runs as root? Oh. It is written in bash? Oh. It
needs a ton of manual setup? Oh. It doesn't actually implement the package
format which is most of the point of Docker? Oh. So is it that easy to
reimplement Docker? Despite the obvious snarky intent, it appears not.

~~~
alwaysagopher
@Shykes, is that your alter-ego account?

~~~
pekk
No, that's a dumb joke. Actually I don't really even use Docker personally.

HN's standard middlebrow dismissal of Docker is to claim that it's nothing
except LXC, which really misses the point entirely. But at least it lets
people pretend they are smarter, which is what really counts.

~~~
Spivak
They are rightfully dismissive of Docker because it's just the current cycle
of trendy abstractions. It's a barely passable solution to a bigger problem.

"We're running many services on a single machine. But this is complicated and
difficult to update and maintain."

"We took our machine, ran a virtualization platform on it, and split each
service into its own VM. But this comes at the cost of increased resource
usage."

"Instead of separate VMs we created a container format to decrease the
overhead while retaining many of the benefits of virtualization. But this is
still resource heavy, and insecure as the containers will rarely see updates."

"So we created 'lightweight' containers which are very thin wrapper around the
base OS so that containers can take advantage of updated shared libraries to
mitigate the security problems, and further decrease the overhead."

"We're running many services on a single machine..."

The cycle will eventually come around and we'll be in a better place having
learned that was was really needed was improvements to the base OS, package
management, more robust MAC policy, and name-spacing rather than containers.

~~~
vidarh
> and name-spacing rather than containers.

Containers _are_ name-spacing.

~~~
juliangregorian
No, they are isolation, a much stronger proposition.

------
npx
If anyone wants to earn some huge brownie points with me (and who wouldn't),
you could implement a PaaS on top of Joyent's Triton system[1]. Purely in
terms of the cost structure you could offer with such a PaaS, this could be a
Heroku killer. Huge bonus points if it's totally open source!

1\. [https://github.com/joyent/sdc-docker](https://github.com/joyent/sdc-
docker)

