* We have a whole lot of HTTP services in a range of languages. Managing them all with fat binaries would be a chore - the author would have to give me a way to set port and listen address, and I have to keep track of every way to set a port. With a net namespace and clever iptables routing, docker can do that for me.
* sometimes, I have to deploy and insecure app. Usually, it's a badly configured memcache or similar. With net namespaces, I can make sure only a certain server has access to that service, and that the service cannot ruin my host server.
* Its possible for me to namespace everything myself with unshare(1) and "ip netns" and cgroups and chroot and iptables... but that would consume all my available time. Docker can do that for me.
* When you reach more then 20 or so services to keep track of, you need tools to help you out.
* load balancing. Luckily, I don't have to deal with extreme network loads which would require hand-made solutions, but just pushing up that number a little to do ad-hoc load balance makes things a lot easier.
Of course you do, you just moved the logic into the "orchestration" and "management" layer. You still need to write the code to correctly handle it. Throwing K8S at it is putting lipstick on a pig. It is still a pig.
> * We have a whole lot of HTTP services in a range of languages. Managing them all with fat binaries would be a chore - the author would have to give me a way to set port and listen address, and I have to keep track of every way to set a port. With a net namespace and clever iptables routing, docker can do that for me.
Nope, you wrote a set of rules and as long as everyone adheres to those rules things kind of work ( in a clever way ). Of course if you had the same kind of rules written and followed in any other method, you would arrive at the exactly the same place. In fact, you probably would arrive at a better place because you would stop thinking that your application works because of some clever namespace and iptables thing.
> * sometimes, I have to deploy and insecure app. Usually, it's a badly configured memcache or similar. With net namespaces, I can make sure only a certain server has access to that service, and that the service cannot ruin my host server.
You may be able to guarantee this with a VM but you certainly cannot guarantee it with a container.
> load balancing. Luckily, I don't have to deal with extreme network loads which would require hand-made solutions, but just pushing up that number a little to do ad-hoc load balance makes things a lot easier
The time to build a template and instrumentation for your haproxy/nginx/varnish layer is when you don't have enough traffic and enough complexities.
While a very large company can get away with building a custom in-house ecosystem that takes months for a new employee to come up to speed on, most companies find it more palatable to use industry standard products/systems that a new employee is already familiar with and can come up to speed on quickly.
Of course, but who would you rather hire, someone with solid fundamentals who can get up to speed quickly with anything, or someone who job-hops to use the latest buzzword library/framework/language that may have a useful lifetime of just a few years or even less? Given an either/or decision with other things being equal, I would almost always prefer to hire the former, and then use whatever tools worked best for the job regardless of whether they were built in-house or imported. A job-hopping resume full of trendy buzzwords is a big red flag for me.
While a very large company can get away with building a custom in-house ecosystem that takes months for a new employee to come up to speed on
This argument gets made all the time in discussions about frameworks, but I just don't see it. In my entire programming career, I have yet to encounter this hypothetical nightmare architecture that takes months before you can become productive and has no useful documentation or examples even though the entire application is built around it. (YMMV, obviously.)
Being able to get up to speed quickly on a new framework or library is an essential skill for a working professional developer. Whether that is the latest shiny Web World buzzword or something internal in your new employer's environment makes little difference, as long as the internal design is also sensible and reasonably documented -- and if it's not, you have much bigger problems than who you're thinking about hiring next.
Many highly competent individuals like to understand the fundamentals and first principles of how systems work and how different problems can be solved, and thus find work that provides no transferable knowledge/experience to be less appealing.
Of course, there is a balance here. Sometimes frustrating tasks must be done. In that case, ideally the end goal is sufficiently motivating.
Right, but to those people, whether you use Docker or Kubernetes or custom scripting or a unicorn whose horn magically creates new instances when you need them is mostly just an implementation detail. It's no more interesting than exactly which compiler or DB you use; these are just tools, means to an end. If the differences are significant for your use case, sure, you evaluate and choose accordingly, but then you use the tools to get on with the job. The deep, interesting stuff is almost always in what you can build once you're into that territory.
However, I do think it's likely there are some fundamental principles at play that make tool 1 better than tool 2 in specific contexts, and I think it is valuable and interesting to explore that.
Perhaps we are getting into the scientist vs. engineer mindset. I would guess, though, that the best engineers undoubtedly have some scientific curiousity helping them excel in their craft.
One extreme is that I see companies hold on to toxic employees because they're brilliant, even if they are bad for the company long term.
I can guarantee it pretty well on my RHEL and CentOS systems, SELinux prevents containers from accessing things they shouldn’t just like it prevents Apache from dumping /etc/passwd.
Slashdot tried that with CentOS and failed miserably. I have my doubts regarding your guarantee.
Don't get me wrong I love people doing this. It makes my skills in uncomplicating needlessly complicated systems extremely valuable.
Don't group all container implementations into one category. You could certainly use FreeBSD Jails and Illumos Zones for isolating apps.
The most secure isolation is provided with physical separation. Second best is VMs. Containers are a distant third.
If I keep my boxes up to date I only need to worry about zero day kernel exploits. So if someone comes along and uses a zero day kernel exploit to escape my FreeBSD jail then they were going to get on the box anyway, because any attacker using a zero day is highly skilled and highly targeted.
> The most secure isolation is provided with physical separation. Second best is VMs. Containers are a distant third.
While I agree that physical isolation is the safest, I don't think it's so cut and dry between containers and VMs. Virtual machines have significantly more attack surface than either Jails or Zones, think of all the emulated devices.
Nothing prevents you from using jails or zones in your VM.
I get that but that's no longer VMs vs containers, Thats VM plus container vs container.
I will grant you that containers are harder to isolate since it's same problem as isolating individual processes. But there isn't really a meaningful difference between a kernel exploit and a hypervisor exploit.
Take a look at Clear Containers (despite the name, actually using VMs) and their qemu-lite minimal machine type.
* Overhead. It might be getting smaller but it's certainly still there. Not just the application but all the OS services that are duplicated are extremely wasteful when you're billed for your resources directly.
* Your vendor distributes applications as docker containers.
* Your developers are building and testing in docker.
* Clustered resources are dead simple with swarm. It can obviously be done with VMs but it's work.
* Devs and sysadmins alike enjoy using off-the-shelf services that are maintained by someone else.
There are enterprise use cases where docker makes sense if the app support people are less impossible to deal with than the SAs. But you don’t have that issue in cloud.
Docker nowadays at least for me is essential for encapsulating st
* Not all our libraries are available in the same ecosystem. We need to mix JVM with C++ with a sprinkle of Python [via jep].
* We run a lot of small variations over the same code base. We'd rather not wait to re-deploy 99% of a fat binary for each run.
We have an old .net app we pushed to servers via zip files, and the sheer size of the zip ended up being a burden after many deployments. Docker's caching and "docker prune" make things a lot easier.
This is probably true, but it isn't good.
Because it still allows the base problem of crap coding practice to proliferate.
but nobody have to worry about it. just start a container the last guy left. /sarcasm
I don't think I believe that equality is nearly as neat you'd hope. Unless we're willing to call all code crap code, a lot of projects with well-respected code and development practices have had serious zero days, and we've also found that sandboxing software is highly effective. Insecure code in a secure, well-written sandbox can get you a secure deployment, and you probably want to have your "good" code in a well-written sandbox anyway.
bad code is often plaged with injection/sql injection. Which sandbox does an absolute NOTHING to protect you from. After all, it is all within the expected side effects of your code.
my point is that false sense of security makes it even easier for an attacker to use readily available privileges in the badly coded application.
* 'sufficiently' is doing a lot of work here. Go binaries might still need libc; Java tars-of-jars (or fat jars, if you're confused) still need a JVM, which needs a handful of shared libraries. But those things are relatively stable, generally backward-compatible, and one single big thing (libc or JVM), so it's fairly easy to manage having multiple versions installed anyway. More than zero work? Yes. Less work than managing a Kubernetes or Docker installation? Also yes.
...let's say I'm working on an existing code base that has been built in the old-style scripting paradigm using a scripting language like Ruby, PHP, or (god help us) node.js. Let's say we're well aware of the shortcomings are are looking for a migration path to move into the future.
I can just about see how we can package up all our existing code into docker containers, sprinkle some magic orchestration all over the top, and ship that.
I can also see, as per the article, an argument that we'd be much better off with fat binaries. But here's the thing: You can dockerise a PHP app. How am I meant to make a fat binary out of one? And if your answer is "rewrite your entire codebase in golang", then you clearly don't understand the question; we don't have the resources to do that, we wouldn't want to spend them on a big bang rewrite even if we did, and in any case, we don't really like golang.
All of which makes this article seem oddly academic to me. There's a lot of value to something that can be layered on top of your existing solution for those of us who aren't starting greenfield projects.
: Also, while "fat binaries" is a great term, it's one that already exists and means something totally different.
Containers offer a standard where you can deploy your applications in a language-independent way, so they avoid this scaling problem.
The same is true for most OS-level packages (like rpm and deb), except that they provide much less isolation.
I feel like I didn't get the memo... is the new-style we're all supposed to use microservices?
Having an easy way to wrap and deploy, publish/download over HTTP, and use a simple API to run and keep running is new and valuable, hence all the popularity.
For years Facebook (reportedly) did this with hPHPc. Not a very good idea these days, but it's certainly within the realm of possibilities for large companies.
(I'm replying to clarify - not sure if I agree. I'm a Python guy - I think everyone should switch to languages with significant white space ;-) )
That being said, there are many people who are using Docker primarily to solve the DLL hell problem of shared libraries. And containers don't really provide that great of a solution as far as security is concerned; VM's will also be more secure.
And the statement that the Go language is the pioneer for fat binaries is, well, just wrong. People were using static binaries (with, in some cases, built-in data resources) to solve that problem for literally decades. MacOS, for one.
I also remember working with one of the eventual co-founders of Red Hat back when MIT was interested in a proprietary math analysis tool for SCO, but which we were planning on running on Linux. MIT had purchased a site license, which he was going to implement by checking the network address to see if it was 18.x.x.x. He was going to provide a statically linked SCO binary that had this check. The funny thing was that his development system was Linux (it was more developer friendly), and he was cross-compiling to create a statically-linked SCO binary --- which we were then going to run on Linux using SCO emulation. But that statically-linked binary was basically a "fat binary", which would work on any system (including any Linux distribution) that supported the set of system calls SCO used. (Ah, the early-90's, before Red Hat was founded and before IBM had "discovered" Linux, were simpler times....)
Storage (and file system uids), networking, are all elements of the outer system that inevitably leak in the container. Using just Docker does not save you from having to deal with those, and IMHO, those are the really hard problems.
What does having an x86 and x64 binary in the same executable have to do with dependency management? You can have a fat binary that is dynamically linked. If you have an x64 OS then the 64 bits run, but they still call OS libraries if the app is dynamically linked.
Static linking  is what compiles all the dependencies into one big executable, but every compiled language has that. It is not exclusive to Go.
 - https://en.wikipedia.org/wiki/Fat_binary
 - https://en.wikipedia.org/wiki/Static_library
> every compiled language has that. It is not exclusive to Go.
I don't think the article claimed it was. But it is a lot harder with C than Golang for various technical reasons, and it's much, much harder still with most scripting languages.
It only hard if one happens to mix glibc, a specific implementation of the ANSI C standard library, with any implementation of it.
The story on Windows is much better of course, but even there making a proper dependency-free (as much as possible) binary is much easier with Go than any other language. It's possible in Go because they don't depend on the C standard library and wrote their own linker.
Even still, it's possible to compile glibc such that those NSS modules are statically linkable. See here for further information: https://stackoverflow.com/questions/3430400/linux-static-lin...
However, in 1992 it was entirely possible to statically link an application to libc4 and have it have no run-time dependencies other than the kernel ABI.
The glibc nss breakage only came later. I'm still unclear why it hasn't been fixed to once again allow easy static linking after all these years.
All of them allowed for static linking, and only a few supported dynamic linking.
Regarding Linux, the very first version to properly support dynamic linking was kernel 1.0.9, which introduced ELF support. Slackware 2.0 was one of the first distributions supporting it.
Before that, binutils had a set of patching tools to make dynamic loading work in the a.out format.
GNU/Linux is just one OS among many supported by C compilers.
Once you have a mix of different technologies its easier to just say "everything in Docker".
The ship container analogy on the old Docker website was a great illustration of this. Its a shame the new website is not so clear about the benefits and is full empty phrases such as "accelerating innovation".
Managing resource files in fat binaries is really problematic.
You press "get docker community image". Then you find your distro. Go to "download from the docker store", discover that there is no dowload link https://store.docker.com/editions/community/docker-ce-server... then you click on the long form link to the docs. Which gets you here: https://docs.docker.com/engine/installation/linux/docker-ce/...
And then that is not simple at all.
So much for fat binaries making things easy. Good-ness grief.
curl -o /tmp/get-docker.sh https://get.docker.com && bash /tmp/get-docker.sh
curl https://get.docker.com | bash
> Yes, the network can be very powerful, but trying to use it for everything is a royal pain.
He says this after saying docker is unsuited for the massive scale of microservices... Where everything communicates through the network anyways.
Also I have a feeling he doesn't know the distinction between kubernetes and docker. They don't actually compete with each other, I don't even understand the comparison. Kubernetes uses docker.
All in all, I'm not convinced.
PS: At least in python, there are ways to create fat binaries anyway. And I have no doubt that this guy is very good at operations, as he seems to have not found problems with go's dependency management and even implicitly compares it favorably with those of the scripting languages.
Docker (the company) is itself responsible for this confusion. They're redefined what "docker" means a whole bunch of times.
Kubernetes is scheduling and orchestration. It does compete with Docker, the company - it's a direct competitor of Swarm.
Docker, the container technology, is the de facto standard container at this time. That may not always be the case. Kubernetes is not Docker-specific, thus his mentioning of the open container initiative.
They are absolutely competitors.
In scenario 1, I put a Go binary onto a server and make a systemd unit file. In scenario 2, I put a Go binary in a docker container and launch it on a Kubernetes cluster. Scenario 2 is wasting a ton more cycles and RAM, but other than that, what's the difference?
* With containers I can put a Python app right alongside my binary but with total isolation. No need to futz around with chroots or making a static build of Python and embedding my scripts into it.
* What if I need libc, for example to link to SQLite or something? Suddenly my Go binary requires libc. The isolation is broken!
* Systemd can use Cgroups to limit RAM or CPU, but schedulers like Kubernetes can also use your CPU and RAM limits to schedule containers. As far as I know, there's no equivalent tooling for doing this with fat binaries.
* Without Docker, I have to manually manage each container port. Can't run two apps with pprof servers on the same box. With Docker, I only have to care about public ports, and can port-forward into debug ports manually, and with Kubernetes I never have to care about port conflicts.
* Kubernetes, with enough hackery to work around bugs, can actually do seamless deployments where it checks your internal health endpoint or command before making the container available. As far as I know, the alternative to this is writing crappy scripts that try to do this without declarative logic to back it.
You could go on and on. It doesn't have to be Docker. Could use rkt as well, or really any container engine. The point is that the container engine + scheduler pattern is immensely useful, and if it weren't, Google would already be on the next thing.
If you're just setting up a single server with a single program, fine. Drop a binary on it and call it a day. But when you want to implement CI/CD and schedule applications across multiple servers and do load balancing and so on, you might feel like you're reinventing the wheel a bit considering those problems were all already solved by Kubernetes.
The need to configure, manage, monitor and maintain a whole extra set of programs. Also, systemd can just as easily start the binary in a cgroup, eliminating most of the following concerns.
> The isolation is broken
Worth noting that behind the scenes, Linux is using the same in-memory versions of libc, even for containers. That isolation you speak of already doesn't exist.
> [...] scheduling [...]
Ansible, Chef, Saltstack, etc.
> I have to manually manage each container port
You have 65,000 ports available to you. I don't think this is as big an issue as made out to be - people are lazy, which is the only reason there are so many conflicts over 8000 and 8443.
> Kubernetes, with enough hackery to work around bugs
Given the context that I am someone currently attempting that hackery - this shit just kinda works, some of the time. And when it fails, it's a morass of conflicting documentation, and responses of "well, you're not using the default distribution for your environment" (no shit, it's no longer supported) and "works on my single node minikube cluster". Give me ansible/chef/saltstack any day of the week. Hell, I'll even take cfengine at this point.
Kubernetes was built for Google's use case, in Google's environment. Attempting to use it with other restrictions and requirements is a nightmare.
You could make pretty much the exact same set of complaints against all those configuration management tools (ansible/chef/salt/cfengine/puppet). They're all a huge mess of spaghetti and hackery that works when they work and can be a nightmare otherwise.
All these tools need at least a couple of decades more to mature.
That's a tremendous overstatement. The first part is just wrong, as was pointed out in earlier responses. The second part just smacks of your having been soured by a poor experience. We started with kubernetes in production at very small scale, just a few supporting services to begin with since there was a lot to learn. Currently we're running five or six of our core production apps on the GKE version of the platform and the experience has been very positive.
In the former case, I need to start a (for example) Ubuntu image on my machine or VM. In the latter case, I need to start a Kubernetes image. It's not as different as you think. Kops can do the heavy lifting and even generate terraform manifests, which will put you far ahead of the manual solution in terms of controlling your infra.
Also, Cgroups alone aren't enough, and is only a small part of Docker's isolation, although I did actually acknowledge that Systemd supports them...
>Worth noting that behind the scenes, Linux is using the same in-memory versions of libc, even for containers. That isolation you speak of already doesn't exist.
No it doesn't. Assuming elf binaries, the elf binfmt will call into ld-linux shared object when you hit a dynamic binary, which then does the runtime linking that links libc dynamically. You can confirm that this will call the ld-linux from the container when in a container. Compare how printf acts inside of Alpine containers versus on its Debian host. Or try hosting a CGO_ENABLED=1 Go binary in a container with no libc - it won't load because the kernel won't find the linker. If you put it in an Alpine container and it was built on a Ubuntu host, you might find it cryptically fails to load!
>Ansible, Chef, Saltstack, etc.
This is not scheduling. That is automation, configuration. SaltStack does not calculate how much CPU and RAM is available on a box to balance applications that are running. Before I used Kubernetes, I used SaltStack to manage fat Go binaries and even Docker containers for a moment, so I have a pretty good idea of what the differences are.
>You have 65,000 ports available to you. I don't think this is as big an issue as made out to be - people are lazy, which is the only reason there are so many conflicts over 8000 and 8443.
Nothing to do with laziness. Why would I want to have to pick a new pprof port every single time then remember what app has which? Why would I want to then manage a firewall for all of that? If you're doing this all by hand eventually you're going to screw up. I'd rather have isolated networking deny-by-default.
>Given the context that I am someone currently attempting that hackery - this shit just kinda works, some of the time. And when it fails, it's a morass of conflicting documentation, and responses of "well, you're not using the default distribution for your environment" (no shit, it's no longer supported) and "works on my single node minikube cluster". Give me ansible/chef/saltstack any day of the week. Hell, I'll even take cfengine at this point.
That is an absolutely inaccurate characterization of the very professional and helpful Kubernetes community. I've only had a few run-ins with them, but the experiences were absolutely completely the opposite and I've literally never heard "works on my single node minikube cluster" and I doubt you have either.
>Kubernetes was built for Google's use case, in Google's environment. Attempting to use it with other restrictions and requirements is a nightmare.
And finally, no it wasn't. It was built by the Google Cloud Platform team to help Google Cloud Platform users better utilize their VMs. There's a pretty detailed history that's all publicly available. As far as we know, Google is yet to use Kubernetes itself, other than their spin-off using it for Pokemon Go. Google internally uses some progression of Borg.
We no longer have the concern of defining static ports for each app, configuring App A with the port for App B, autoscaling when traffic increases 3x as someone attempts a L7 DDOS... I could go on!
I much prefer the Docker and Kubernetes world, but you could actually do this (scheduling and bin-packing fat binaries) with Nomad's exec driver:
They're called cgroups. You don't need systemd, Docker or Kubernetes to use them.
I had to laugh on this one.
Usually everone that somehow thinks Go compilation model is inovative, never has used anything beyond scripting languages and possibily C in addition to them.
MS-DOS used to call them "XCopy installs", NeXTSTEP had its fat binaries with directory structure, Windows and MacOS(pre OS X) can store all dependencies inside the .exe file and so on.
In any case, fat binaries don't cover the dependencies with file system, other running servers or sandboxing.
On macOS / OS X, it’s actually a .app folder
I strongly disagree with this part. To make progress as a technological civilization, without constantly wasting time reinventing things, we need to keep old technologies working. So, if Docker keeps that Rails app from 2007 running, that's great. And maybe we should still develop new apps in Rails, Django, PHP, and the like. It's good to use mature platforms and tools, even if they're not fashionable.
That word "fashionable" brings me to something that really rubs me the wrong way about this piece, and our field in general. Can we stop being so fashion-driven? It's tempting to conflate technology with pop culture, to assume that anything developed during the reign of grunge music, for example, must not be good now. But good technology isn't like popular music; something that was a good idea and well executed in 1993 is probably still good today.
Besides all that, as others have explained, platforms like Kubernetes have other advantages over just dropping a self-contained binary on a Linux server.
Also, I think the author is simply confused about what docker itself is, complaining about its lack of network orchestration - and comparing it directly to kubernetes!
I have used it when I cannot guarantee to have Node.js installed on location I am deploying app to. It works pretty well, but the C compilation of all the Node bindings took quite awhile when I was using it (about 2 years ago). I think a build of a small project took around 10 minutes at the time.
This left the resource management to the scheduler, where it belongs.
Docker combines cgroups with a layered filesystem and a tarfile to copy everything around in. It's the total package that is meaningful; the article argues against just one aspect and thus fails to cover the whole picture.
why is that good? because it means that the thing that knows about resource utilisation has control of what resources it give out. quite patch on the farm? give the low priority jobs more memory/cpu.
You also have to remember that borg is part of the secret sauce that makes google tick. Having a single interface to a smart, efficient, fast and scalable scheduler is invaluable at scale.
Kebernetes is nice, but its nowhere near as advanced or fully featured as slurm/gridengine/tractor etc. its also not that efficient or fast. (I worked in VFX with various schedulers some were able to dispatch 100k execution instructions a second)
Docker standardizes containerization. Kubernetes relies on a standardized container platform like Docker in order to be able to automate the running of different systems in a consistent and repeatable way.
Also, from the point of view of someone who has to provide support for software for multiple customers, it helps to know that the Linux environment between all customers is consistent... That way you don't get strange issues that only happen to one customer and not the others because of some slight OS level differences.
What did you think the V in JVM stood for?
Docker container is an extension of that idea. If you can distribute some of the libraries, why not distribute entire execution environment? Why not isolate your application from external environment even further?
The more you control the environment the less work you have to do to make sure the application will work.
My point of view, there are two types of applications with diametrally different development process:
1. applications where you have to invest a lot of effort into making it resistant to the environment -- it has to work regardless of the differences in user environment. Think for example in terms of a PC game that must run on different GPUs, with different driver versions, on different OS versions, with different applications installed.
2. applications where it is enough to show that there is a set of circumstances when the application works correctly -- think in terms of typical enterprise application where DEVs would place it on the server and then the environment would be religiously preserved to not risk application breaking.
Developing type 2 applications is much, much, much less effort.
By distributing type 1 applications along with the environment we basically reduce the cost of developing them because now it would be enough to develop them up to type 2 applications standard.
For example: you may create a simple bash script to do something. If you distribute it as just script file, you need to make the script work on every bash on every type of mainstream Linux distribution.
If you distribute the script as a docker image it is enough for you to make that script work on that particular image (for example Ubuntu x.x) and you don't need to worry about other possibilities.
This is the essence and value in distributing of applications as Docker images.
It won't be long before we see Go library distributions with versions in packages and security fixes rolled out to them...
What we've missed over the years is an agreed improvement to the Unix process abstraction that the kernel manages. IPC was never solved. So now we're trying to do ipc over http with JSON
I'm far from a big fan of Docker, but can definitely see the appeal for providing a solution to those problems, even if some see it as unprincipled. In our case we'd already put a lot of effort into building mostly self-contained binaries and Docker is still useful (admittedly mostly just as a component of Kubernetes; we'd cheerfully replace it with rkt or something if there was a good option).
Why take an all-or-nothing approach when you could pick the best of both approaches and use them in tandem? :)
This article is confusing in its focus on docker. It really proposes a much more restricting way to build apps, in which all assumptions except linking-level dependencies should be portable. Its answer to filesystem isolation: don't rely on the filesystem being a certain way. Or, rely on it being a certain way and push the complexity into your deployment layer.
However, there is a bunch more stuff in Docker (or equivalent runtimes) that this article doesn't touch on. It's similar in some ways, for some use cases. It could be interesting to see what a fat binary orchestration tool would look like, but I would bet you end up reinventing 99% of what Docker already does.
The author is right about k8s. And what do you need to run stuff in k8s? Containers. And what is a pretty good container format? Oh yeah, Docker.
Although other containers exist, let’s be honest, people running k8s are almost always running Docker containers in them. And it all works amazingly well.
How do I run fat binaries in k8s? In a production ready and battle tested way please.
1) container overhead <<<< vm overhead
2) better bin-packing because of 1
3) enforce resource constraints (I know what your app
4) enforce 12-factor app patterns, build apps that don't rely on local state
Each program doesn't have to worry about what else is running on the machine. It also helps you deal with the differences between say Ubuntu and redhat. Not all nix os have the same filesystem directory setup.
Contrarian views like the one in the article are healthy!
That alone is worth it, and it's what makes Kubernetes so powerful, by taking that basic runtime abstraction and further lifting it up beyond the VMs/nodes themselves.
> That alone is worth it, and it's what makes Kubernetes so powerful, by taking that basic runtime abstraction and further lifting it up beyond the VMs/nodes themselves.
how is a damn yaml file over 40 lines simpler than a 7 lines systemd? (i didn't even count the setup of kubernetes, since you prolly never did)
also anything you actually said makes no sense....
since you actually compared the things that are comparable...
edit: yes, both have pro's and cons, still your things make no sense
Both are simple to write, but K8S services for me are much easier to define. They're also far more configurable and usable across machines. Write it once and you're done, and add in the ability to use service-oriented storage and networking policies and it provides far greater power and flexibility than ever before.
My point is that there is no point comparing these things since a docker container can hold all the fat binaries you want inside... so it's not either/or at all.
Docker is a way to distribute an executable environment, very similar to distributing an application platform.
A "fat binary" is literally just a single monolithic executable application. It isn't an environment. It isn't a platform. It's ridgid and difficult to work with and, as the name implies, BIG.
There are many cases where a "fat binary" simply can not, does not, will not work. It's a very limiting way to distribute and run an application. Suggesting that a "fat binary" is somehow superior to a Docker environment is not only missing the forest for the trees, it's forgetting that there's a forest. The binary is just one tree.
Docker isn't more complex than what we had before. People just haven't been trained on the different patterns of deploying environments and platforms, in addition to applications. We used to build Docker-esque application platforms all the frigging time before Linux containers even existed, and they still exist without Docker. If you have a problem with Docker, that's fine, but it doesn't mean you have to box yourself into a totally different restrictive model.
People in this industry need to wake up and admit there are no simple solutions to complex problems. Don't believe the hype.
EDIT: "Or rather, if the problem is resource and dependency and configuration management, we should solve the problem by moving to those languages that support fat binaries."
Oh, so this article is really just an advertisement for Go. That sure spoils it.
You can easily break out of a chroot jail: http://pentestmonkey.net/blog/chroot-breakout-perl
Thats not possible in Docker (okay, it is if you're running a container in privileged mode but that's another can of worms and you shouldn't do it unless absolutely neccessary). Also, Docker gives you networking isolation and especially RAM/CPU usage limitation, which is a real headache doing with chroot.
Believe this at your own peril.
The only ways I know of that can be used to jailbreak are, as documented on https://security.stackexchange.com/a/153016: kernel vulns (which are not inherent to Docker, and can hit you on any kind of Linux environment as soon as you achieve RCE in a hosted application), running a container with --privileged, and being careless with bindmounts - either by bind-mounting /dev, /proc and friends which you shouldn't do in any case unless you're aware of the pitfalls or by bind-mounting the Docker socket file.
The latter is something that I see far too often (especially with docker-in-docker setups for Jenkins slaves), but if you're avoiding this, you're safe from that vector.
Docker for the longest time didn't even try to constrict the root user. Later versions does, but the results will vary depending on your configuration. That's why the Docker documentation states it should not be depended on for security. It is wise to heed that recommendation.
and also namespaces for file-system, network etc. etc.
Everything old is new again, this time with an orchestration layer.
Now here's Docker getting mountains of press for seemingly reinventing the wheel. But, to be fair, cgroups didn't exist back in the days of Linux vservers, so there was no easy way to constrain per-vserver resource use as there is now, the isolation wasn't as complete as with Docker, and there was no use of layered filesystems so you couldn't build up a Linux vserver layer by layer as you can a Docker container.
It was still very useful back then, though. Way ahead of its time, and it's a shame most people never knew it even ever existed.
On UNIX world older examples than the ones you provided, would be HP-UX vaults and Tru64.
Isn't Kubernetes build around etcd?
I really don't get this blogpost, it seems like "I don't have this problem, so nobody else has."
It feels like the Linux Kernel Library project would be a way to do this - redirect all the filesystem calls into mapped portions of the binary to abstract over the filesystem which I think would get you 90% of the way there practically?
docker helped alleviate fears of vendor lock-in with cloud providers.
wait some more time and you will see unikernel applications as new thin binaries
go build xyz.go
xyz: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, not stripped
This attitude is wrong. It is wrong because programming is hard. We have spent many decades trying to figure out how to program effectively. To produce simple maintainable code. Solving the hard problems, the interesting problems, is still too hard for us. The idea that "any language will do so long as it is portable" might as well lead us to write in LLVM-IR!