When the big debate about systemd was going on, I couldn't really care less. In fact, I thought a future where the big distros used the same init system was a rosy one. No need to have to rewrite some init script to yet another system depending on the flavour of my clients. No need for our config management tools to have a gazallion 'if os == foo' statements.
Now, I'm just not so sure. The feature creep here is remarkable. Why on earth would systemd have anything to do with container management? Surely the 'docker' (or whatever) service which systemd is supposed to be managing should be doing this?
Another nail in the coffin of simple software that does one thing well.
Where was the simple software that does one thing well? Did it ever exist?
I'd believe you if Docker was some elegant, tiny, well-written tool, and systemd was a bloated mess of AbstractLinuxContainerFactoryImpls. But as far as I can tell, Docker is the thing that is neither simple nor does its one thing particularly well, and systemd implements this straightforwardly, because of all things you might want to do, booting a system is kind of easier if you're already an initsystem.
If systemd decides to implement WebGL or take on Sublime Text, let's talk, but these 200 lines of C don't seem like a reason to believe that we've moved away from simple software that does things well:
The entire diff being discussed isn't just about import.c, but a bunch of other things too. For example, to get it to do something interesting, you also want import-dkr.c. Add it all up, and we're talking 2000, not 200, lines of C:
Oops. I got confused by gitweb's interface... thanks. I have some opinions, mostly negative, about new C code, but even so it doesn't seem all that bad by UNIX standards. The bulk of the stuff I missed mostly looks like it's about connecting to the Docker registry and linking to libcurl, and this is about as big as I'd expect for C.
I don't think systemd is moving into this territory just for fun. I seem to recall it's because now the kernel is moving towards unified cgroups-trees if systemd wants to be able to use cgroups it more or less has to be "in charge" of them at the root, meaning it's going to have to take care of this sort of functionality.
Last i read on LWN the cgroups kernel dev was backing away from the idea of having a singular cgroups management daemon.
And no, its not just for fun. RH is betting big on competing with Amazon and MS on cloud services. It would not surprise me if they are trying to get old RHEL server customers to move their servers onto the RH cloud with the switch to RHEL7.
I'm not super familiar with either, but as I understand it, systemd-nspawn is only an alternative to "docker run". The docker command supports all sorts of other things, like creating container images, manipulating, uploading, etc. systemd-nspawn just runs things, and the upstream docs suggest using debootstrap, yum, etc. to create a thing for nspawn to run. (Running things is both the easy part, and the part that most makes sense for an initsystem / system daemon to do.)
systemd-import will grab Docker images from the internet, but I'm not sure it will do very much else. In other words, even if you adopt systemd's way of doing things in deployment, your developers are still using the docker command.
>When the big debate about systemd was going on, I couldn't really care less.
>Now, I'm just not so sure.
You're starting to see what the original opponents feared a long time ago. It doesn't look like the feature creep is going to end any time soon, although I wonder if there will be a breaking point when even the fanboys say, "Hang on, this is a bit much..."
It is singular software by implying a plural tense. It may be miss branded as a single monolithic unit but it exist to an extent as separate entities, and yes, some have a hard dependency between each other. It is no different to that of iproute2 which has succeeded net-tools; what were your thoughts on that?
I was looking into systemd-nspawn recently because I wanted some container features (tcp port namespace, so I can have multiple groups of processes run and connect to an instance of mysqld on the default port, on a ci server).
I installed it, ran it, and it immediately complained "not a systemd system" and refused to run. I've not looked into things further, but presumably systemd-nspawn requires that systemd be running, which was a surprise to me since systemd-nspawn calls itself "chroot on steroids", and chroot cares nothing about the init system.
> systemd-nspawn calls itself "chroot on steroids", and chroot cares nothing about the init system.
If I remember correctly, systemd-nspawn uses cgroups (hence the "steroids" part of of "chroot on steroids").
On Linux, cgroups require a single manager for all cgroups (in this case systemd). This is not a systemd limitation; it is a requirement set by the Linux kernel[0].
In theory you could bring-your-own cgroup manager, but cgroups are nascent enough that trying to make a userland tool like systemd-nspan work with a completely pluggable cgroup manager would be a nightmare.
Hm, interesting. I'm certainly largely ignorant of how cgroups works, but I was wishing for a tool like "chroot" that could also "ch-network-namespace" or something. I've skimmed over cgroups documentation [1] several times, but never quite gotten a solid mental model of how everything fits together.
You're probably looking for the "unshare" command and corresponding system call (or the same flags to the "clone" system call). Specifically, running "sudo unshare -n bash" will get you a shell with no network devices other than lo. You can then find the pid and use "ip link set dev eth0 netns 1234" from outside to move eth0 into the new namespace (more practically, you might make a virtual network device and move one end into the new namespace).
Loosely a "container" (in the LXC or Docker sense) is a combination of making new namespaces, which isolate the process tree from the rest of the system in various ways (filesystem, network, hostname as returned by "uname", etc.), and making a cgroup, which allows for process tracking and resource allocation.
It'd still be part of the same source tree, and there was tons of controversy about udev and systemd being joined into the same source tree. Alternatively, it could be a separate source tree if systemd implemented deep APIs for everything it does, but they don't want to do that.
Great, and it's a bunch of C with obvious memory leaks (remember, if theres a dup in that function name, it's gonna allocate memory) and other problems parsing complex formats it's downloaded off the web.
This is terrible in both the "why is systemd doing this" and "it's just plain terrible software" sense.
> it's a bunch of C with obvious memory leaks (remember, if theres a dup in that function name, it's gonna allocate memory)
That's only really an issue if someone ever turns this into a library and calls it from long-running code. As a command line utility the small amounts of memory that are leaked don't ammount to too much waste and everything gets reclaimed on exit.
The point has been made elsewhere already, but I'll mention it again: because theres no guarantee this code will stay in a simple command line utility.
At some point someone may come along and refactor this Docker import code into a library or something bigger.
If you can't be trusted to get the bookkeeping right what makes you think the rest of the program is solid?
Being able to manage your allocation is a pretty good sign that a C programmer knows what he's doing. Relying on 'exit' to free your memory is backporting the web mentality to unix land, it just simply doesn't work that way. There is no different attitude when you write a long running daemon versus a utility program because for all you know your utility program code will be re-purposed to become part of a longer running daemon. So you write your code in as clean a manner as possible and balance your allocs/frees and make sure that you don't have any latent buffer overflows which you may not care about today because of the context your code executes in today because tomorrow that context of execution might change and then we're looking at yet another exploit.
The liberal use of completely unportable cleanup attributes shifts preventing memory leaks to the equally intractable problem of tracking stack allocated variable assignments, with an extra helping of double frees. I won't even comment on the show of horrors that is util:
systemd is explicitly unportable (it uses glibc/gcc/linux-specific features liberally), this is not some sign of sloppy coding, but an explicit design decision as codified in the CODING_STYLE file.
Preventing memory leaks and tracking stack allocated variable assignment are not comparable at all. In the vast majority of cases the tracking of stack allocated variable assignments is entirely trivial. The problem with double freeing is also very hard to run into in practice. Getting a memory-leak when you don't use _cleanup_ is all too common on the other hand. Basically, as soon as there are more than one branch in your function you either have to use ugly goto's or remember to free on every branch, in practice either way you are just implementing the _cleanup_ macros manually all over the place (with all the bugs that implies).
Why don't you point to some code that would have been better without the use of the macros? It seems you think util.c is a god source of examples...
Note that unfreed results of strdupa(), despite having "dup" in the name, does not result in memory leaks. The "a" in the name means it uses alloca(), thus the memory comes from the stack and it's properly released on function return. It's also probably slightly faster than malloc() since it just moves the stack pointer.
Before jumping to conclusions, please verify what the code really does. C is a complex beast and what's "obvious" may not be such.
Relying on such niceties (as weird as it may seem, to a C programmer those are niceties) is what you gain if you are willing to trade them for portability (strndupa() is a GNU extension).
I don't like the usage of that attribute either (but if that's their strategy, fine by me.), but I still don't see where the obvious memory leaks are. Your first link links to a line with a strdup() call, but the corresponding free() can be found in L168.
Sorry, I linked to line 1140, but it doesn't highlight that and can't scroll to it as it is towards the end.
I did just check again and realize that the single 'a' at the strndupa means it's using alloca, which is of course similarly unportable and has a whole slew of other problems, all of them intractable (did you pass that pointer to something that will use it later on? stack overflows?).
So hey, not a memory leak! Rejoice! It's just similarly broken.
Just because strdupa is not magical, does not mean it has its valid uses. In particular, the problem you allude to (knowing the scope of your memory) is something you would have to get right also if you use heap-allocated memory (unless you just leak everything of course, but I think we have established that that's not something we want to do).
Oh, I am the one that has to be sorry! I didn't check the URL, even though I am aware of that issue with cgit (no line highlighting).
Yeah, using alloca() (in any form, incl. VLA) isn't good style. At least it's bounded, since filename_is_valid() checks the length. But it uses FILENAME_MAX, which could be too large for the stack.
> But it uses FILENAME_MAX, which could be too large for the stack.
That's extremely unlikely.
This is definitely not pretty or good quality C code by any standard but let's not start spreading wrong information.
The stack overflowing because you use alloc for something of FILENAME_MAX length is just as likely as a malloc call running out of memory. After all, the heap grows 'up' and the stack grows 'down'. Other than that the mechanisms are roughly (very roughly, ok) identical.
The only time when you have to be extremely careful with alloca is when you use it in functions that might be called in a recursive manner. But then you're playing with fire anyway if you do not have hard upper bounds on the depth of your recursion.
Is there anything more we can throw into systemd, perhaps http/https/v8 server so we can replace apache, nginx and nodejs as well?
I don't like how systemd starts doing everything, people keep pointing to alternatives but when things start to depend on systemd for functioning the alternatives start to disappear one after another.
I'm also thinking that each new feature like this would widen the area for potential security vulnerabilities.
Now, I'm just not so sure. The feature creep here is remarkable. Why on earth would systemd have anything to do with container management? Surely the 'docker' (or whatever) service which systemd is supposed to be managing should be doing this?
Another nail in the coffin of simple software that does one thing well.
Edit: s/docker/container management/