The description states: "Running Clang in a container has the advantage that it is light on resources and won’t mess with your underlying OS.". Containers aren't really light on resources if the host kernel is not Linux and not messing with OS is easily attainable by installing the binaries in a custom prefix directory (e.g. ~/bin). This is especially true for compilers which have minimal or zero dependencies.
I just ran "pkg install llvm50" on FreeBSD. Easy, quick, and noninvasive; it's not like installing another compiler is going to trash your system, you still have to take action to use it in place of the default. Likewise on Linux, you've always been able to install multiple compilers, and I've never had a problem with that in the two decades I've been doing it.
I can see containers are sometimes useful and/or a convenience, but you're right it is currently some weird cargo-cult. I don't find it that lightweight either; it can waste serious amounts of space storing all the different images and intermediate states in Dockerfiles. At least on FreeBSD I can use jails with ZFS snapshots (and dataset compression) to keep space wastage to a minimum, and have it quickly freed when I'm done with them without any extra effort.
I find your example of installing a package to be somewhat amusing.
The fact that one can install a package in some OS means that somewhere, one or more people sat down and wrote a recipe for building and packaging it, and then hopefully performed some minimal QA on the final product to make sure it actually works.
This is not trivial, particularly not if multiple versions of a compiler may need to coexist (because of other packages depending on a particular version), and it needs to be done for each combination of compiler version and OS.
To me, examples like: "just run <insert package installation command here>", expecting a package to just magically install itself on some OS are the symptoms of a cargo-cult.
Rather, I think that using containers is more like outfitting a cargo container with a fully stocked and staffed workshop, then shipping the container and personnel to where the client needs it to be. The client then only needs to supply the storage space for the container and the goods that need processing (a shared directory in this particular case) and wait for the finished result.
Sure, some of the workers may not speak your language, they may use different tools, but until (if?) the workshop gets rebuilt and staffed on "native soil", the inconvenience should be acceptable.
Not to mention Docker on OSX just runs an underlying Virtualbox (HyperKit on newer versions) instance. So just run the VM, Docker is not doing anything
Not to mention if I had a dime for every case of people running full-blown distros under docker I would have been a millionaire by now
> so I have no idea why you'd want to do this on Windows.
If you're trying to write portable software, that's the easiest way to validate compilation for Linux without: a) using an external CI, or b) running a full VM in parallel.
Well, since we are speaking about clang there is always cross compilation option: "clang -target x86_64-linux" (note, that it supports all targets that your current build of clang has enabled).
But probably the real option (c) that you are missing here is the fabulously named "Windows Subsystem for Linux".
Re crosscompilation: Sure, if you want to prepare an environment which has all the dependencies, then that is a way you can take. I always found it harder than using the target system directly. And in the end if you're running any tests after compilation, you still need the target system for that.
Re WSL: If the app you're working on works on WSL, great. It's not an option for many projects though.
Technically, yes. But by full VM, I mean a parallel desktop/server system with permanent storage that you have to maintain in different ways to a temporary container. Same technologies, different user experience. Docker hides it by providing a nice layer on top.
Clang on Windows (not cross targeting) is faster for basically no performance loss (it's in the noise for chrome). It's actually very significant performance gain in some cases. Build time savings is literally hours on official builds.
Cross-targeting from linux is probably slower ATM (clang-cl is not fast :P)
That said, if you have a single compiler you can use across platforms, and it generates the best code (or within a small enough percent), and enables a suite of cross-platform dynamic error checking capabilities (ASAN, etc), that provides significant productivity/etc advantages for people. Especially when you can easily improve them according to your own engprod priorities.
The label worst std support would go to: EDG eccp, Intel C++, IBM XLC++, Sun/Oracle C++, Embarcadero C++ Builder, Cray, HP aCC [Collapse], Digital Mars C++
MSVC has very bad C support, this is their problem, not C++.
Yes, that's true. Their overall C support is horrible, esp. up to C99.
But they were still the first ones who implemented C11, which gcc and clang refuse to do.
I'm sorry but this MSVC C11 claim is complete nonsense.
C11 is an extension of C99, so you can't have C11 support without C99.
But this nonwithstanding, MSVC doesn't appear have wide support for post-C99 C11 features either. For example, _Generic and t.
Are there any parts of C11 that MSVC supports aside from the Annex K functions, which is optional in C11? Annex K happens to be an extension proposed by Microsoft, so it's small wonder they implement it and it's quite disingenious to talk up microsoft for C11 support in this case. Other vendors didn't think it was very good, so didn't implement that optional feature.
And contrary to your claim, gcc and clang seem to have quite good C11 support.
Well, MSVC supports those eye candy features yes. But it doesn't implement the C++ in the standard conforming way. Name lookup, overload resolution, and templates and practically everything is not implemented according to the spec.
Really, it's garbage and it shouldn't be considered as C++ compiler.
Maybe one reason one might prefer using Clang over MSVC is that it's open source and it runs on all major platforms so you're hopefully less likely to run into issues. Although I admit I'm just taking a guess, since I'm only a casual Windows user.
I write many simple scripts with node just because it works everywhere with minimal quirks. I also have a few incredibly simple web apps which I use to visualize or explore small datasets. It's great to able to open up an html file in the browser and have it work everywhere.
Docker containers aren't an ideal solution, but they make it easier to get things running everywhere. Consider why stuff like Electron is so popular, despite its many quirks.
> Granted, the state of C++ on MacOS is embarrassingly bad.
In what sense? Apple keeps Clang up to date on macOS, and it is essentially no work to get the associated Clang tooling and other C++ tools (valgrind, etc) installed through something like homebrew.
Actually, not especially. The system compiler that ships with Xcode is still quite old and the libraries are quite out of date. For example, though the most recent Xcode finally added -std=c++17 it still lacks core library features like std::variant. I wish they could keep close to mainline but for whatever reason they can't.
I'm not quite clear about your point. The version of Clang that Apple ships, for whatever reason, is pretty far behind the released Clang. I cited std::variant as an example of something which has been in the public Clang for ages.
As far as compiler support goes, Clang supports all of C++17 except for P0522R0 which is a defect report (ambiguity in specification) so cannot be implemented as written. Ditto GCC.
However std::variant is a library entity, not a compiler entity, and indeed, the libraries are a bit behind. Neither GCC's nor LLVM's runtime library support the parallelism extensions and have other gaps in areas exotic (memory, unfortunately which I would like to use) and prosaic (filesystem, which though I'd like to use it as well don't really care about). But they have been solid on C++17 core functionality for quite a long time.
C++ isn't as important to Apple as Swift (or even, still, ObjC) and C++17 is pretty new, so one can hardly blame Apple for this, especially since it's so trivial to simply use another compiler.
No it's awful. They are most certainly not up to date, and even worse, the compilers they ship that advertise certain flags aren't even necessarily feature complete. A glaring example of this was the lack of thread_local support for ages.
Yup. It was only about a year ago that the undefined behaviour sanitizer stopped segfaulting when I turned it on, despite it being clearly "documented" in the man page. I'm amazed that a company with the resources of Apple has such shoddy developer tools.
"AppleClang" is some strange fork with some features (both compiler and library) omitted or broken. OpenMP, for example, is one thing I've been missing for a good while; I've not checked if that's now present with new releases. I can understand adding extra functionality to a fork, like Objective C and swift stuff, but dropping bits is strange.
It's due to issues like this that I do all our clang testing on FreeBSD, where it's a more vanilla build of the llvm codebase, and I treat AppleClang as a wholly separate compiler variant which requires independent testing.
I think clang's OpenMP implementation might not be 100% ready? Many FreeBSD ports that have an OPENMP option straight up use gcc when that option is enabled.
While docker is pretty bloated and doesn't really add new capabilities, I've found it a real time-saver for distributing cross-platform build environments to a dev team running multiple Linux/osx releases, especially when those build environments are used only a few times per year.
Probably a VM or really good scripts could take care of it, but since we use docker elsewhere people don't have to learn something new that they'd treat as a black box anyway
Eh, I made a G++7 container for using with CircleCI. Makes life a lot easier (and is faster) than screwing around with running scripts before each build
No, it is rather vague what the original article means by "won’t mess with your underlying OS". And your "underlying OS won't mess with it" is equally vague too, IMHO. Providing some examples of "mess with" will be helpful.
Unless you've found a serious bug / security vulnerability, Docker doesn't mess with your OS. On OSX / Windows (and this is definitely an argument against Docker ux), it's not even running on your OS but a separate VM.
Just pick a random tool, throw it in docker and farm karma.
IIRC even though windows 10 might support docker through WSL sometime in the future, for now it's just spinning up a vm like on mac. Is it possible to have multiple lxss with different base images in windows 10(because then you'd just use a ppa or something)?
So we have a VM which pulls in a bunch of layers. At that point, you might as well just distribute a full vm through vagrant, then your vagrant up is just a one liner.
I dual boot my laptop with macos and linux, but am I seriously the only one with 256GB space that has to worry about how many vms/docker layers get pulled?
I kinda liked gentoo prefix, far superior to homebrew imho. I used to update some llvm patch for various versions[1], but the lack of binary packages and smooth developer experience is what killed it IMHO.
Docker for mac doesn't use a virtualbox, it uses "HyperKit".
> Docker for Mac does not use VirtualBox, but rather HyperKit, a lightweight macOS virtualization solution built on top of Hypervisor.framework in macOS 10.10 Yosemite and higher.
Does Docker for build environments make a lot of sense? I want to package up an existing environment that predates Docker (by long enough that I don't think there's an old enough base OS image on Docker Hub unless I leverage J. Random Hacker's image that's been tweaked for another project) to make it more portable. Would I be fighting against the design of the tool? Perhaps it would be better to just make a chroot and tar it up? I've tried to educate myself on the tool and make my own judgment, but it's surprisingly hard to find material about Docker that is actually about Docker instead of primarily being a sales pitch for containers and microservices that happens to describe them in Docker terms.
I do exactly this; my build system does its builds for most platforms inside docker containers. (I haven't figured out how to do OSX builds inside a docker container yet, but I'll figure it out eventually!)
One of my targets is SteamOS (i.e.: Linux with a particular set of pre-installed shared libraries), and the preferred build environment to target that is inside a chroot. So my SteamOS builds get built inside a chroot inside a docker container. And that was surprisingly tricky to get working! There are some gotchas to overcome around how chroots handle symbolic links inside a docker container. But I've got it all working now. :)
The big benefit is that host-OS-upgrades-for-security-purposes no longer upgrade my build's toolchain as a side-effect, potentially breaking or changing the performance of my build artifacts; it's fine to keep running ancient software inside that docker container that's just going to be wiped out in 10 minutes and never gets directly connected to the Internet anyway. And the docker image can be copied around to wherever I need it, if I need extra build workers or whatever. Once you've got the thing set up once, it's fantastic for scaling up your build server if you eventually want to do that.
I think it makes a bunch of sense for both Dev and Build. I wrote a small tool, stowage.org, to make it easier to bundle stuff up into containers and then execute like a local command, and to be honest it works great. A lot of predefined containers don't play easily, but doing custom stuff is very straightforward and it's a nice workflow.
This is actually one case I've been thinking of using docker for, that is, having a development image for your project's build environment to provide a consistent base that has all the development libraries installed and doesn't require any fussing around with library versions.
I'm sure with certain languages/runtimes you can leverage the package/dependency manager to serve the same purpose, but I imagine having a docker image for this would mean everyone's using the same base OS with the same installed library versions. This way anyone who wants to contribute to your OSS project can clone the repo, run a simple command to initialize the dev environment, and start hacking right away.
As an aside, I completely agree with your sentiment about trying to learn about the actual tech rather than just being sold on some sales pitch targeted towards less tech-savvy crowds. The best "solution" I've found for this problem myself is to educate myself as much as I can on the fundamentals of the problem space and try to reason about that decision myself.
For example, I haven't done much of a deep dive into docker yet but have spent a lot of time trying to learn as much as I can about the fundamentals/primitives provided by operating systems and kernels. I feel this way I can always reference back to basics and make a better decision of whether something is gimick-y or not.
Lately I've been so frustrated with people selling these "solutions" yet not having any understanding of the how things work that I've spent the last 3 years trying to learn as much as I can about infrastructure primitives (TCP/IP + general networking theory, database theory + database engines, and OS/kernel internals). It takes a lot of time but I definitely think it's worth it since it's a lot harder to be "fooled" when you actually understand things from the bottom up, for some definition of "bottom". It also helps that I work in this space professionally, so the investment in learning these things literally pays off.
Actually, that's pretty much a perfect use case. "This environment is nothing like where I want it to run, and may even actively conflict with it". It's not really hard to take a chroot and make it an image, either.
For building, you can create an image that is ready to build your application, then run it as needed. After each run, unless you've set up some persistence, it goes back to the pristine state. If you create lots of almost-identical environments, you can use layering to save space (but space is cheap, so...)
If you want to ship an application that way, you just take that a step further. Create an image ready to run the application. Just remember that storage is, by default, ephemeral. It's a config option away, but annoying if you forget it.
I like that I can just ship an image to someone and it'll work on their machine. Great for our OSX developers.
The interesting functionality is the use of namespaces, which are ways to create isolation of {IPC,networking,users,filesystems,processes} between the containers. Docker didn't invent them, they've been in the Linux kernel since forever, but Docker has made great use of them.
Volume mapping is useful too. Most of what it does is achievable via bind mounts, though, volume mapping is just cleaner, IMO.
That said, I've found the most usage out of Docker from the networking side of things, as extraordinarily convoluted as it can be. I have a legacy application that cannot run on any port other than the one that's hardcoded in the application. It tries to connect to itself immediately on startup, too. Obviously it prevents itself from running another instance... by killing the old instance when the new one starts up. Putting it in a Docker container let me run 12 of them, side-by-side, with the ports mapped externally to different ports, invisible to the application.
Just an FYI, I'm not one of the converted. I think the current fad of "Dockerize all the things" is a bit silly, especially when I start seeing people doing things like putting ssh access for the host machine in a container. I have no interest in their tools other than Docker itself. I'm not a fan of their whole "one process per container" suggestion, and basically ignore it whenever I want (you should make sure you have something running as the init system in your container if you do this). But it has pretty much replaced chroots for me, no small task.
Recent versions of several Linux distributions and FreeBSD have made newer clang and g++ versions available, which both have decent support for C++17 features.
For example, Ubuntu 17.10 provides clang-5.0-3 and g++ 7.2.0. Installing clang 5 there is as easy as
sudo apt install clang-5.0
It's also easy to install various compilers side-by-side there, without messing up the entire system.
For die hard everyday C++ development, one probably does not want the C++ compiler run in a somewhat restricted container and experience all the potential trouble and limitations.
However, containerizing the compiler is a good way for providing a standardized build environment, because one can pin the exact versions of the compiler, libraries etc.
I have also seen people moving all their not-so-important stuff and/or the components that they do own maintain themselves out into Docker containers, in order to keep their base systems lean and clean. And so they can get rid of the containerized things easily should they become irrelevant. Plus keeping the containerized things relatively stable and unaffected by changes to the base system.
Seems like a lot of work for what really is a simple process.
We build our code on MacOS with a simple Makefile that builds all of xcodebuild, g++7 (from Homebrew) and llvm-5 (again via homebrew) without any fancy standing on our heads. Works out of the box and doesn't interfere with the system tools.
If you can't control what's on your PATH variable and your compiler options a container won't save your build process.
Sure, I wouldn't use Clang in a Docker container (heh). But there are some points. If building from LLVM+Clang from source, installing files/headers and such can mess up systems royally. One docker container could theoretically test drive clang everywhere without mucking up a system.
Docker is useful for many things but why is is necessary here? It is trivial to have a second copy of clang in a separate directory and just run it from there.
And can it interact nicely with the "native" environment? I.e. build MSVC projects or ".exe" apps in general that can be used outside of Nix or WSL? I am not a Windows dev but I am very curious, it would be nice to move some cross-platform projects I work on to Nix completely for dev environment...
“Docker Engine” doesn’t but “Docker for Windows” provides several tools including a wrapper around Hyper-V. I’m sure “Docker for Mac” does something similar.
A year or two ago, it used to spin up Virtual Box.
The description states: "Running Clang in a container has the advantage that it is light on resources and won’t mess with your underlying OS.". Containers aren't really light on resources if the host kernel is not Linux and not messing with OS is easily attainable by installing the binaries in a custom prefix directory (e.g. ~/bin). This is especially true for compilers which have minimal or zero dependencies.