Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Bake – A modular build system and package manager for C/C++ (github.com/sandermertens)
132 points by ajmmertens on Dec 30, 2018 | hide | past | favorite | 70 comments



Cool.

One more "new" build system after Meson (https://mesonbuild.com/), Scons (https://scons.org/), "Modern" CMake (https://cmake.org/), Waf (https://waf.io/), qmake (http://doc.qt.io/qt-5/qmake-language.html), b2 (https://boostorg.github.io/build/tutorial.html), Jam (https://www.boost.org/doc/libs/1_31_0/tools/build/jam_src/in...), premake (https://premake.github.io/), build2 (https://build2.org/build2-toolchain/doc/build2-toolchain-int...), GYP(https://gyp.gsrc.io/), vcpkg (https://github.com/Microsoft/vcpkg), Bucks (https://buckbuild.com/), Bazel (https://github.com/bazelbuild) and autotools.

It is true that one more was really needed.

One more that has the genetic disease of :

- Trying of being "purely declarative", a thing that at least 4 build system already tried and failed as soon as portability and configuration kicks into the picture.

- Try to merge the concept of build system and package manager: A terrible idea for anything touching C/C++ for reasons discussed 20 times already.

- Use its own syntax and config, because that is really what every developer want to do: Learn how to use a build system zoo for every component they want to use.

Well done guys.

Edit: I forgot some...



Thanks for highlighting this- yes, there are quite a few other projects named bake. The internet is a big place, and while that means that it's almost impossible to come up with a name that's not already in use, the likelihood of running into each other is also small. In other words: I don't see this as a big problem unless one of the "bakes" becomes really big (like CMake big).

So what is the chance of this happening? Lets look at the list you provided:

- some aren't (C/C++) buildsystems (hackage::bake, bitbake, cake bake).

- some projects are dead (linuxrocks123/bake, openhub.net/bake-build, nsnam.org/bake)

- some projects are almost dead (esrlabs/bake)

That leaves one active project, which is Bakefile, which is not a build system but a makefile generator akin to premake. Bakefile (not bake) has a very different front-end (a bkl file) and different intent, so the potential for confusion is not big. IMO the internet is big enough for two different projects with a similar name :-)

Worst case the similarity in names makes it harder for me to get traction, but that's ok really. For me bake is a means to an end, and by no means the most important thing I'm working on. Would I enjoy it if people adopt it? Sure. But it's not the end of the world if that doesn't happen.


Thanks for the comprehensive list of alternatives, there's some that I wasn't even aware of ;-) A few observations:

- Bake has been used across platforms for large projects and its declarative approach held up pretty well. Note that being declarative does not mean a configuration cannot contain conditional attributes (which bake supports). You need conditionals for things to work properly on different platforms, like documented here: https://github.com/SanderMertens/bake#can-i-link-with-non-ba...

- I share the frustration, but I do not share your conclusion. Bake takes a refreshing approach to package management in that it doesn't try to do too much, which is perhaps what turned you off in other projects. It doesn't even compete with package managers like brew or vcpkg (which is not a build system, you might want to correct that). I would go into more detail, but then again I've already done so in a few comments, and the README is pretty comprehensive in explaining how it works.

- I did not invent my own syntax, the configuration is written plain JSON. Whether this was a good or bad decision we'll have to see, the jury is still out on that. NPM seems to do alright though.

So far, the people that have actually used bake have been unanimously positive. I can confidently state that the learning curve of bake is much lower than a tool like CMake or premake (which I like), let alone make or autotools. Based on the feedback of Bake's users I decided to spin off the project, as I figured others could benefit from it too. So yes, it is yet another tool, but IMHO there's little harm in that.


NPM isn't really json if you leave the easy JavaScript only place.

For anything which requires a "build" (i.e. transpiling typescript or any form of test integration, minification, ...) you are in the land of "arbitrary scripts doing whatever they like" even worse when getting to C++ modules which default to gyp ...


Like... Make? I’ve been slowly converting the Jr devs to Make. You know the best part about Make? The documentation provided by GNU: https://www.gnu.org/software/make/manual/make.html.


You are doing them a disservice. At least for cross platform C++ the best thing to use right now is CMake, followed by Bazel and gn (what chrome uses). The reason CMake is a good idea is because a large number of C++ projects support it and it has decent integration with XCode, Visual Studio (Code) and pretty much everything else as well. Using ninja gives you decent speed.


CMake is painful to use for so many common things I find it hard to believe it's achieved the popularity it has. For instance, given the focus on out-of-source builds, the -H and -B options remain undocumented for vague reasons. The community has also sworn off clean builds for some reason. The most baffling part is how anemic the command line interface is compared to the scripting language. For a tool targeting the command line aware C/C++ communities, it's far too common to rewrite build scripts with awk/sed/perl because the command line doesn't have the options you need.


I think Bazel is a really cool option. It really does deliver on the 'fast, correct' promise strongly.

That said, it certainly isn't perfect. Windows support is something that could use some improvements. I've contributed a bit on that front, but since I don't actually use Windows myself it's mostly for the sake of making Bazel usable for more people so I can feel less bad adopting it in FOSS projects. There's a lot of small bits I just don't have time to try and help with right now.

(The usual disclaimer: I work at Google but not on Bazel. At least not during work hours :) )


Yeah, make can be useful tool but it solves different set of problems than Autotools or CMake. Definitely not a fan of either of them but currently CMake seems to be the least bad choice for cross-platform development. Like the C++ language itself ;-)


I'm still sticking with CMake.

Autotools on Windows is an absolute nightmare and none of the other build systems have nearly as much support as CMake when it comes to integration with various tooling (as in tools which understand CMake).

I like gn, but it has the annoying problem of needing anything in-tree to be converted to gn.


Is it really necessary to be so mean and passive-aggressive?

You could have worded this in a MUCH more pleasant fashion and still gotten your point across. Why did you choose to humiliate?


> for reasons discussed 20 times

Could you provide some pointers, ideally a link?


It was discussed for Meson when they included the "subproject" support as major issue for all the GNOME components for most linux distributions.

There is some note here

https://mesonbuild.com/Subprojects.html

or here

https://blogs.gnome.org/ignatenko/2015/05/21/meson-and-3rd-p...

Or more recently with Bazel that do the same but arrives to do it much worst:

https://github.com/tensorflow/tensorflow/issues/6210

or

https://archive.fosdem.org/2018/schedule/event/how_to_make_p...

Same with Hunter and CMake External:

https://github.com/ruslo/hunter/issues/296

I have managed an ecosystem of ~250 software for a team in the past.

Software bundling their dependencies are the worst and any package manager will tell you that.

Due to diamond dependency problem, they explode as soon as you try to integrate them into a bigger picture... It is a pure waste of time for every packager.

And I am not even speaking of bug fixes update on dependencies, security update, environment without Internet access or anything like that.... because dependency bundling prevent or make hard all of that.


"How to make a package manager cry" - lol.

In all seriousness, the links you provided outline valid (and annoying) issues with package managers, but none of them apply to bake. First of all, bake does not rely on an online package repository. As long as you have access to git, or have local copies of your git repositories, you're good. You'll never lose productivity because your internet connection went down (well- at least not because of bake).

Secondly, the bake package manager does not mandate how you install your projects. This is perhaps the point of greatest confusion. Bake is a package manager for developers, and it provides a nice & seamless workflow for integrating remote git dependencies into your local workflow. How you package and distribute your projects is entirely up to you.

Also, bake has been used to support an ecosystem of 150+ projects, across multiple teams, across multiple organizations, on multiple platforms before it was forked off. In these cases, organizations used their own preferred tools for deployment, but got a good productivity boost from using bake in development.


> First of all, bake does not rely on an online package repository. As long as you have access to git, or have local copies of your git repositories, you're good. You'll never lose productivity because your internet connection went down (well- at least not because of bake).

It does not change anything to all the other problems of bundling dependencies. Including the problems related to diamond dependencies.

> Bake is a package manager for developers, and it provides a nice & seamless workflow for integrating remote git dependencies into your local workflow

Then just use a developper package manager like Nix, Guix, Spack or EasyBuild that do the staff. Instead of converting every other project to Bake for the sake of it.


> It does not change anything to all the other problems of bundling dependencies. Including the problems related to diamond dependencies.

As you put so eloquently before, that should not be the prerogative of build systems like bake, but of package managers like apt-get, brew and vcpkg. Bake is blissfully agnostic to those problems (thank god).

One thing that hasn't been discussed so far and I would like to highlight is code that is not distributed through package managers, as so often is the case for commercial closed source code. Bake is a great fit for these use cases, as these projects can simply zip up the bake environment (together with miscellaneous files like HTML, JS and what have you), unzip it on a different machine, and everything just "magically" works.

> Then just use a developper package manager like Nix, Guix, Spack or EasyBuild that do the staff. Instead of converting every other project to Bake for the sake of it.

If you got the impression that I am on a quest to convert every non-bake project to bake, then let me put that concern to rest. I love using bake, it is saving me lots of time and, TBH, I think there are some neat ideas in bake that other people might like, but that's it. After the code is built, it's like every other C/C++ library, and people are free to use whatever build system they feel comfortable with to link with those libraries.


vcpkg as another alternative package manager


thanks



Funny, when I posted bake on reddit, I got the exact same reply: https://www.reddit.com/r/cpp/comments/a8d7ny/meet_bake_a_new...

The comic suggests that bake tries to replace all other build systems with a single, unified approach. That is not the case. Bake is a standalone tool, and does not use or depend on other build systems (unless you consider a compiler a build system).


> Funny, when I posted bake on reddit, I got the exact same reply:

TBF the comic is very appropriate and spot on.


Is this because you feel there is already a lot (perhaps too many) other build tools around?


I was trying to remember where I'd last seen this comic and now I think that reddit is indeed exactly where I saw it about a week ago. Coincidence, same project thread.


Interesting README:

> The Dutch IRS has a catchy slogan, which goes like this: "Leuker kunnen we 't niet maken, wel makkelijker". Roughly translated, this means: "We can't make it more fun, but we can make it easier".

If only filing taxes in the U.S. was like this…

> Bake does not collect any information when you clone, build or publish projects.

It's sad that this needs to be said.


The worst part is that a large portion of the push back against improving the tax system here in the US is all the tax software companies. Intuit (and H&R Block) spend millions every year lobbying against tax code simplifications or at least simplifying how you interact with the IRS.


Someone has to keep breaking windows for the good of the economy.


I moved to the Netherlands five years ago from the US - dealing with the tax agency here is such a breeze compared to the IRS.


Having lived in a bunch of countries over the past decade, it's only when I moved back home to NL that I fully appreciated how easy things are.

Firstly, digital filing is the norm and for "simple" returns, just works.

Secondly, if you ever have a question, you can just call the tax authority on their free phone number, and pretty much ask them anything related to individual or business tax returns.


I once had to call the IRS for something and it also “worked”. The wait time was pretty reasonable (5 min or so), the person I spoke with was polite, knowledgeable, and actually resolved the problem on the first try.

It was heads and shoulders above most interactions I’ve had with with ISPs, big banks, and the like, where you get transferred around between departments that don’t seem particularly interested in helping you out....


In the UK most people don't have to file taxes at all. Can't get easier than that.


How much is that due to the UK being a surveillance state?


None.


The comparison to CMake isn't apt:

https://github.com/SanderMertens/bake/blob/master/README.md#...

CMake projects allow proper recursive inclusion, so this would be sufficient to include 'bar' and link against it:

    add_subdirectory(bar)
    target_link_libraries(Foo bar)
...this only assumes that somewhere in the 'bar' directory, 'add_library(bar .....)' is called.

Alternately, one can (and often does) use a 'find_package(bar)' call instead of 'add_subdirectory' to pull in a library installed on the system somewhere.

The main difference seems to be that bake is configured with a particular json schema while CMake is configured with scripts written in the CMake language.

I'm skeptical that the json approach stays as same as advertised once large and distributed organizations use the tool seriously. At scale, I expect a bake json to resemble a maven xml, albeit in json.

I'll also note that bake doesn't seem to support a distinct test phase, which will likely cause issues in large organizations; they would need a separate test configuration system on the side somehow.


Thanks for bringing that up, I agree that the example is quite trivial, though I think it's at least a fair comparison when it comes to general complexity of two simple projects.

Regarding find_package; it does sound similar to bake's discovery. There is a not insignificant difference though between how both work in practice. Without going into too much depth, CMake attempts to find a package by looking at (a potentially large number of) system paths. It attempts to adhere as well as possible to the conventions of that platform.

On the other hand, bake recursively searches for projects from where it is being invoked, in addition to looking in a "bake environment". The latter is a self-contained, fixed-format directory structure that contains all projects built with bake. A bake environment is by default stored in a user's $HOME folder, but can be configured to (for example) /opt. Bake environments can be easily moved to different directories and even machines.

There is nothing inherently right or wrong with either approach, but they are certainly different. Bake's approach was designed to accommodate a scenario where a developer is working on a large number of (potentially shared) projects at the same time, and for that it works really well.

Bake has been used for well over 4 years (in various iterations) in big and small organizations. It held up pretty well for large-scale projects, though I should mention that, as with any tool, it can be misused. Big organizations in particular will have to establish conventions and guidelines around using bake, just as they would have to do with any other approach.


Declarative file formats mostly seem to work fine for every other language. The only thing that I can think of that is unique to C++ is that the ecosystem is full of projects which use cobbled-together build scripts in imperative languages, which means your project may need to interface with these scripts; however, a package manager means that only the package maintainer needs to worry about these scripts; not every project that depends on them, so the declarative approach should work just fine as with other languages.


> only maintainer will write it

In C++ everyone is a combination of developer, tester, package maintainer and deployer. According to my experience, even the best package repos like conan or vcpkg's still lack many important packages and I still need to write my own scripts. The underlying problem is that it takes much more time for a maintainer to create a package description file compared to other languages because of missing dependency list or implicit version requirements, so community (or even a company's funding) simply doesn't work. The end of nightmares can only come from the adoption of modules (expected to appear in C++20) or hiring a number of specialized full-time C++ packagers with fundings at scale of millions from the committee/some organizations/whatever.


Yeah. You hit the nail on the head. Lacking a standard package manager, no module system, need for incremental compilation, and semantically significant preprocessor symbols (among other things) make C and C++ require a lot of power tools for building at all correctly.

Even relatively cleaner languages like Java need fairly involved declarative config files like pom.xml.

> ...only the package maintainer needs to worry about these scripts...

Package maintenance is only one part of the puzzle. There are still analysis, code transformation, documentation generation, and other kinds of automation.

There's also packaging for different ecosystems. Each package manager, Conan, monorepo systems, and each company specific system out there (maybe the most common concern).

A language like python already has standard languages and formats to both keep the number of integration points small and to keep the remaining ones supported.

In other words, I expect project.json to mention Doxygen and clang-tidy and DPKG at some point. Otherwise there will need to be pretty flexible ways to expose bake-deduced data to those tools.


Funny you mentioned that- before it was forked bake had doxygen & test framework integration. I'm working on porting the test/doxygen drivers as we speak.


I suspect you'll need to iterate or offer a variety of drivers in that space. Doxygen has a lot of configuration needed, often including generation of a suitable Doxyfile.

Testing is often just as complicated as the build. There can be interconnected packaging and runtime requirements for integration tests, for instance.

You could probably support portable, package-time unit tests without a lot of work though.


https://www.github.com/SanderMertens/bake is a fork of https://github.com/cortoproject/bake... Is there any reason why the title link doesn't go to the original?


Around halfway through the README it explains its origins in the Corto framework and how it has been extracted as an independent project.


Is this related to Bakefiles? There's already something with that name. [1]

[1] https://bakefile.org/


It's like the second name everyone comes up with. There are at least five build systems (or adjacent) named bake. Probably a few orders of magnitude more.


No, they seem unrelated.


My two favorites parts:

- always cross: native builds are just build=host builds.

- preprocesses as separate step efore building


Cue the pull requests adding yet another build system to my projects..


No need, bake can link with your projects like it can link with any other C/C++ library: https://github.com/SanderMertens/bake#can-i-link-with-non-ba...


I'm honored


So what I understood from the readme is that all dependencies would also need to be configured around Bake, is that correct? I guess this will be the main chicken and egg problem, as with other build tools, given the variety of tools that exist and are used across different libraries. Although I've found that autotools tends to be a common denominator in most of the ones I've had to work with...

A minor detail: the table under Template Functions is not properly formatted and Github doesn't render it as a table.


Thanks for finding the formatting issue, fixed it!

You don't need to specify all dependencies "the bake way". If you want to link with a non-bake C library (like pthread or dl), you can configure it like shown here:

https://github.com/SanderMertens/bake#can-i-link-with-non-ba...

It is also possible to wrap existing libraries in bake projects. This section of the README describes how:

https://github.com/SanderMertens/bake/blob/master/README.md#...


Oh sorry, I hadn't read the whole document. Thanks for the link, now I'll finish reading it!


Interesting.

I've been messing for a while with my own boilerplate C++ project [0], since I got sick of having to recreate the CMake set up for every app I created. It's taken a lot of CMake magic to get it working and I'm still not totally happy with the config using ExternalProject_add to automagically deal with dependencies that aren't present.

I've been evaluating Hunter [1] and wondering whether to step over.

If I understand correctly, the big advantage of Bake is that it completely replaces CMake for dependency management (at least that's what I take away from the readme)?

[0] https://github.com/kartikkumar/cppbase

[1] https://github.com/ruslo/hunter


Avoid ExternalProject_Add. You're better off using 'find_package', 'add_subdirectory', 'pkg_check_modules', or Conan depending on where your dependency comes from.

In my opinion, CMake is best as a (meta)build system, leaving packaging details to a different layer.


I use ExternalProject_Add of building external dependencies from source, like this: https://github.com/kartikkumar/cppbase/blob/master/Dependenc...


Yes, that is correct. Bake can find dependencies regardless of where the dependency's source code is. You also don't need to first install your project before it can be used as a dependency, building is enough.

A big difference with CMake is that bake stores package binaries, include files and metadata in a $HOME/bake directory. It won't actually install anything to /usr/local (or equivalent), unless you explicitly tell it to. Users typically first build everything to their $HOME/bake folder, and then when they want to make it available for all users, copy the contents of that folder to /opt when distributing the project.


Was YAML, CSON or some other format considered for config? JSON is not exactly human-friendly.


Yes, other formats were considered. There is some irony here, as I've been a vocal critic of JSON in the past (https://www.linkedin.com/pulse/may-time-stop-using-json-sand...). I ultimately went with JSON because it's ubiquitous, it has been used for similar purposes (NPM) and it has a great parser (parson) that was easy to embed without adding more dependencies to bake.

Time will tell whether JSON scales, but so far it has held up pretty well. Bake has been used by small & big organizations, while building large numbers of packages simultaneously (150+). The most complex bake project.json I've seen was still far less verbose than the average CMake file (using CMake as a benchmark since it has been widely adopted).


Toml would be the best choice IMO


TOML is syntactically noisy and about 50% more verbose.


I'm not sure how

    [general]
    foo = bar
is 50% more verbose than

    { "general": { "foo": "bar" } }
But also it's significantly more human-readable (and critically, it's much more human-writable).


It starts to get more verbose when you have more keys and deeper indentation since the whole hierarchy is repeated for each key e.g. [general.foo.bar.bar].

JSON wasn't intended for maximal human readability/writability like CSON/YAML were. It was intended first and foremost as a simple data transmission format.


Large hierarchies are always difficult in plain text formats. I'd argue if you need the user programming the config to understand a large hierarchy then you've already got a problem.


By large I mean two/three levels and above. Nothing's perfect but YAML does that fairly well (e.g. ansible playbooks)

TOML handles anything three levels or above in a less readable way than indented JSON (which doesn't even prioritize readability).

It looks find provided you're only dealing with tiny ultra simple config files.


   [foo.blah.bee]
   hi = 4
seems readable enough.


GRON and KVIN are better because: (1) I wrote KVIN as a long term joke to give my boss a stroke; and, (2) I like GRON’s author’s style.

KVIN: https://github.com/jaroslov/kvin

GRON: I can’t find the spec; sorry.


JSON seems to verbose for this


the criticism above was that make alone is not machine-understandable, therefore an IDE or other toolchain tooling parts cannot assist. JSON output (and the assumed contents of the JSON structures) enable another computer program to understand something about the build, and also is not terrible for humans to read.


How do you run unit tests in a bake project?


I am currently working to reintegrate a feature in bake (from before the fork) that will allow you to run unit tests on discovered projects. Stay tuned ...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: