I've experienced this myself in a number of different environments. But I'm still not satisfied with the results with the more flexible tooling, usually. I think it's because they commonly lack scaffolding that I want a majority of the time.
The older I get, the more I realize that the lisp DSL approach is probably best: provide a simple, declarative walled garden. If I want to change something for my project using the full language, let me, and may the PR reviewers determine my fate. Ruby also seems to adhere to this philosophy, although I don't have a lot of experience with it.
> I'm still not satisfied with ... more flexible tooling ... they commonly lack scaffolding that I want a majority of the time.
I think this dichotomy, flexibility vs maintainability, reflects build complexity.
Not only do I think the same-language-DSL (a la LISP, as you pointed out), is the 'least bad' choice, I think if we raise our gaze a little the answer here is painfully obvious: how do we handle those issues everywhere else?
In other cases if you need to move files, call some sys commands, structure some data, and express some logic? What if you have to call a webservice with some throttling logic, where is your "build language"? Plop something in a DB? Not to mention that complex builds often have specialized command line apps called by their specialized build langauge.... why?
Complex logic we use programming languages for. Why sacrifice power, components, libraries, compiler-time-checking, and all the rest to use a lesser tool? Full language power, it's The Right Thing.
Confused here. I can't really stop progress waiting on build system features. Extensibility is required like a programming language. Otherwise, you must stop using the build system or build wrappers around it or outside of it making your build even more bespoke than if you used a more flexible system.
The older I get, the more I realize that the lisp DSL approach is probably best: provide a simple, declarative walled garden. If I want to change something for my project using the full language, let me, and may the PR reviewers determine my fate. Ruby also seems to adhere to this philosophy, although I don't have a lot of experience with it.