My hot take after all this is that it doesn't actually matter if your config language is Turing complete. Just use your regular programming language for config, I use typescript, it's great.
If for some reason you write an infinite loop in your config... you'll find out really quickly and fix it. Non-turing completeness doesn't actually solve any problems you care about, but using a crippled language does make simple things needlessly difficult.
I don't particularly care whether a language is Turing-complete or not. In the case of CUE, I like it because (a) unification is commutative, so I can define things in the most appropriate place (e.g. a company-wide repo, or project-specific, or for dev environments, etc.) and it makes no difference to the end result and (b) it has no side-effects, which avoids the temptation to e.g. open external files, or look in env vars, or download something, etc.
One big advantage of CUE being a total language is that you can always evaluate it in very different contexts. Yes, it doesn't matter whether your config is TC or not if all you do with it is configure one single program. If you have an infinite loop in your configuration instance (or if you think you have, assuming you haven't solved the halting problem) you will just fix your configuration for that specific program.
But large systems are not like this. There is no single configuration instance, configuration is pulled from all kinds of places, and each part of the system sees a different subset of configuration. And very importantly, this changes over time. The impact of a single change in some place becomes very hard to predict. Making the language total is one way we can limit the impact of configuration changes. Others that you mentioned are commutativity and the lack of side effects.
Also, being total means you can freely evaluate CUE outside other contexts than simply program configuration. For example, you can do configuration testing independently of deployment, or you can do large scale automatic refactoring, you can inspect the system in arbitrary ways, etc.
It's the eternal opposition between taking shortcuts and doing things the easy, cool and dangerous way vs. planning and optimizing and doing things the boring, slow and cautious way.
In the specific case of configuration files, the danger is easy to underestimate because configurations are often considered a minor, easy,low value part of an application.
I think you may be overestimating how universal the lessons are.
For more complicated configs, a common approach I see is that the “config file” is a program that generates the actual configuration data. If you just need to parse and introspect the config data, you fetch the build output and parse that, which does not have side effects.
The sweet spot I found for CUE was when you have existing YAML, you add CUE types externally and then validate your yaml without needing to change the program. It's like a baby step towards moving beyond YAML.
Hum... I do agree that totality is overrated. I never saw anybody complaining that they had to write configuration in a Turing complete language, and I can't imagine a situation where people will scratch their heads looking at typical configuration code and wondering where the infinite loop comes from.
But purity is extremely underrated. Configuration changing due to changes on the environment is a huge problem. Network dependency is kinda ok if it exists on a single place, and determinism is assured when everything works, but if it is going to exist in a single place, that place can not be every configuration file. And all the extra I/O features of your random language must not be used, what isn't a big problem, but having things around that must not be used is always a small drag.
At the end of the day we’re structuring syntax in a file within the constraints of some compiler or interpreter, which itself is implemented as structured text.
Engineering is about keeping it simple. So I agree; we can avoid dozens of flavors of structured text (using up worker time to learn, compute cycles to use) and stick to general programming languages rather than prove how clever we can be by creating yet another tokenizer, parser, writer with some hand wavy, tangential connection to mathematical concepts.
The problem with using a general programming language for configuration is you end up needing to use that config in different places, in different contexts, and from different languages. So, you have to marshall out and marshall in that structure to some intermediate format.
You want it to be easy for humans to edit and grok, so you find a way to represent the core parts you care about as text and cordone off the general programming language to another area.
You're successful and the number of use cases you cover grows, so the size of that config grows.
And before you know it, you've invented YAML.
Whereas, if you use Cue instead of YAML it looks pretty similar - in fact some large subset of it will be parsed correctly by a YAML parser. But the difference is with Cue you can:
1. Validate the structures in your config
2. Deduplicate by referencing other values in your config (something you can't do in JSON/YAML).
3. Use language built-ins to reduce boiler plate and repetitive text.
An infinite loop is not the most frequent type of non-termination. Crashing is.
When your "config" fails to load halfway through its interpretation because of a KeyError, ValueError, NullPointerException, "null has no method <whatever>", you see some of the downsides of Turing-completeness.
Crashing is termination; the reason it's tricky is that we often treat programs as having return type "Foo" when they actually have return type more like "Foo | Stacktrace".
(Sometimes it's easier to do theoretical work by replacing crashes with infinite loops; but that's only because it's impossible to do the converse ;) )
Main takeaway for me: CUE is not just a configuration language (I knew that). CUE has an ecosystem built around it, with verification tools, translators to other config languages, and conveniences like a language server and a formatter.
Also, the language is built to be modular, and many ready-made reusable parts are available, so you don't start from scratch when migrating.
Go, Python, and every other known programming language have ecosystems, with syntax generators, validators, and SDKs for most APIs. Best part is I can hire people today who know how to use them.
AWS SDK for Go has snippets that just work right in the docs. Writing some logic like “if attempting to use AWS types a, b, c, or send values outside this regex, error” is not that hard.
Having learned Puppet, Chef, Ansible, and having zero use for them day-to-day in the current reality of “beam API params”, a problem specific DSL seems like a dubious proposition.
The friction is still too much IMO. While I love the configuration languages, I still find myself writing YAML because it's much simpler. The tooling support is getting there but I don't understand why these tools introduce their own syntax and type system rather than being a superset of YML or JSON and using JSON Schema as the type system.
I was an evangelist for Jsonnet but somehow it didn't get enough traction over time. I blame the lack of type support and integrations with the IDEs such as VSCode. I regularly check the VSCode & Intellij support for CUE as well but the support is still at the level of syntax highlighing, not more than that, which is sad.
Nowadays, I use VSCode with YML extensions (anchor, etc.) and use JSON Schema to validate the YML in the IDE with its language server. It's far from perfect but easy to iterate and portable.
Same as Jsonnet but it's only a small part of the picture. For someone who use JSON, I expect the adoption of CUE to be incremental but in reality I need to use CUE's own syntax to get the value out of it. Jsonnet was was closer to to the mindset that I have with that respect. Some context: https://github.com/google/jsonnet/issues/605#issuecomment-72...
I have tried both typescript and cue-lang to generate json configuration for a while and I have to say that I prefer typescript to cue-lang, especially after deno provides native support for typescript.
I don't think Turing complete or not matters a lot for a configuration language. What really matter are the toolchain support and the ability of express complex concept or relationships. Writing typescript with VsCode is a really pleasure and its type system is powerful enough to express complicate concepts. Besides typescript has mature packages systems which cue-lang is not ready yet.
With typescript you can easily do things like reading a json configuration via HTTP and override some varibles to generate the target json file you want, which would be hard to do the same thing with cue-lang.
As a conclusion, I don't think currently cue-lang would be a good solution for configuration generation. But I do believe it would get better in the future if the community keeps working on it.
I've been gradually introducing CUE at my workplace. So far I've used it to generate AWS CloudFormation templates, GitHub Actions workflows, and OCI container configs. I'm also using it in git pre-commit hooks to validate files against existing jsonschemas.
I try to find what's the definition of "wins" and looking for some mass use case by which companies. "wins" in this article seems to be "I think this is good", am I missing something?
There are a ton of semi-competing configuration languages and systems that folks are using to tame cloud system configurations (which grow to immense complexity). Cue, dhall, jsonnet, starlark, KCL, HCL, etc. are just a few options. It's very hard to pick a 'best' one though as they each have interesting strengths.
I've read their comparison and it really feels like word salad.
Having gone decently down the Cue rabbit hole, I feel that Cue is remarkable in being able to do everything I need it to do without looking significantly different than JSON. The validations and boiler plate reduction are fantastic. And even without inheritance, I'm confident it can represent anything I need to without having to duplicate the same thing over and over.
It definitely solves some problems but I don't think it is the best bang for the buck. A lot of problems are solved with annoying json/yaml, those fire and forget stuff, barely need updates. Idk, I'd love some venn diagram of all problems Cue aims to solve, and also have some notes on some part of circle (problems) that are solved by more lightweight or better tools.
CUE is a superset of JSON, so the correct way to use CUE for problems that only need JSON is... to just write JSON. If you later decide to introduce CUE, then your existing files will work as-is.
If you've ever had to deal with REDCap (a clinical trial "CRM") and its atrocious data dictionary system, CUE makes your life much easier! Just use CUE to generate all the configs for you.
The CUE command line tool is written in Go, and works on pretty much any operating system and architecture supported by Go, which now includes almost everything[0], so I am not sure how you came to that conclusion.
If you want to use CUE as a library, you can only do it from Go at the moment. We are acutely aware of this limitation and we are investigating possible solutions.
Having to install binary manually is extra friction: you can't rely on a tool to manage it for you, you have to have a place for storing them, you have to remember where it is if you want to delete it.
My hot take after all this is that it doesn't actually matter if your config language is Turing complete. Just use your regular programming language for config, I use typescript, it's great.
If for some reason you write an infinite loop in your config... you'll find out really quickly and fix it. Non-turing completeness doesn't actually solve any problems you care about, but using a crippled language does make simple things needlessly difficult.