One thing I'd add to this list: lots and lots of examples.
Coding agents are absurdly good at understanding and adapting examples. If your new language includes a large collection of searchable examples even a weaker model should be able to find something close to what it's trying to achieve and directly adapt that example code.
I've been working on DML, a Prolog-based DSL [0] used to define and orchestrate agents and LLM workflows. It's been quite fun, although - given the amazing capabilities of SOTA models - I am not so sure anymore how meaningful it will be to continue with this work. Anyways, the language and also supports DCGs, so it should allow for plenty of interesting ways to combine grammars, LLMs, agents etc.
> I've been working on DML, a Prolog-based DSL [0] used to define and orchestrate agents and LLM workflows. ... I am not so sure anymore how meaningful it will be to continue with this work.
Perhaps the research documented in "Combining Constraint Programming Reasoning with Large Language Model Predictions"[0] can provide meaning and/or options to your work.
I hope so, because the idea of Prolog leveraging language model offerings is very compelling.
I don't think we need to worry about the survival of DSLs. DSLs + LLMs are a powerful combo. I've hit limits with building MCP tools for LLMs using low-level interfaces, and I overcame it by making small, quickly teachable DSLs that the LLM can interface with better.
For example, working with MIDI-based music at a low level was a struggle to get musical results from an LLM, so I switched to a custom notation for representing MIDI data [1]. Then, doing interesting operations on MIDI wasn't working well so I designed a "language" for transforming MIDI data [2]. To get the LLM to understand it, a skill-like context was designed that teachers it the grammar and use cases via many examples [3]. To ensure this actually works better, I have automated evaluations in this repo that ask the LLM to do various musical operations and judge the efficacy.
So in my experience, tight little DSLs for specific complex things can make an AI much more effective. The tricky part is token management. If the DSL is too complex and requires too many examples, then you have to burn a lot of tokens every conversation to make it work well. It's not a big deal with frontier models, but once you start targeting on-device small language models, the tokens really matter and it's a challenging balancing act. I've dealt with this via a simplified skill set that doesn't teach any of the advanced DSL features and simply pretends they don't exist.
In any case, the beginning of the article sounds like it's wondering how we're going to make new programming languages, which is a different concern from DSLs.
A DSL exists to give you the power and flexibility of a language, with functionality built-in to make it easier to accomplish specific tasks. It's like an application with a library and config file, but takes it a step further by allowing you to express complex logic.
Humans need help in forming, understanding, and expressing logic; that's what a language is. But computers have an easier time of it. They can essentially read and write whole lot of 0's and 1's and automate the same process without language. An AI is an in-between state: a computer designed to deal with language, to think and act like a human. AIs are pretty good at dealing with human stuff, but they're a poor choice for doing computer stuff.
So really, if you want to do something you'd normally use a DSL for, you should be talking to your AI, and telling the AI to encode it into computer-speak. Binary files, libraries, programs, composeable piped applications. The AI can take what you think and turn it into a regular old computer program, just as easy as you would write a DSL. But you don't need a DSL to do this; the AI can already program.
In fact, talking to an AI might have better results. Humans screw up code because they can't really hold all the logical permutations in their head at once. But the AI can take instructions, decompose them, explode all possible permutations, identify outliers, and encode the result in a programmatic format, that a library that parses actions can then turn into a deterministic program. The AI can take your instructions and convert it into executable binary. No need for pseudocode.
So the future of programming, if we desired to go there, would be Natural Language Programming. Our speech is the programming language, and the AI is the compiler. The trick would be to work on the loop between the human and AI, to be as specific and deterministic as possible, to ensure the compiled program does what we intend and expect. It's not hard for an AI to make a deterministic program, but it is hard to be sure that it's deterministic in the specific way we want.
>So really, if you want to do something you'd normally use a DSL for, you should be talking to your AI, and telling the AI to encode it into computer-speak. Binary files, libraries, programs, composeable piped applications. The AI can take what you think and turn it into a regular old computer program, just as easy as you would write a DSL. But you don't need a DSL to do this; the AI can already program.
I use LLMs to write DSLs that I write all the time! What else is an LLM doing when it is stringing together a bash pipeline than coordinating a number of DSLs? jq is what, 15 years old? I see LLMs using it all the time. Who knows, maybe there are a few more tools that have yet to be made that will further bridge the expressivity gap between human language and computers.
When LLMs write bash one-liners today, it often leads to errors. There's a large range of possible functionality, different versions, extra abstractions, uncertain errors, varying functionality, lack of types and schemas. The CLI is kinda like a language, but much more abstract; this confuses the LLM. Imagine if the English language changed as often and as widely as a CLI program's arguments, options, outputs can across versions, platforms. On the other hand, if the LLM writes python instead of bash, it often leads to more reliable results for the same task, since it varies less frequently, is more specific, can be syntax-checked, has standard metadata, more expressive logic, etc. But there's also a lot of useful functionality exposed by applications that doesn't exist in many libraries, so there are limits.
We do need more tools for the AI to turn our requests into deterministic, reliable, correct results. But this isn't a DSL thing, it's more like a pipeline of steps to get from A to Z. This will likely require multiple bidirectional passes, to confirm with the human along each step, and fix and re-do the pipeline when a mistake is found. You could encode the final result in some kind of DSL, but it'd only be useful as a read-only artifact; if you change a line of it, without extensive testing in an immutable environment, it introduces bugs. We need to lean more into reliability with LLMs since they are so fallible.
Coding agents are absurdly good at understanding and adapting examples. If your new language includes a large collection of searchable examples even a weaker model should be able to find something close to what it's trying to achieve and directly adapt that example code.
reply