The problem with writing a DSL for every problem (and every programmer writing different DSLs) is that you end up with all these different languages with different vocabularies and different semantics. It's like a million different variations of Esperanto.
Using a programming language is like speaking in common tongue to every other programmer who knows that language. And most languages are so similar that most programmers can communicate even between languages.
Having code that looks the same, regardless of what it's actually doing, allows everybody to be able to read it.
>Using a programming language is like speaking in common tongue to every other programmer who knows that language
As much as I love DSLs, I Couldn't agree more.
>The problem with writing a DSL for every problem (and every programmer writing different DSLs) is that you end up with all these different languages
When writing in a DSL of my design, it is amazing. It feels like the program models exactly how I am thinking about it.
When modifying/maintaining a DSL made by someone other than me, often this can be horrifying, it feels as though this models my thoughts so poorly (and it does, it was designed to model their thoughts).
Programmers already know the DSL for every problem: English (or their native tongue). If they're writing programs for a specific domain without knowing the language of the domain, they have problems that won't be solved by using programming terms they already know.
How many people have learned what MVC means from Rails? Or the term "routing" in the context of a webapp?
One caveat I'll give: the DSL must be well-conceived. Writing a DSL that is dissimilar to both the underlying language and the domain vernacular is obviously horrible.
I couldn't agree more. As much as I like languages that facilitate DSLs, this is an unrealistic view of software development.
To add to what you wrote, you have to also consider that all developers in one team have to agree on a common DSL beforehand, which is an investment of time in itself, and the fact that if there is an unusually big change in requirements mid-development (which, let's be honest, is not a rare event), you might have to go back and change your "pyramid of DSLs" to accommodate it, maintain compatibility and update all your colleagues on the new languages.
The syntax required to use the API will be instantly accessible to new developers. Many programming languages provide semantic guidelines for APIs as well. So knowing the language an API is programmed in gives you a huge advantage in beginning to understand it.
With a DSL, all that goes out the window. The syntax of the language is potentially unknown to you. The semantics are an even deeper mystery. You're in terra incognito, trying to understand the mental process of the developer of the DSL before you can even understand what the code written in the DSL intends to do.
The primary goal of programming languages is not to describe how to perform a complex task to a computer. It is to describe how to perform a complex task to other humans in a form that computer can also execute.
I'm not sure I buy the whole syntax argument. I programmed in Forth for a while and it seemed like it was much easier to build up an understandable vocabulary than some of the C++ / Smalltalk class hierarchies I have seen. REBOL has dialects that work fairly well. I cannot help but think that DSLs are refinement of APIs the same way C is a refinement of Assembler.
When I talk to someone about what happens in a system it seems like a DSL flows from that conversation.
"It is to describe how to perform a complex task to other humans in a form that computer can also execute."
I think a DSL does that much better than a set of API calls, particularly when you involve non-developers. That is the point of a DSL.
I see APIs as a less extreme version of DSLs (in that they share a common language, while building a DSL frees you from almost any restriction), but I honestly don't have enough experience building APIs or DSLs to give you a satisfying answer. Sorry.
For a lot of languages, I agree, because there's no way to create a DSL as part of the language itself. Think of the difference as one of direction; are you fitting the domain into the language, or extending the language to fit the domain? Most languages are either general-purpose (C, Java) or their domain is fixed upon creation (SQL, Mathematica).
In a Lisp, you can use macros to accomplish things that go way beyond defining a domain-specific library / API. A typical library is restricted to the capabilities of the language, but a DSL in Lisp can extend the syntax itself.
Want to alter the reader so you have a way to express normal code as remote computations? Go for it. Feel like adding new math operators to use matrices instead of scalars? Knock yourself out.
Common Lisp itself did exactly this when object-oriented programming came along; using nothing but macros and closures, the Common Lisp Object System (CLOS) was built.
"and here is the spec for our in-house framework".
"and here is the 1M LOC codebase, which was carefully crafted to avoid using any of the complicated frameworks; all you gotta know is the C standard library".
You joke, but a tool I have seen lauded for being really "smart" at automatically generating doxygen comments does exactly that. I guess it at least saves the time of a human doing it by hand . . .
Actually this is how it was when I started at my current position.
We are doing web-testing in clojure, and in past year our little framework evolved into a DSL of a sorts.
On one hand it is great, we a have consturct like (with-client [name connection & body]), that behind the scenes sets up a virtual machine, registers it as a client of our web-service and destroys it after commands in body are done.
I can't imagine we would be doing setup and teardown by hand.
On the other hand it took me a month to get into it (two weeks learning the language, two weeks messing around with the framework) ... and there are still lots of things only our lead-tester knows how to fix, after our devs decide to change something we were relying on for testing.
I am curious, did you already know the JVM? When I learned Clojure, I was also learning the JVM, and for me the JVM was much harder than Clojure. So I would say it took me 6 months to learn Clojure, but much of that time was the time it took to learn about the JVM and all the related Java weirdness.
JVM? I knew Java, but most of what I did in java was using autocomplete in Netbeans and cursing Maven, when it didn't work.
What really helped me that, we had "programing paradigm" course in college, where I learned haskell for 3 weeks, what was enough that anonymous functions, map, reduce and using lists for everything didn't feel new.
If you start a sentence with "Welcome to our team" and end it with "See you in a month." whatever you put in between, your company is doomed, and your team is highly dysfunctional.
Using a programming language is like speaking in common tongue to every other programmer who knows that language. And most languages are so similar that most programmers can communicate even between languages.
Having code that looks the same, regardless of what it's actually doing, allows everybody to be able to read it.