Hacker News new | past | comments | ask | show | jobs | submit login
D2: A new declarative language to turn text into diagrams (d2-lang.com)
295 points by snikolaev 7 months ago | hide | past | favorite | 126 comments

The last question in the FAQ makes it clear that this can only be used in the context of their account-based online tool. Their free offering literally says "3 diagrams". Above this, the FAQ tries to compare their tool to Mermaid, Graphviz, PlantUML which are all open source, locally runnable, etc. Their comparison totally skips the fact that, because D2 can only be used in their service, a broad number of use cases (including programmatically generating diagrams inside some other application) basically aren't supported.

Seems like it could have been cool, but likely will not seriously try it because of these limits.

This is a complete non starter for any regulated industry.

Shame. Mermaid works almost okay and often looks like garbage. I wish there was an open source effort somewhere to do better, or even a paid locally runnable tool that has no service connections.

It's why I keep falling back to excalidraw over tools like mermaid with auto-layout. When I need to communicate an idea, often I really need to fuss with layout in order to get my point across, and having the control to drag things around is the only way to get there.

Problem with non automated tools is that editing becomes a big issue since you have to reflow big parts of the diagram to make an addition. This is always my biggest issue with any diagram, the fact that they are evolving.

Oh of course. This is the tradeoff.

That said, over time a diagram's value will diminish regardless if you have auto-layout or not, since the codebase will always outpace documentation.

It won't work for every use case but https://flowchart.fun is one open-source alternative

Ilograph [0] is a similar (and, in my biased opinion, better) tool that has a 100% local runtime option (called Ilograph Desktop). You can still test-drive it with the browser version, of course.

[0] https://www.ilograph.com

Those tools are mature 10-20+ year and we're just getting feedback for our little ol' alpha born ~yesterday. The limitations are temporary -- we are working on an offline, locally runnable experience (it's in the roadmap), but, alas, engineering bandwidth of a 6 person team.

edit: I've updated the docs to make this clear

Maybe starting this way makes sense.

But when the project is trying to compare itself to others, it's only fair to compare based on what's true of the project currently vs what's aspirational. I would guess that for many potential users, the temporary limitations trump most of the stuff in the comparison portion of the FAQ, or the "current shortcomings" section.

And the "Getting Started" section could also say "create an account and familiarize yourself with the existing terrastruct product" or whatever.

How is this different/better than, say, Mermaid?

For one thing, the output looks lightyears better.

You can fully modify styles in mermaid using CSS and style attributes. You can also load it as a JS library, attach events to chart components, programmatically render charts plus way more.

Wow, thanks for the clarification. Totally missed this part. After having already evangelized this with my coworkers, I may need to take it back.

If they made the core open and allowed users to use this language independently of their online service, there's still a ton of value they could have added with premium services on top of this. For writing the docs, sharing with teams, comments / versioning, etc. Visual figma type features essentially. A team bought into the specification would very likely pay for this.

But it's hard to imagine it taking off being completely locked into their service. Ah well, a fine idea and looked pretty well executed.

EDIT: maybe that's on their roadmap "When the language is more stable, we intend to open source D2. This will happen sometime in 2022.". Ok, back on board

Looks like it will, just not yet:

> INFO: It's currently in alpha. During this time, it's only housed on Terrastruct, which provides the most well-supported interface (IDE) for it. Once we have a stable v1, we'll release and continue development of it in open source and make local options available.

Also, to me this was a reminder of the importance of context and assumptions in good communication. Never having heard of this org, I clicked the link for this post, saw the top-level blurb describing the project and assumed that a 'language' which generates diagrams from declarations would be something I could _run_. I clicked on the 'Getting Started' section and was annoyed that there was no 'Installation' section. I started from a context that inclined me to think of this as a stand-alone project which ought to have certain characteristics.

I think the author of this page started with the assumption that of _course_ given their business and product, a 'language' for diagrams is only giving users access to something that maps on to whatever internal representation their tool already uses, and allows technically inclined users to work with text and fewer clicks. To them, the assumption seems to have been that readers will already be users of their product, and the assumption that the 'language' is built into their tool _and nowhere else_ is so far in the background that it's only somewhat obliquely addressed as the last question in a FAQ.

People seem to be missing that this is a preview, not a product (yet). And that: "When the language is more stable, we intend to open source D2. This will happen sometime in 2022."

In D2 Tour video, at 4:30, they say

> When it's more stable, we will be open-sourcing the language.

If thats the case, that only the language will be open-sourced, not the visualization, then I'm not entirely sure what that really gets you, beyond perhaps alternative editors?

But its the layout algorithms that are actually notable.

Terrastruct engineer here. The visualization will be open source though I can't say for sure yet whether it'll be with our layout algorithm or with https://github.com/dagrejs/dagre.

Perhaps then "Preview/Beta online service" should be part of the Headline/title.

Perhaps reading the article is a useful way to get information? If the headline contained everything everyone would conceivably want to know, it would be the article. It's not like the headline was misleading: "a new $tool" is a common phrase for software in alpha and beta stages.

So, to you, "Tool" or "Language" do NOT imply that you can use it independently of a specific online service?

I mean, I agree, that "new" is marketing speak for "alpha/beta", but -to me- there is a difference between a "language/tool" and a "Single-Vendor-WebService". Or is my understanding of the semantics of the English language outdated?

Not even outdated - just incorrect. There have been proprietary tools longer than there have been computers, it's the point of patents - grant a monopoly on your proprietary tool for X years in exchange for making it's workings public knowledge.

Even patents aside, why wouldn't this D2 thing qualify as:

    a piece of software that carries out a particular function, typically creating
    or modifying another program.
(source: google "define tool")

While it isn't creating or modifying a program, I'd consider "parsing text in a specific syntax to create a diagram" more like a compiler than a site that just provides information or crud sites with some forms. I'm not entirely certain how where it runs affects it's tool status, but it certainly is a consideration to be made while evaluating it for any particular use case.

(also from define tool:

   a thing used in an occupation or pursuit.
and depending on where you stand on the "thingness" of software may also apply here, but that feels diversionary)

I can think of quite a few single vendor languages - various IBM mainframe languages like JCL, K, matlab, multiple .Net and MS languages until recently, applescript, etc.

Agree. To be honest, I'd find it useful if it were closed source but native installable tool that can convert text to diagram which then could be exported to image/pdf etc.

Yup. I use many paid, local tools. And because there’s no network service nonsense it’s not a problem at work.

Would you believe me if I said this was just an ad and you wasted time on it?

Just because HN isn't Reddit doesn't mean ads aren't thrown in with all the curated "content"

May be an ad but this is something I would actually use if it wasn't paid.

Like other said, maybe it will inspire someone to take this and make an open-source version

If it was open source with a permissive license I'd start paying immediately, like I have with other projects.

> if it wasn't paid.

Maybe you intended to say 'if it wasn't closed', or similar.

> Mermaid, Graphviz, PlantUML which are all open source, locally runnable, etc

And there's C4 Model for Visualising Architecture [0] that has upcoming tool support, for instance Structurizr DSL [1]. It has an online editor [2] that can export to Mermaid, PlantUML and other formats. And has various open source repo's, like this Java codebase for the DSL itself [3].

[0] https://c4model.com/

[1] https://structurizr.com/help/dsl

[2] https://structurizr.com/dsl

[3] https://github.com/structurizr/dsl

Yikes. This looks like an amazing idea, but the online limitation kneecaps it.

When I want to make diagrams graphically, I use diagrams.net. when I want to make diagrams from code, I use PIC. What else do people use? PIC works pretty well, but it has its limits and it's ancient.


Is a modern implementatiom that's pretty nice.

could you share a link to PIC ?


An alternative implementation with good docs.

Here is the Wikipedia article, maybe a good starting point.


Agreed! Interesting idea, but charging for this is not going to get them very far.

Gotta find a way to do an "open core" or similar.

A simple check for an infrastructure tool (and a programming language is a king of infrastructure tools): does it have an open-source, unencumbered implementation?

If not, I won't use it for anything serious.

Good to know. Thanks for the list of alternatives.

Services as a software substitute...

Hi all, creator of D2 here. I posted this on Reddit to get some feedback but was hoping to improve the docs and offer a playground environment before posting on HackerNews. Ah well. I wanted to clarify some things:

1. D2 will be open source and usable outside of Terrastruct. Terrastruct will remain the best interface to D2, with bidirectional updates from GUI, but we already have vscode and vim plugins ready for local editing. We're a small team and working on one thing at a time.

2. We're aware of PlantUML, Graphviz, Mermaid, of course. These are mature offerings, but I see plenty room for improvement that we intend to tackle with D2. https://d2-lang.com/tour/faq#how-does-this-compare-to-mermai...

Feel free to ask any questions!

It seems that open sourcing it while it is growing up actually makes more sense. If you want community involvement, ideas, testing, contributions, and so on, then hiding the project source is antithetical to why most people participate in open source in the first place.

Open source can mean more than just a license. It should mean more than just gifting a code base once you think you're done.

I don't mean to come across as bitter or entitled. I just question the value of waiting till the last minute. For now, the promise only stands as a way to pull people into the fold. I've seen this kind of promise many times before in projects much like this.

thank you for the advice, I do appreciate it, as I'm no expert in open source. one thing to clarify is the idea that we're tying a bow and chucking it across the fence. we'll be far from finished when we open source d2. the intention was to internally seed the initial design (of both the language and the code) -- the first minute of building -- quickly, within a small team. i'm really grateful for all the interest today from everyone, hopefully HN will let me post the repo to update people shortly (w/o considering it a dupe).

The guidelines for dupes require "substantially new information", IMHO the source absolutely constitutes substantially new information and I look forward to seeing it - I think your video demo was very compelling.

It reminds me of that adage, "Start your business, the tool you build to run your business is now your business"

Nice intro video on the landing page. I liked the capabilities shown.

I do think you could have made it obvious in the Getting Started section that this is not a local runnable language. I spent some time looking for how to install it only for me to realise that it is some form of SaaS.

Thanks! The caveat is on the intro/home page. We very much intend to support a high quality locally runnable experience before we'd call it a v1.

By the way do you have a BNF grammar (or something more of a formal spec for this?). The cheat sheet does look good though.

No unfortunately not yet. The docs are like 75% caught up to all the language right now and it's still changing not infrequently, so we'll make some formal specs once it's stable. This reveal caught us off guard =).

> I posted this on Reddit to get some feedback

> This reveal caught us off guard


hah, so the line of logic was: "let's post on reddit to get some low stakes feedback in preparation for a HN launch"

I can buy this reasoning. This crowd gives solid, tough, but highly constructive feedback.

I, like others, will keep an eye out for when it becomes open source mainly to see which license.

Will you be able to make computer network diagrams? None of the existing open source programs are very useful for this mostly because you can’t put text at the beginning, middle, and end of connectors.

we only support 1 label for connections right now, but that's a good suggestion. i'll put it on our roadmap, ty!

> D2 will be open source and usable outside of Terrastruct. Terrastruct will remain the best interface to D2, with bidirectional updates from GUI

Any honest open source project would not say such a thing categorically, and would welcome and support community improvements that take it to an even higher level.

But instead it sounds like you will rig the game, with "open source" just meaning "let's get people to work on our proprietary product for free, and we can fly the banner of open source while we're at it!".

The syntax reminds me a bit of PIC. Is that coincidence, or intentional?

coincidental, i actually thought i'd researched all the available options before embarking. can you share a link to an example? I found the wikipedia but haven't found what the syntax looks like

Okay, maybe it's not as close as I remember it!

PIC is designed to be embedded inside of TROFF, so it's... not pretty, to say the least. It does have the -> arrow thing, and the label: value syntax, though. For as hideous as it is, it's surprisingly easy to use.


Those probably descended through PlantUML

the -> probably descended from cave paintings

I am taken a little aback by the name. There is, of course, already the D programming language, which has even had a major version number of 2 for quite some time.

It also brings to mind the JavaScript library d3, which, while not strictly for making diagrams, can easily lend itself to the purpose.

Calling this thing "D2" seems potentially confusing.

We have:

"D" the programming language [0]

"D" the data language specification [1]

"D" the programming language for DTrace [2]

"D3" the javascript library [3]

"D4" library/tool for Declarative Data-Driven Documents[4]

"D4" implementation[5] of the data language specification[1]

Overall, I think "D2" is objectively the best choice here. We have at least three "D"s, two "D4"s, and one "D3", so it makes sense to put it in as "D2". I certainly wouldn't want another "D" or, heaven forbid, a "D5".

[0]: https://en.wikipedia.org/wiki/D_(programming_language)

[1]: https://en.wikipedia.org/wiki/D_(data_language_specification...

[2]: https://en.wikipedia.org/wiki/DTrace#Description

[3]: https://d3js.org/

[4]: https://github.com/joelburget/d4

[5]: https://en.wikipedia.org/wiki/Dataphor#Languages

As a gamer, after reading just "D2" my brain immediately assumed it's about Diablo 2 Resurrected.

I thought of the D-2 extract on Reserve in Escape from Tarkov.

"D1" CloudFlare's database thing

Calling it now, someone will make a tool for automatic documentation of R codebases, and call it R2D2.

I wish developers would stop with the one or two letter names for their products. In most cases there is already another product with the same name and you just cannot do a websearch for it without a hassle.

https://plantuml.com/ is free and nice. Plays well with Markdown and C4 diagrams. The only UML-specific diagram type I use is the sequence diagram but that is very useful IMHO.

It can be a bit clunky setting up the jar (or hosting it yourself), but it's still the best worst option. It scales pretty well and can draw some very complex flows, while still giving you the best in just plain ole text editing.

I could have never maintained the diagrams I used for other developers on my teams or folks on the audit/regulation side of thing without it.

The default styles are really unprofessional looking and I think a lot of folks look the other way once they see that. C4 diagraming with PlantUML is also a breeze for systems diagrams.


I use https://github.com/yuzutech/kroki in my github actions to build plantuml and other diagrams.

Also playing with 11ty and this to replace our home grown tooling based on markdown with embedded dot.

Have you seen https://sequencediagram.org/ ? No JAR needed, just a web browser

Here is a github repo embedding PlantUML in the Markdown: https://github.com/jonashackt/plantuml-markdown

Mermaid also does this, and is integrated in github/gitea/gitlab.

To try it out, just open a new github issue and add

      graph TD;
Or try the live editor: https://mermaid.live/

Mermaid has heaps of problems. I discovered this when I tried extending it myself.

Firstly, neither its parser and its visualiser can be extracted from the library. You can't process a grammar on a backend and then render out an SVG. You can't generate an SVG as part of a CLI tool - not without spinning up a web browser in automation mode.

Secondly, the grammar is messy. Really really awkward. Flowcharts are OK because they were the first Mermaid usecase, but other grammars are quite weirdly written, brittle, and try to be a mismash of other DSLs. Which in turn leads to very awkward token choices to avoid stepping on the toes of parts that are UML-like, parts that are DOT-like, etc etc

My impression reading Mermaid's issues is that it was built from scratch by a JS developer who hadn't any previous experience with DSLs, parsers, or language design, and largely learned as he went along. That's acceptable for a proof of concept. But as a finished product Mermaid is a lumbering mess

A problem with semantic markup is when it doesn't do what you want, and you need to wig-wam it. Does this mean you actually wanted presentation markup? Or, since you are trying to convey meaning, does it mean that the semantics aren't rich enough to describe what you mean?

In this D2 example (https://d2-lang.com/assets/images/intro-example-a917149ff3b7...), the diagram is nicely designed to be centered on the nexus node crawler. But the choices of which side the tributary nodes are on is not. You might want cron below and ps->express below - or to the sides.

The grammar can be extended to accommodate this (maybe already has been), but what is the semantic meaning of above, below, left or right?

One semantic choice is made: the "persists" arc is unidirectional, and is presented left-to-right - a natural order for many languages.

(Technically, "declarative" needn't be "semantic", but arguably is the most useful one)

If there is semantic meaning in where your options are (or you want to assign it anyway) then an automatically generated diagram is probably a bad fit.

before introducing anything, I'd like to see more instances where something needs to be on one side of something else, which I haven't found to be the case in software architecture diagrams.

It's more a research question.

I think it'll be more like ordered keys in JSON - not ordered, according to the spec, but very useful in practice, because easier to compare, locate keys by eye etc. This order might not merely be "the same" as the input happened to be, but an order customarily used - and there might be a semantic reason for that order in the first place, used in some original, non-JSON, representation.


Having a diagram drawing tool that automatically lays things out in an elegant fashion by default is really wonderful -- especially when I am just getting started on any given project.

But if that diagram drawing tool does not graduate to allowing me full control over every detail of the output (albeit that might be verbose or awkward) then it cannot grow with me as my requirements become more complex and it is good only for toy-sized or throw-away projects.

It's bidirectional if that helps. That's the outstanding feature to me.

See also:

https://structurizr.com/ - diagrams as code

https://c4model.com/ - C4 model for visualising software architecture

The consistency of structurizr + c4 is great for sharing information across scaled teams of teams.


The XState Visualizer generates nice looking statecharts.


Click See an example in the left pane, then click Visualize at the bottom of the right pane.

On the roadmap are visualizations for XState's actors facility, i.e. multiple statecharts interacting with one another.

If you login, you can save/fork statecharts and get shareable links for them. There doesn't seem to be a way to export the diagrams (as PNG or SVG) but maybe that will be added in the future.

A declarative language for diagrams without so much as a passing mention of Graphviz, or even Mermaid.js anywhere in the docs?

It does look like a material upgrade from Graphviz, but even so...

graphviz svg output can be (with some tricks) CSS-styled + JS-animated, so the sky is the limit w.r.t. how fancy you want to make you DOT graphs, and their elements.

actually the effective limit seems to be around 50 nodes

Limit of what exactly? I've made much bigger than 50 nodes SVG-graphs using GV, and styled them with probably 50+ CSS styles and animated more than 50 nodes.

So, could you please elaborate?

sorry...I do language development sometimes and GV (dot) has been a godsend for debugging. but at a certain size it just goes sideways and stops being useful and is more of a distraction. to be fair I think this more of a fundamental problem about visual understanding than some screwup in GV

50 nodes may be close the limit of what a human can understand.

> D2 has no diagram types. In other tools, you specify "this is a class diagram", or "this is a state diagram". One of the goals of D2's design is to have minimal syntax that you compose with.

I think this can be really difficult...?

I was fiddling with RDF and N-ary relationships some time ago, but the easiest way to express a complex relationship is to pre-define the whole relationship as an entity, and relate it to actual entities. This ends up with something similar to function call (e.g. score(subject: playerA, object: playerB, using: weapon, when: time)) or RDBMS tables.

But "no DSL-in-DSL" should be a right direction. The core language should be carefully designed to allow modular approaches.

indeed, i don't know if it's a principle that can be maintained as we scale features. for example, even sequence diagrams, i'd like to just be automatically detected, e.g. if you have connections numbered in order that looks like the structure of sequence diagrams, it'll render as sequence diagram. we're testing to see if there's any blocking scenarios where inference just cannot get it right

I've used Plantuml extensively, and still use it a bit, now and then. I don't remember ever needing or even wanting bi-directional editing. There's a lot of interesting things in terms of syntax and semantics here, but I really do not see WHY they are tackling a very hard problem on top of an already not easy one.

As a counterpoint, I also use PlantUML a lot mostly with C4 and I thought the bidirectional interaction looked very cool. I think one benefit might be getting those who don't think in specific syntax and code involved in progress elaboration of the diagrams by using the GUI portion.

a low-hanging motivation: how do you choose colors for nodes?

I don't. I just care about boxes and lines, most of the time. When I need to differentiate between types of boxes, I use shapes or stereotypes.

It's interesting.

This notation is pretty similar to the notation in the FOSS tool (admittedly that I wrote) over here: https://stonecypher.github.io/jssm-viz-demo/graph_explorer.h...

Fortunately, no need for accounts or payment there.

Time for someone to copy the good parts and implement them as FOSS. Good luck getting enterprises to pay for diagrams.

At the end of the video the presenter mentions that they plan to open source the language when it reaches a greater stage of maturity.

A code dump does not make it an open source project. Sustainable OSS projects are developed publicly.

That's probably why they're doing a code dump

The language spec, or the rendering backend code? And even if the "complete package", if the "locally runable version" is an afterthought of the OnlineWebVersion, have previous situations like that not regularly resulted in *Paks wrapping an electron instance with some blobs under the hood?

Lucidchart has entered the chat.

Unfortunately the real power ask is creating these offline so they can be embedded and shared in other contexts. I don’t have confidence in gating by a web service. Others have come before you and tried

Time for a shout-out for (in my opinion) the coolest and most useful free tool on the web: https://sequencediagram.org/

Use this all the time for my work and the ability to share a diagram with a url is a killer use case

Is there a declarative language or framework for creating ad-hoc GUIs that consume structured data from stdin stream and spit-out a GUI?

Like feedgnuplot [1] but not only restricted to graphs.

[1] https://github.com/dkogan/feedgnuplot

Looks cool. Any inherent limitations or performance issues with graphs that are very large? (20M+ elements)

Unless you use instanced rendering via some GPU API it's going to take some galaxy brain code to make that work.

We do webgl rendering of big graphs as part of Graphistry, and before the 20M nodes break the system, the ~10X more edges will ;+)

But more seriously, a careful webgl implementation gets you maybe halfway there, and most I've seen break well before then. It gets tricky with browser js memory limits and raw perf. We are looking into hitting more 100M level with interactive speeds... But takes creativity & opts I wouldn't expect for a diagramming tool :) and, if your kind of thing, we are hiring :)

Forget about rendering, just doing the layout would be extremely challenging.

Some of the graphs (near the bottom) of the GraphViz example page are quite large: https://graphviz.org/gallery/

And there's a link to a gallery with even bigger ones, so #NumberOfNodes alone does not seem to be the main limiting metric. Maybe combined with the choice of rendering layout.

Why uh. Would you expect that to work?

How much of your life were you planning to dedicate to scrolling from one side of the image to another?

I think he expects to be able to zoom out.

Yup for zoomed out views. It’s handy when visualizing large websites.

we're going to have to fallback to stable diffusion for that one

It's not a language. It's a service. Since diagrams like this need to be embedded into doc it's not good at all for Dev documentation.

Sadly, any start-up tool firm can't have any other business model other than subscription.

Isnt subscription the most win win revenue model?

It is. But, someone can come and disrupt that too

Something was posted here recently in a similar light - does anyone recall what it was?

there are a few of these tools but https://news.ycombinator.com/item?id=31273777 was a recent reference to https://mermaid-js.github.io/mermaid/#/

This was up yesterday https://news.ycombinator.com/item?id=32637689 It's similar in that it's a diagramming DSL

Does anyone know what the clipboard tool that was being used in the youtube demo is?

I believe FlyCut.

Does this use Stable Diffusion?

Excellent question.

Hoping the community will work more on declarative 2D diagramming methods and tools.

Is this project basically the New Mermaid? Why not go in a new direction. People want diagrams, not just networks, not just series-parallel graphs. Anyway, ML is changing everything. Can Stable Diffusion make crossings at right angles? Just wondering.

Very cool, looking forward to the open-sourced version.

What was used to author/render these docs?

Looking forward to the open-sourced version.

Applications are open for YC Summer 2023

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact