What do you think about treesitter? https://github.com/tree-sitter/tree-sitter T...

some-mthfka · on Jan 14, 2023

I like the fact of make people's lives easier. It's a universal solution which may be used in many editors: can't argue against reuse.

But is it perfect? I don't think so. Whatever is responsible for structural analysis should also be responsible for editing and all the extension capabilities, if you are to have a truly powerful system.

For me, the real problem lies in the integration with the client environment. Tree-sitter is fundamentally separate from the client application. That places questions on the introspection capabilities.

I mean, suppose your comment has structure. Suppose it's a markdown document, that comment in your code. And suppose you want to edit it that way: as a markdown document (or anything more complex). So you may want a markdown parser for that. Suppose you can write that parser in tree-sitter. Suppose you can even do structural-embedding. But how are you going to control and customize it? How are you going to write complex interaction policies between different structures? Embedding is where such seperation starts running into limits.

Why does this happen? I believe it's simple: APIs aren't enough. What you want is for the user to have access to the building blocks of whatever is being manipulated. And, so, keeping those blocks away from the user is never going to be that.

What's more, tree-sitter is still not really structural. That means it can't handle ambiguity well. For instance, in a lisp expression, you can insert a stray parenthesis. Paredit or a tree-sitter solution is going to tell you that your structure is malformed. A truly powerful structural editor is going to simply assume that parenthesis doesn't disrupt the structure around it, and so, it will simple keep track of it, and maybe it will find a match within the same expression at some later point in time.

Yes: a structural lisp editor doesn't have to care about a stray parenthesis. It just keeps keeping everything around it intact. It's a local ambiguity.

And, at last, the very important idea of structural editing is not even editing: it's the ability to treat structures as objects. It's the ability to present objects using a textual interface. Suppose you have a table structure, multiple layers of it. You need direct access to all that. Editing it should be easy, but it's really secondary.

Tree sitter doesn't deal with this.

I still see tree-sitter as useful, because you still need to deal with all the plain-text files out there. Perhaps, it could be used to import plain-text into a structural-editor representation, and then it may be, again, used on small subexpressions to identify their nature, incrementally, at runtime (but not the whole document, like what a usual incremental parser would do). So, that's the value of it as a parsing tool.

So, yeah, it's a useful tool, but when it comes to some advanced points of power-use and extension, it's not really enough.

Thanks for the question, it's thought provoking in many ways.