This paper provides a nice overview of past and also very recent language developments. For example, support for bignums (Section 7.3) is one of the most exciting new features.
Thank you very much for putting this together, and for all your work on Emacs!
For example, if you want to use a map, you have three choices: you can use alists, plists or hash maps. There are no namespaces in Emacs Lisp, so for each of the three data types you get a bunch of functions with weird names. For alists get is assoc and set is add-to-list, for hash maps get is gethash and set is puthash, for plists get is plist-get and set is plist-put. For each of those types it is easy to find basic use cases that are not covered by the standard library, plus it is easy to run into performance pitfalls, so you end up rewriting everything several times to get something working. The experience is the same across the board, when working with files, working with strings, running external processes etc. There are 3rd party libraries for all those things now because using the builtins is so painful. In many ways it feels like programming some old web browser where you either need to patch the standard libary with 15 dependencies or learn complicated incantations for really basic tasks.
The ideas behind emacs and emacs lisp are great, so I really wish someone succeeded in creating a modern implementation from scratch and it gaining some traction, basically what Clojure did for Common Lisp. By the way even Common Lisp is way more modern than Emacs Lisp...
"... other than inertia one of the main arguments in favor of the status quo was that Elisp’s poor man’s namespacing makes cross-referencing easier: any simple textual search can be used to find definitions and uses of any global function or variable ... "
A few of the issues you mention can likely be improved, or resolved by using a better abstraction, or by deprecating features that are no longer needed. The Emacs community is typically very responsive and willing to discuss many issues in great detail. For example, if you run into performance issues or missing features, you can use Emacs itself to file a bug report, using:
M-x report-emacs-bug RET
However, some of the basic things you mention are in my experience quite straight-forward. For example, with only a few lines of code, I can copy the front page of HN as HTML text (in fact, the whole HTTP response) into the current Emacs buffer:
(let* ((host "news.ycombinator.com")
(proc (open-network-stream "HN" (current-buffer) host 443 :type 'tls)))
(format "GET / HTTP/1.1%sHost: %s%s%s" eol host eol eol)))
give a flavour of this (and of not having keyword arguments, of libary functions relying on global variables, ...).
This is a very hard problem to recover from, because backwards compatibility is so important for something like Emacs. Breaking old versions would be completely unacceptable, which makes it very difficult to remove any features.
One of my emacs experiences is playing "escaping the regex" about once annually, where I need to use backslashes in a regex, but first have to escape an emacs string.
I'm not very good at elisp, so this is what a basic begineer regex looks like to escape a ' in a shell (so, I want the bash command to go from ' to \'). The regex in emacs I came up with was:
(replace-regexp-in-string "'" "'\\\\''" $)
Now maybe there is some sort of shortcut for passing a string straight to a regex that will make me look silly, but for a text editor with emacs' power and flexibility to make a regex look that hard bought a smile to my face.
(regexp-quote "\\") ⇒ "\\\\"
(rx (or "a" "b")) ⇒ "[ab]"
(rx (and (or "a" "b" "test") "c")) ⇒ "\\(?:test\\|[ab]\\)c"
Elisp is odd to me, because you need to get used to thinking in terms of buffers of text. Most commands require a trip through a buffer. Which is not at all bad, but is very foreign to my expected view.
That said, once you get used to basically programming how you interact with a buffer, it does get very fast very quickly. And can often read rather nicely.
The second way is to have a model that is so strong that it is worth the user learning that model (this is a rarer approach, but is more common in the lisp families - Clojure, for example, does this with variables). If elisp has this, it would be a good idea for someone to mention that in the documentation, because I don't recall ever seeing anyone say "Wow! This elisp model of processing text is amazing! I've been doing it wrong my whole life!". I have seen variants of "which is not at all bad, but is very foreign".
I don't know of a best language, but manipulating text strings is the foundation of most web serving. So I'd say that the popular languages for that (python, perl, php, etc) are quite a bit better than elisp at processing text.
Once you get used to manipulating buffers, elisp actually is easier than most other languages. What it is bad at is processing strings. Because it is good at processing text buffers.
I am on my phone, but if this thread is still going this evening, I'll show roughly what that snippet would look like. Will be, roughly, (forward-word) (move -1) (insert "/"). Obviously, some differences if you were moving words or paragraphs. But basic idea is the same.
That is, don't try and manipulate strings. Manipulate the entire buffer. If needed, build a new buffer with what all you want.
The irony is that elisp has a decent dsl for manipulating text. But you have to embrace the mutable nature of it. Which is unexpected for many working with a lisp.
TXR Lisp allows various classic Lisp operations to also be used on vectors and strings (think car, cdr, mapcar* and numerous others). It has array referencing that support Python's negative index convention. Sequence subranges are assignable places, so if s holds "firetruck" then (set [s 1..-1] "lic") changes s to "flick".
TXR Lisp is loaded with useful features, all packed into a small executable accompanied by a small library of Lisp files.
I think most of these are supported through generic functions. Though, I think moving beyond alists is something you probably only ever have to do once you are doing some heavy lifting programs. In which case, I am curious if elisp is the best place to write that program.
Your list of things that are painful in Elisp: working with lists and maps? For real? Opening files? `find-file is hard?
As for hard to find out how to accomplish basic things what about the extensive in-editor help system? C-h f for functions, C-h a for apropos and C-h i for info? It would be hard to find a better documented software system anywhere.
Sure you need a few libraries to make things better, but the fact that there is a healthy ecosystem of packages is a good thing.
Maps are fiddly in Common Lisp too, all the data structures feel a bit crufty compared to Clojure.
Elisp is being improved with every release. Lexical scoping was added without much ado; this release got concurrent threads, radix trees. It's not that broken we need to throw it away and start again. IMHO it's not broken at all.
Rewriting to modernize would effectively kill emacs lisp.
It reads like fortran77 to me.
(--map (list (substring it 3)
(aref it 0)
(aref it 1))
(magit-git-items "status" "-z" "--porcelain" "-u" "--ignored"))))
it's not a new implementation - it's a fully incompatible new language. Zero CL code runs in Clojure because the operators are incompatible/different and any existing CL code needs to mostly re-architected, since Clojure has a very different approach (more functional, less object-oriented, less state, lazy data structures, no linked lists, hybrid libraries /language partly using hosted language, ...).
The project seem dead in the water though.
As a stand-alone language it’s pretty weak, even compared to other lisps. Default dynamic scoping comes to mind.
As it is, I've found that you can do a lot with just vi, tmux, and bash (along with all of the utilities).
Isn't this what neovim is for?
I'm thinking of something altogether different: you leave the editor alone and you script the operating system around it, using the editor only as a flexible control interface for everything. This is the original vi philosophy. It means you don't have to rewrite anything. Any new program you install, if it can send/receive text on STDIN/STDOUT/STDERR, then you can use it in vi.
So what's wrong with vi? It has a few shortcomings: no unlimited undo, no ability to pass the cursor (row,col) position to an external command, and working with multiple buffers is more painful than it needs to be.
Users will need to rewrite many things, if they use multiple platforms.
> Any new program you install, if it can send/receive text on STDIN/STDOUT/STDERR, then you can use it in vi.
Except even the unix toolspace is not cross-platform, despite POSIX.
Minimalism is good, but the OSes failed to provide good, composable, cross-platform tools.
By the way, Neovim's :terminal command enables more scripting and re-use of the many unix tools. w3m and lynx are much more useful if you can overlay editor keybindings over them instead of writing a pile of code to use their (insufficient) CLI APIs.
For example, every time you press a key (or key sequence) that results in any action, Emacs internally invokes a function (most of them written in Elisp, some of them in C) that achieves this action. So, when you want to automate a task and have a rough idea how to achieve it by pressing key sequences, you can inspect their associated functions and their source code, and study how they are called and even how they do it.
As a quick start, you can—using Emacs itself!—display a list of all functions that are bound to any key sequence. To get this, simply press:
But then, your next question may be: What did C-h b actually do? That is, which function did this actually invoke? Of course, one straight-forward way is to simply search for "C-h b" in the buffer that shows all these key bindings, because you know it must appear there. Another way is to use C-h k C-h b.
But then, what did C-h k actually do?
To find out, we apply C-h k to its own key binding:
C-h k C-h k
At this point, it is essential that you also have the (Elisp) source code installed. If you have it, then you can simply browse the Elisp definition of describe-key, and see how it achieves its task.
The same goes for every other function you may be interested in: You can browse its definition, copy it (for example) into the scratch buffer, change it, evaluate it and run it. You will find that not much is needed to get started with Elisp, because many tasks can simply be written as a sequence of standard commands of which C-h b provides a good first overview. You can also press C-h l to get a list of the last few input keystrokes and the invoked commands.
In addition to that, to automate simple tasks in Elisp, you often only need a few simple commands that let you fetch text from any position in the buffer and to insert it (see for example the functions buffer-substring-no-properties and insert), the ability to launch external processes (the "Processes" chapter in the Elisp info documentation is great) and a way to perform editing operations in a temporary buffer (see with-temp-buffer). To find out more about these functions, simply use C-h f.
And, of course, you can simply use C-h k C-h f to see what C-h f does!
One essential point: To make Emacs keybindings more convenient, I recommend to turn Caps Lock into an additional Ctrl key.
The only thing really good about elisp is that it is integrated into Emacs.
But that's really, really good compared to most programs, which are customized by dialog boxes or INI files. Imagine how great it would be if you could script your web browser using some semi-decent programming language, and your scripts would work 30 years from now.