
Emacs Lisp's Future - rutenspitz
https://lists.gnu.org/archive/html/emacs-devel/2014-09/msg00434.html
======
mark_l_watson
If it isn't broken, don't try to fix it.

I am an old guy (professional Lisp developer since about 1981) and my age
probably affects my opinion:

It would be a disaster to mess up the Emacs ecosystem. I don't think that
Emacs/elisp runs slowly and since I have to use so many different programming
languages anyway, needing to know a little elisp is no problem.

I don't care if elisp is not a modern language.

Way off topic, something that I have written before about: think forwards
several hundred years. What will the software landscape look like? My bet is
that there will be many ancient software systems that have been debugged to
near absolute stability over the centuries. Sure new software will be written,
but I bet there will be many very old and stable systems that will see little
change.

~~~
lispm
Wouldn't it be nice when Emacs would not be blocked when some Lisp routine
runs?

Lots of people have written excellent code for GNU Emacs, but the
implementation runtime hasn't improved that much.

~~~
e40
_Wouldn 't it be nice when Emacs would not be blocked when some Lisp routine
runs?_

Yes, it would be nice, but can you imagine the bugs that will happen while
everyone works out the details of how to do it properly?

~~~
taylanub
Even in the current pre-alpha/alpha stage of Guile-Emacs, I can launch a
thread from Scheme, do work, return to the main thread, then call an Elisp
function on my result. (Scheme and Elisp data types are unified.) No bugs or
unpleasant details there.

Calling Elisp functions and accessing Emacs data types (buffers, windows) from
multiple threads is another issue; if you don't want to bother with it then
don't; you still get all the other benefits of being on Guile. (Calling to
_any_ Guile module agnostically as if it were an Elisp library (including,
say, Guile's OpenGL module), having an FFI, getting JIT or AOT native
compilation in the future, etc.)

------
yason
Given all the reworks of software projects that have a) been started b) then
delayed c) then delayed more, "because this time we'll make it perfect" and d)
then failed to gather traction because of incompatibilities, it's my humble
guess that the only way would be to continue ELisp as it is and slowly rework
the compiler/interpreter and the runtime into a more modern codebase, and add
missing features such as thread/process based concurrency. Guile, Clojure,
Common Lisp or whatever language are going to basically restart Emacs from
scratch, no matter how well they're coated with compatibility macros.

It's better to create slight incompatibilities gradually (so that packages can
catch up in some humane timeframe). I could imagine the ELisp runtime being
intertwined with the Emacs C code badly enough that making radical changes to
it would seem "impossible", just like in the CPython codebase it is considered
impossible to ditch GIL and modernise the code. But those kind of comments are
often excuses because people are lazy and it's nicer to write new code.

~~~
JulianMorrison
If I was them, I'd gradually introduce the minimum set of breaking changes to
ELisp that converge it on Common Lisp. Even if those changes were unhelpful or
removed features. Then eventually, when it maps 1:1 onto a subset of CL, just
swap the implementation for a supported CL.

~~~
mjn
Some years ago the a clisp maintainer actually made a minimal demo of Emacs
hosted on clisp with just a compatibility layer for Elisp via
macros/functions, which seemed to work pretty well. But it didn't progress any
further because Stallman vetoed basing Emacs on Common Lisp (he considers CL
to be too big a language, and to some extent just doesn't like it). In
retrospect I'm not sure this was a good call: Guile as a general VM with
custom infrastructure to support both Scheme and Elisp is not really a more
clean, minimalistic infrastructure anyway. The semantics of Elisp/Scheme are
too different to be able to just stick in a Scheme implementation with a
smallish compatibility layer, like you could with Elisp hosted on CL.

~~~
JulianMorrison
Too often it seems RMS is why GNU can't have nice things.

~~~
fleitz
RMS would never let practicality get in the way of ideology.

The GNU/RMS is best because you're free! (As long as your computing needs are
a text editor, compiler, and some unix system utils)

~~~
rurban
That's why I believe their FFI plans are doomed.

We implemented FFI's for emacs over the last decades, but RMS always strongly
opposed it, because you could call windows DLL's then. gtk-emacs e.g.

~~~
deng
RMS is fine with an FFI as long as you cannot use it to inject non-free code.
This is usually done by requiring an exported symbol saying that the code is
GPL compatible. GCC does it this way:

[https://gcc.gnu.org/onlinedocs/gccint/Plugin-
API.html](https://gcc.gnu.org/onlinedocs/gccint/Plugin-API.html)

~~~
rurban
But you cannot call that a FFI then. A FFI is the abbrevation for "Foreign
Function Interface", not "Friendly Function Interface". FFI's are not sugar
coated.

------
hraberg
I started a project to port Emacs (the C parts) manually to Clojure, with the
Emacs Lisp automatically transpiled into Clojure in 2012, most of the work was
done in early 2013:
[https://github.com/hraberg/deuce](https://github.com/hraberg/deuce)

It can boot Emacs on the JVM and take simple commands, but not much more. It's
my intention to eventually revisit this project, but not sure when.

------
atgreen
"Lack of some features, most notably FFI and concurrency."

Cool -- my '90s hacks are in demand! I added threading support to Guile and
wrote libffi back in the 90s.

~~~
slacka
It seems your libffi has been used in Guile since 1.9.8, so does that mean
Guile does support FFI and concurrency?

~~~
craigching
Guile does support FFI and concurrency.

------
TeMPOraL
I've been bringing the following thought up inside various threads but I think
it needs to be said in separate one, since I see many people here and
elsewhere missing this about Emacs.

Emacs is _not_ a _scriptable editor_. You don't "script it" or "write plugins
for it" in a classical sense of those terms. You reprogram, extend and augment
a piece of running code on the fly. Seems similar, but feels different. Emacs
is a live Lisp image with text-editing features bolted on top. It's basically
backwards of how a typical editor/IDE is implemented.

Therefore, it matters what language is used as a base. People mentioned Lua or
JavaScript, but they are nowhere near useful enough for the task.

Therefore, it feels to me - and I believe to many other Emacs users as well -
it matters that there should be _one base language_. Emacs as an Elisp system
with text editing capabilities _feels_ like a whole. Everything fits together
nicely and interacts with each other. It is elegant. Aside for inviting
maintenance upkeep and general chaos, making Emacs "run" multiple languages at
the core is sort of like shattering its soul into many pieces. I don't want to
have an editor with multiple-personality disorder.

Imagine you're writing an executable in three different programming languages
mixed together at the same time. That class is written in Common Lisp, but
it's child classes are written in C++. And exception handling everywhere is
written in Python.

The sheer mental effort to make all of these work in a conceptual harmony
inside a single program would be enormous. And it would still feel weird.

That is what multiple-extension-language Emacs would feel.

~~~
lispm
> It's basically backwards of how a typical editor/IDE is implemented.

The main difference is that the implementation language is a dynamic language,
which is also mostly the implementation language.

That's similar to how some other IDEs work like Smalltalk or Clozure CL on the
Mac. But those are not focused on implementing an extensible editor. Those are
IDEs with editing features.

------
terminalcommand
If emacs supported new languages other than elisp, a lot of new blood will
join the community. Once we get the new hackers, they will try and find ways
to modify the old codebase. Emacs is a wonderful editor, but to a newcomer it
can be a lot daunting.

For example, I have completed the emacs tutorial, I've even read the infos
info tutorial and several info pages regarding emacs and still don't know much
about ELisp. I downloaded the official emacs manual, which is more than 1000
pdf pages and plan to read it in the next 10 years.

I love the editor, I want to continue to program in Emacs but Kate seems to be
much simpler, elegant and quite functional.

~~~
rwmj
Does emacs want the kind of developers who cannot be bothered to learn Elisp?
Apart from the syntax, Javascript and Lisp are not really so different.

~~~
TeMPOraL
They differ in semantics and elegance, and for that reason alone I wouldn't
want Emacs to end up scripted in JavaScript.

Sure it would bring new blood. It will also drive away old one; people, for
whom Emacs is a Lisp runtime with a text editor bolted on top of it. Switching
Emacs from Elisp to something else feels like selling its soul. Adding
parallel support for JavaScript and bunch of other languages feels like
turning it into a Frankenstein monster.

~~~
josteink
> Sure it would bring new blood. It will also drive away old one; people, for
> whom Emacs is a Lisp runtime with a text editor bolted on top of it.
> Switching Emacs from Elisp to something else feels like selling its soul.

While I agree with most of your statement, having Emacs
switch/transition/support another, more modern form of Lisp like CL surely
wouldn't count as selling your soul.

On the contrary you would stay true to your ideals (a Lisp-runtime with an
editor) except now the Lisp runtime just got a lot better.

I like Emacs as much as the next guy, and I have dabbled some in Elisp. I can
make due. But I sure as hell wouldn't mind getting some of that neat stuff the
CL-guys are getting.

~~~
TeMPOraL
> _While I agree with most of your statement, having Emacs switch
> /transition/support another, more modern form of Lisp like CL surely
> wouldn't count as selling your soul._

I... I'm not sure, but I'm leaning towards agreeing. I can tell you that I
_don 't get_ those "we're lobotomizing it" vibes when I think about switching
Elisp for CL. Maybe because those two look alike and have similar philosophy.
But even a thought of switching to Scheme, which is still a Lisp, sends chills
down my spine.

I know, this is completely irrational. But maybe this is a part of Emacs's
appeal. You get attached to it, down to its very core.

------
HerrMonnezza
Tom Tromey started an automated rewrite of Emacs in Common Lisp in 2012, see
the announcement at:
[http://tromey.com/blog/?p=709](http://tromey.com/blog/?p=709)

I couldn't find any progress report after
[http://tromey.com/blog/?p=751](http://tromey.com/blog/?p=751) so I guess the
project has been abandoned early.

~~~
eschulte
I think this is the most realistic approach mentioned here, and tromey is
actually familiar with the Emacs internals.

~~~
taylanub
So is BT Templeton, the person who has been hacking away at Guile-Emacs GSoC
after GSoC. :-)

(They seem very skilled, don't be fooled by the fact that it's GSoC; no idea
what the skill level of people who take part in that usually tends to be.)

------
informatimago
I've got the impression that there is more Common Lisp code and programmers
around than guile.

~~~
Blackthorn
Literally dozens more.

Look, neither Common Lisp nor any Scheme implementation are exactly topping
the charts in popularity these days. This should be about the technical merits
of the choice, because I'm pretty sure Emacs Lisp both has more code and more
programmers (both represented in the forms of small scripts we've shoved into
_scratch_ and evaluated) than both.

~~~
muuh-gnu
The decision to go either Common Lisp or Scheme was made based on RMS very
personal reasons and conflicts with close-source Lisp machine companies during
the first half of the 80s.

GNU has had two common Lisp implementations under their umbrella at least
since mid 90s (CLISP (of Viaweb fame) and GCL (of Maxima fame)), but due to
RMS' hate of Symbolics and Common Lisp, all of the effort was invested into
making Guile run properly (which took them 25 years and still nobody is using
it) and the two Lisps were basically orphaned.

The latest effort of guile-ifying Emacs is more pushed by the Guile guys, in
order to attach themselves to an already existing killer app (emacs) and to
basically force more and more people to use Scheme, because nobody is doing it
voluntarily.

The Emacs guys, as you see from the post, are not _really_ excited about the
idea of switching to another _language_. They want FFI, multithreading, etc,
but they do not necessarily want Scheme. Actually they dont want Scheme at
all, adopting Scheme would be just a price to get FFI, etc.

Of course the proper way would be to go Common Lisp. Elisp is and has always
been a subset of Common Lisp, and they've had a Common Lisp compatibility
layer for decades, and a lot of packages use it. The problem is simply highly
political. You'd have to confront RMS, and you'd have to finally get rid of
Guile's "official GNU extension language" status.

~~~
tenfingers
I've been programming with ELisp, CL and Scheme for a long time. Somehow Guile
feels like an odd choice. Why not Chicken scheme, or Racket, or Bigloo; which
are the top scheme environments you can choose from?

AFAIK, Guile is not used much beyond a few random projects. Maybe I just don't
know Guile, really. Maybe it has a decent implementation afterall.

However, I would have gone CL. There are several "emacs-like" editors
implemented in CL:

[http://www.cliki.net/cl-emacs](http://www.cliki.net/cl-emacs)

I would argue that implementing ELisp support in Climacs is actually easier,
so we can run all the elisp stuff we already have, and slowly move on.

~~~
apgwoz
Guile was _designed_ as an extension language, which CL was not. So, that's
one reason. Guile also has a something like 90% working implementation of an
elisp compiler to guile bytecode, a JavaScript implementation and (I think,
though may be wrong) a working Lua compiler. None of the other mentioned
scheme implementations were designed for embedding, even if they are fine
implementations themselves (which they are).

~~~
levosmetalo
If we consider "designed for embedding" as a driving point we can always take
ECL (embeddable/extensible common lisp) as a base and get both very nice ffi
and threading while still being CL.

------
CharlesMerriam2
This problem begs to think larger. Instead of starting with "we are missing
these two critical features that the rest of the world has expected as
standard since the dinosaurs perished", it might be interesting to ask "what
sort of emacs language features would make it the best in the world!"

It is far easier to get a minimum product out when the eventual goal is the
stars.

~~~
fleitz
I believe this kind of thinking is what led to the massive success that is
emacs in the first place.

------
brudgers
As I began really using Emacs about a year ago, I got the idea that the Emacs
code base could actually be a foundation for a computer science curriculum -
The Emacs code base touches on scripting, functional programming, application
programming, systems programming in C, real-time problems, usability, cross-
platform development, etc. etc.

An alliance with Racket would be an interesting option from the standpoint of
language communities.

    
    
         - The community's core is [ relatively ] vibrant.
    
         - The community has more than a passing interest in 
           developing development tools.
    
         - The community is exceptionally stable because
           its members are often linked to academia.
    
         - It copes with endless September really well.
    
         - Its vision of open resources overlaps GNU
           somewhat.
    
         - Programming language problems are right up
           the Racket community's alley.

------
lallysingh
I sent a note about using an elisp-on-llvm solution. Someone had done the work
already, it appears: [https://github.com/boostpro/emacs-llvm-
jit](https://github.com/boostpro/emacs-llvm-jit)

~~~
__david__
There's no way they'd do that. There's a rift because of LLVM's licensing
philosophy:
[https://gcc.gnu.org/ml/gcc/2014-01/msg00247.html](https://gcc.gnu.org/ml/gcc/2014-01/msg00247.html)

~~~
mwfunk
The fact that he even uses the term "adversaries" in this post is
disheartening. It's really disappointing that someone who could potentially be
such a force for good in the world gets derailed by what is basically
tribalism.

Two parties can disagree on 99% of their beliefs, yet still find ways to come
together on the 1% that they happen to agree on. These parties can work
together to each others' mutual benefit, and the world is better for it. But
nooooooooo, not RMS. He has to demonize anyone who is not 100% in lockstep
with him on everything, to everyone's detriment, and throw his "adversaries"
into a trash bin labelled "Others".

It's a self-fulfilling prophecy. It's easy to drive people apart in the name
of some ideal. It takes actual leadership to drive otherwise separate people
together to actually accomplish something. I have wished for so many years
that RMS would care enough to provide the latter instead of the usual former.

~~~
__david__
> It's easy to drive people apart in the name of some ideal. It takes actual
> leadership to drive otherwise separate people together to actually
> accomplish something.

Right there is the crux of the matter. RMS is _only_ focused on the ideal. To
him, "accomplishing something" is only relevant when the accomplishment is
free software that can't be locked away in proprietary codebases.

Viewed through that lens, his actions and beliefs have been remarkably
consistent for the past 30 years.

He really grates on people who don't share his idealism, and he's looked up to
by people who do. He can be quite polarizing, to say the least.

------
swah
Dream: "Common Lisp/Guile, and a Chromium based rendering engine".

~~~
josteink
Maybe I'm in the minority, but I for one _really_ appreciate how Emacs can run
in a terminal, over SSH, with almost no feature-degradation at all.

Besides Emacs being a very nice programming (and programmable editor), this is
one the key features which keeps me from even considering other more "modern"
options.

~~~
thejdude
I haven't used Emacs in a while, so I'm very rusty, but why would you run a
remote Emacs in SSH, and not use your local Emacs to access remote stuff via
TRAMP?

In my Emacs days, I've never really used the terminal version. Multiple
frames, yay!

~~~
krylon
On my remote systems, about the first thing I do after a reboot is call "emacs
--daemon" via SSH and then use emacsclient to edit files. For one thing, TRAMP
has higher latency, and it also keeps state across disconnects and reboots of
my desktop system. I regulary have lots of configuration files open, tail-mode
buffers to watch log files, an eshell session, man pages, web pages (using
w3m), and it is just too much of a hassle to open these all over again all the
time.

Also, if you have to access the remote machine through a low-bandwidth line,
the terminal version at least feels more efficient than using TRAMP from my
workstation. For most purposes, the GUI does not offer _that_ much of an
advantage.

------
eschaton
Why not start with Hemlock or CLIMacs atop a CLIM implementation, or take the
(MIT-licensed) ZWEI codebase and bring that forward? Is there really all that
much in GNU emacs that is really _widely_ used?

~~~
lispm
Starting with ZWEI, written in Lisp Machine Lisp in the late 70s, is a very
cool way forward. Move back to the 70s and start from there. Good idea. Maybe
not.

> Why not start with Hemlock or CLIMacs atop a CLIM implementation,

Why?

------
pnathan
If we (emacs users) can have an emacs that has a "modern" backend & language
but is bug-compatible with the extant elisp code, I think that would be grand.

Particularly nice would be thread support.

I'm pragmatically indifferent as to whether the backend is scheme or common
lisp (I'd personally prefer Common Lisp, but whatevs).

------
burtonator
I spent about 2 years of my life in my 20s hacking on elisp constantly. I was
infatuated with it.

Then I spent about 5-10 years using that platform as my IDE.

Guess what... Last year I migrated to IntelliJ IDEA and won't EVER migrate
back. It's kind of sad... but IDEA is insanely awesome by comparison.

~~~
TeMPOraL
Blasphemy!

On a more serious note, could you elaborate a bit on how IDEA is "insanely
awesome" compared to Emacs? What do you find more useful/enjoyable in the
IntelliJ IDE?

------
systems
who is Stefan Monnier? and why should we care about what he thinks?

is he the main emacs maintainer? in other words, how seriously should we
consider this email?

~~~
tjr
[http://lists.gnu.org/archive/html/emacs-
devel/2008-02/msg021...](http://lists.gnu.org/archive/html/emacs-
devel/2008-02/msg02140.html)

------
riffraff
can someone more knowledgeable than me explain how is elisp closer to CL than
to Scheme?

~~~
lispm
Both Emacs Lisp and Common Lisp are coming from MIT's Maclisp dialect. There
were Emacs variants written in Maclisp or its dialects before GNU Emacs
existed. Emacs Lisp was a very simplified Maclisp. Common Lisp modernized
Maclisp (especially lexical binding), preserved a lot of Maclisp features and
standardized stuff like an Object System (CLOS), error-handling (Conditions),
enhanced argument lists (especially keyword arguments).

RMS especially did not like the addition of an object system, lexical binding
as the default and keyword arguments. Those three things are quite central in
Common Lisp. Emacs Lisp has no object-system in its core, lexical binding has
only recently added and it has no keyword arguments.

Scheme is quite different from both. But Guile is a very special Scheme
dialect with lots of enhancements, which makes it as large as a Common Lisp
implementation. I would expect that code sharing between Scheme
implementations is more difficult than between Common Lisp implementations.

~~~
quotemstr
Keyword argument support is in the Emacs core these days, along with the rest
of the CL library.

    
    
        (require 'cl-lib)
        (cl-defun foo (&key a b c) (list a b c))
        (foo :c 3 :b 4 :a 1)

------
tedks
A lot of people seem to be reading this as if Emacs is choosing between
switching languages to Scheme (Guile) or Common Lisp.

Switching to Guile DOES NOT IMPLY switching to Scheme. There is 0 need for
compatibility layer or what have you with Guile.

Guile is a language-agnostic virtual machine. It has an implementation for
Scheme, but also one for Emacs Lisp. Guile already runs Emacs Lisp faster than
Emacs: [https://lists.gnu.org/archive/html/emacs-
devel/2010-04/msg00...](https://lists.gnu.org/archive/html/emacs-
devel/2010-04/msg00665.html)

The main issue the Emacs developers seem to have with Guile is that it will
give developers choices as to whether to write Emacs extensions in Scheme,
Elisp, or Javascript/Python/Whatever else Guile supports.

~~~
rthomas6
>The main issue the Emacs developers seem to have with Guile is that it will
give developers choices as to whether to write Emacs extensions in Scheme,
Elisp, or Javascript/Python/Whatever else Guile supports

I don't understand why they care? Why would the Emacs developers feel the need
to debug compatibility if someone else wrote a shitty extension? The onus
would be on the extension developer to fix the extension.

~~~
babarock
Quoting the linked email:

    
    
      So if we go for Guile-Emacs, we'll be stuck with Guile, i.e. we'd
      have (old and new) packages that use Elisp, new packages that use
      Scheme, maybe yet other new packages that use, say, Javascript (or some
      other language support by Guile).  That would make the work of Emacs
      (and GNU ELPA) maintenance harder.

~~~
Steltek
So don't accept packages into the Emacs core that don't meet their standards?
It's okay to have project standards but limiting how other people use your
tool seems terribly misguided for a GNU project.

------
sdegutis
Seriously, why not just use Lua? It's perfect for this.

~~~
krylon
I one were to start from scratch, writing an emacs-like text editor by
building it on top of a small programming language you can use for customizing
and extending the editor, Lua would a great choice. Lua has only a tiny
standard library, but in writing the core of the editor, you probably have to
supply the equivalent functionality yourself, anyway.

But I do not see how you could bolt Lua on to GNU Emacs. You would have to
start from scratch, and that would be a _lot_ of work. And even then, there is
the fact, that GNU Emacs is already well-established, has a very loyal user
base and is - by and large - good enough, even great for most purposes, so
even if you wrote LMacs or whatever one would call it, you would probably end
up with, like, a dozen users or so. (IIRC the Zile project, a small emacs-like
editor, has tried this, but I do not know how far they got with this.)

------
linguafranca
Why isn't Lua even a choice? It's small, portable, easily embeddable,
lightning fast, supports high-level functional programming idioms, and is
extremely flexible in creating and enforcing policies (via metaprogramming).

~~~
dbpatterson
Because it is a completely different language and would require rewriting all
of the thousands of packages by hand (and centuries of human labor)?

~~~
thejdude
OTOH, Emacs isn't that accessible to many users, and its idiosynchrasies can
be very offputting. I also quit Emacs after a while because I wasn't totally
happy with it.

I think that's why there are so many attempts to built new extensible editors
in other languages, like Eclipse, or that Ruby thing there once was (probably
still is), or the Chromium/JS monster that was on HN recently.

I haven't yet found an editor I'm really happy with, so if I ever have time,
I'll probably write my own as well. I'll lose the Emacs packages I don't use
or even know about, but if you haven't bought into an existing ecosystem, that
doesn't really matter. I'm unhappy enough with the existing systems that I
don't mind throwing them away for something different.

