Honestly, I'd rather see more progress on the Emacs Lisp/Guile migration than on a putative C/Rust migration. At least with the Guile switch, it's something that will have an obvious (and probably positive) impact on the people that use the editor.
First is that remacs has been able to reduce the amount of non-Elisp code in the editor needed to implement a number of different features. It was able to do this by replacing hand rolled C functionality with high quality crates from cargo, or code in the standard library. I'd attribute this to how easy the cargo system makes implementing libraries into your project.
Second is I believe that this project overall is increasing knowledge share of not just Rust, but also mainline emacs. I personally was intimidated by looking into contributing to mainline emacs. The remacs community was so friendly and welcoming I felt very encouraged to jump in and start working on things, and now I not only have better knowledge of Rust, but a pretty decent knowledge of the C layer in emacs and how it lays out and implements it's data structures etc.
Third is that the majority of the emacs codebase is written in emacs Lisp. Remacs aims to be fully Elisp compatible, and any differences are considered bugs and should be reported. This means that the project is still able to keep up feature and bug parity with the mainline, while incrementally rewriting the core of the editor.
Disclosure: I am a contributor to this project
A lot of people have already written about the difficulty of getting changes to fix things like this (or eliminate dead code, etc.) landed in upstream GNU Emacs. Rewriting parts of Emacs in Rust is an interesting idea, and I applaud it as a fun project. But for people who are interested in making more incremental improvements to Emacs, there's a lot of work to be done in the existing C code base to make it smaller and more secure.
That said, though: wouldn't that argue for things like "remove custom mkstemp" and not "rewrite it all in rust?"
I have already had to deal with python code that do little to no dependency documentation, that just carpet bomb my system with stuff pulled from the net at the install stage.
I wonder where have you lived? I rarely found (if ever) an undocumented dependency list for any serious Python project in the last couple of years.
there's the Pipfile initiative which gives me much hope, though :)
What you mean by no reproducible builds?
Really, significantly easier to not screw up in a large org than bundler, because dependencies don't leak by default.
What is the alternative? Every major Linux distro has its own repository, as do all the BSDs. The picture on Windows and macOS is even more convoluted. Are developers in all these languages supposed to support the packaging of their software in half a dozen different repositories?
Maybe all programmers should get together and build "one repository to rule them all"? I have no idea how we herd that many cats!
Meanwhile, cargo works flawlessly on every system I've tried (Linux, Mac and Windows). Sure, it only compiles Rust programs, but there's no way that I'd advocate moving those programs to a package manager with as little regard for such a large part of its potential userbase.
Or that they just don't have enough manpower to actually support all the platforms.
At a minimum, Linux should have a common packaging system; though there may never be agreement. It's hard to see macOS and windows endingnup with a common one.
But maybe we could do a third party option that is independent of each OS? Those rarely work out well though.
As an example, IBM just released yesterday a new mainframe model.
z/OS is a totally different animal compared with POSIX based OSes or Windows, starting by the file system, catalog based.
For a system utility, you can still use virtualenvs. The launcher script just needs to use a shebang like:
Part of me feels like a better solution is to maintain older branches to give distributions opportunity to catch up.
This single sentences hints at least major two reasons for language-specific package managers:
- non-Linux platforms
- there's many different Linux package managers (apt, rpm, pacman, nix, portage, ...)
A language-specific package manager means developers (i.e. the ones writing the packages) only have do to packaging work once, instead of for every platform they might want to support, including ones that they have no way to test on (and a lot of code, especially in the modern wave of languages like Rust, often works on all platforms with no particular effort from the developer).
From an asymptotic perspective, using the platform package managers results in something like O(# libraries * # platforms) work and likely a lot of code not being packaged for some platforms, whereas language-specific package managers is something closer to O(# languages + # libraries) work (or maybe O(# languages * # platforms + # libraries). Looking at this decision purely in terms of developer effort, the latter seems like the better choice.
(There's also things like fpm that are designed to be more like O(# platforms + # libraries) work, which is nicer than either of the above!)
Far too many times have I had to explain to newcomers to Common Lisp that they can't install stuff from the repositories, and instead have to download their CL implementation separately and then use Quicklisp.
You all are scratching an itch, and increasing the general mindshare for Emacs. If Remacs runs out of steam, then the world has some extra people who grok Emacs guts. If Remacs fully succeeds in remaking Emacs, but better, then maybe Remacs just becomes Emacs.
Thanks for posting this and thanks for your contributions.
I imagine that this, for the most part, isn't even opportunity cost: the people working on it probably wouldn't be contributing to full Emacs (or the Guile effort) anyway. And this sort of effort seems like it could be an on-boarding experience to improving the original code base, e.g. in the process of porting, someone notices a bug and so submits a patch upstream, or maybe the repeated exposure to a variety of the internal components gives someone enough familiarity with them to contribute if this project peters out.
In any event, I think we would do volunteers a disservice if we placed them in a special class to which we could only provide positive feedback. (Maliciousness is never appropriate, of course.)
"It's a neat language, but no one has used it to make anything that isn't a toy" is a really common mindset for the people who make such decisions.
The only way to change that mindset is for the evangelists to write software and show that yes, it is possible to write For Realsies software in that language. If that happens to be a rewrite of a successful project, fine - it's still a large project, and it'll inspire more development than someone coming out of nowhere with a project.
Fortunately, your top level comment was taken in good faith by one of the contributors, but this won't always happen (and, especially in a space like editors, where everyone uses them and many people have unreasonably strong opinions, so people doing "unconventional" things may have a history of feeling attacked for their hobby) but not phrasing a comment as uncompromising criticism can encourage productive discussion (instead of randoms like me bringing up non-technical things like this whole subthread).
I'm sorry that comment got down modded. In fact, I think/hope I voted it up.
One of the interesting tensions this dialog has surfaced (at least for me) is the relationship between volunteerism and the way outsiders interact with the volunteer community. More than a few people effectively suggested my comment shouldn't have been made. While I do agree with the need to protect contributors from a lot of what they have to put up with, I'm not sure I'd ring fence those communities quite that securely.
Put another way, if my comment really does fall into a class of those that shouldn't have been made, what are the 'rules' for outsiders engaging open source (or other volunteer) communities?
> Fortunately, your top level comment was taken in good faith
Agreed. But after 30 years on the net, it still surprises me how many people just automatically default to whatever the 'bad faith' interpretation might be. We all read what we read in our own context, and this suggests to me that, for many of us, that context is rather angry and upset.
This doesn't even seem unreasonable to me: if every time my project was discussed it was met with scepticism and outright "why are you bothering to do this?", I would get fairly tired and, after the first few times, get fed up with putting in the emotional effort to see the positive nugget buried in the negative phrasing.
Someone taking 5 minutes to write a comment about a project they've just been made aware of is unlikely to have the context to know whether that project regularly gets these sort of comments ("I really meant it as a way to learn more about the project, please, authors, can you politely answer my rudely phrased question/criticism") and so they should be the ones putting in the effort of research and of starting the conversation on a good foot: questions about the points of scepticism (rather than blank statements) are a good start.
It is, a little, but then again it's par for the course for Open Source projects, which are both volunteer AND seeking to engage a larger community.
It's not like its just some people doing what they like and keeping it to themselves or just offering it as-is. Open Source projects that want to become larger communities and attract more developers AND end users, have to accept external opinion and criticism.
- are mainly driven by paid contributors, with some volunteers
- some number of paid contributors with many volunteers
- have only volunteers
And they are open source:
- in order to engage a larger community of end users
- in order to engage a larger community of developers (doing things with people is often more fun, and/or having more people poking at keyboards allows more, say, of an experiment to be accomplished)
- to serve as a portfolio, e.g., for job seeking
- as a political position (some variation of "code should be open")
- out of habit/because there's no reason to not be open source
(Or some combination of the above.)
As far as I know, remacs is driven entirely by volunteers, but I don't know where it fits into the latter category. The original author has written blog post(s) about it, so clearly they want people to know about it, but it isn't obvious to me if they care particularly about people using the result for editing text rather than, say, investigating and showing off the solutions to problems encountered along the way (portfolio) or being able to collaborate on a fun project with people (community of developers).
In any case, I agree with you about projects accepting criticism, but I think the style of criticism needs to take into account the specifics of the project: something like, there should be less "this is a waste of effort"/"I think you should do something else" and more of the pure technical stuff ("algorithm ABC might be more efficient than the XYZ currently used, as ABC is O(n log n)", or something) as one moves further down either list.
Moreover, while some people work on the endless clones of the same thing, the completely different set of contributors is wasting their time cleaning up / fixing bugs / writing documentation for those endless clones.
On the other side, there is Spacemacs, which tries to provide solid modern IDE-like experience on top of existing Emacs solution, leaving low-level stuff to emacs core team.
In a business setting, you pay people.
In a hobby setting like that, people compensate by having fun.
Either approach is compatible with producing open-source software; I suspect most LOC of open-source projects are written by salaried contributors at large corporations.
Somewhere in the middle. If you consider the work that goes into making a plane, you'll see quite a bit of it happens outside Boeing. To give one example, think of the aluminium that goes into the plane. Boeing doesn't have to mine bauxite, extract the aluminium from the bauxite, transport it to the place where its sold, etc...
A classic example of this ground-up view was this example given by Milton Friedman about the pencil. I don't agree with many of his views, but I think he makes a good point in this video:
EDIT: Turns out the source of this example may have been Leonard Read:
"On the other side Boeing pay them money so they can proceed to mine."
Perhaps you know more about Boeing's business arrangements than me, but I certainly couldn't verify that that's true. From what little I know, it's possible Boeing's aluminium suppliers work exclusively for Boeing, but it's also possible these same aluminium suppliers have multiple customers. The point being, there could be people who work as part of the chain to produce and transport the aluminium who have no idea who will eventually use it.
You could say the same thing about commercial software (and other) development. They just call it competition. In business they claim it is efficient and drives innovation. In open source it's said to be inefficient and a waste of time.
Creating new things is better than sitting on a sofa watching TV or playing video games all day. Furthermore, it often opens the door to possible new, unforeseen use cases, concepts, and success. Open source has a natural evolutionary path. Survival of the fittest, rather than survival of the richest.
Honesty doesn't keep it from being useless stop energy.
>in the process of porting, someone notices a bug and so submits a patch upstream
I find it a little bit controversial.
But it's not: one might very plausibly not care to hack on C emacs, but care to hack on a Rust emacs.
And at the same time, the same person, might be able to spot a bug while porting C code to the Rust emacs, that will also help fix C-emacs.
If remacs didn't exist the hypothetical developer wouldn't be contributing to full Emacs (it never crossed their mind, or it's too intimidating, or whatever other reason), but through their work on remacs, they've also contributed to the original emacs. The point is that the latter contribution would never have happened without remacs.
I mean, if you have a lot of energy to use and give, and you want to clone something huge, clone one the proprietary tools that have no free/open source equivalent yet in your domain or one of the domains where there are the most serious lacks of FOSS; or clone a web app into a real desktop program; or reverse engineer a proper open driver for whatever device you have which request a proprietary OS, a proprietary driver or a manufacturer specific tool; rewrite a library which only exists in Python or C# into a C or C-compatible library so that every language on Earth can wrap it, etc. Don't go cloning something that is already free multiplatform software just for the purpose of cloning.
We have never had so many people writing open-source programs, taking part in open-source projects, and yet it looks like the progress of open-source over proprietary (and its new form: services) is not what it was in the 90's and 2000's, it stalled, and it regresses in some areas. There has to be a problem somewhere.
This strikes me as a very entitled viewpoint. Why should people work on what you want them to work on? They could be watching TV or playing video games in their free time, but if they work on a project you don't think is valuable, it's somehow a problem?
This is probably the most informative comment of the whole thread.
"We aim to be a drop-in replacement with bug-for-bug compatibility."
Also maybe it will help testing emacs, and even improving faster since rust typesystem is said to help refactoring tremendously.
The sons of Hermes love to play
And only do their best when they
Are told they oughtn't;
Apollo's children never shrink
From boring jobs but have to think
Their work important.
Now, there's a fairly recent (and active) project for implementing a common lisp in c++[clasp] -- perhaps making a real common lisp in rust would make sense?
I have no idea how feasible this is in any implementation of common lisp, but the mere fact the language (AFAIK) requires a runtime beyond the C runtime (garbage collector, etc.) suggests it may not be as easy as with Rust. If it's possible, maybe someone else could start a project attempting that, if they thought they'd enjoy it.
* replace the GNU Emacs runtime written in C/Rust with one which is already provided by a Common Lisp implementation. Each Common Lisp implementation has its own runtime already, but those are not application specific to an editor. Several Common Lisp runtimes for example support multi-threaded Lisp code, native Lisp code, various GCs, support for more data types, better error handling, ... plus they support much of the basic Lisp stuff which is in the GNU Emacs runtime (evaluating code, loading code, ...)
* move some C/Rust code into Common Lisp, since Common Lisp compilers make it possible to write a wider range of code directly in Lisp.
* provide some low-level C code to Common Lisp via it's FFI. That way the C code does not need to be a part of the runtime.
But the topic is dead, because the expertise to do something like that is rare and the GNU Emacs community has no interest moving in this direction.
Oh, but how wrong you are.
At London Emacs meetup Wilfred gave a talk about his motivations for this work. One of the big and most obvious benefits is this:
while porting some feature to Rust, he discovered a bug in Emacs, fixed it and got given commit access for his work.
You can read the transcript of his presentation here:
IMHO, a potentially successful Emacs ports depends on a new plugin system which supports code in other languages (Rust for instance) rather than elisp so that new developers get attracted who dislike Lisp. In other words, don't attempt to clone emacs but develop a completely new editor which uses the successful concepts of Emacs.
Unfortunately, plugins require dynamic code generation, and AFAIK only scripting languages like Lisp, Scheme and Python are able to do that. Emacs has that already. Emacs's plugin system has an unmatched simplicity and reliability, and that's likely the reason why any clone has a very hard time to compete.
(I'm not saying remacs is the same thing by any means, but the relationship between effort expended and the type of value realized is something I've become more attuned to the older I get. Maybe it's a mild mid-life crisis, but I'm pretty cognizant these days that there's an upper bound on the length of time we have available.)
I mean why not do it properly if it's not an hostile fork.
But sadly, the lead-dev seems to have made up his mind. Being a nOOb with emacs I'm in a position to judge the reasons though.
What’s the breakout of C vs Elisp?
Ideally more of Emacs could be rewritten in Lisp with a smaller interpreter.
1,632,280 lines of elisp
So... Yeah. The c lines are in a way, "overcounted".
Agree, but then we would have a new case of Greenspun's 10th rule.
Leo Editor (http://leoeditor.com) fits that bill. It's been around since the 90's (and the web page looks like it's from the 90's). Even though it has many users, the documentation is quite poor. If I had another life to live, I would learn it well and improve the docs - and then port over everything I like in Emacs to it.
It is sorely in need of more tutorials. You'll find some in various people's blog posts, but not enough. I know enough of it to know that it is very powerful, but not enough to use it well day-to-day.
This is an interesting project and I wish them well. I have no opinion on Rust at the moment, but I do see this as part of a general trend towards safer languages.
What makes you say that?
But how extensible is Sublime itself? For example, in Emacs I can do network connections (e.g. Gnus). Will Sublime allow this? Can I make a Tetris clone in Sublime? Both are doable in Leo.
Leo is all Python, and you have access to all the internals. I'm sure there are design decisions in Leo that will make changing some aspects hard, though.
If it gets faster, I might give it a try again (in evil ofcourse, gotta have them keys).
I dunno. First of all it wants to replace the C-bits of Emacs, not all that is in elisp land. Second of all, why not? I do not think the goals of Remacs are overly ambitious: port the C-platform under Emacs's elisp env to Rust. This will probably result in a much healthier and hackable codebase for that functionality, as it: is a more modern language, has less legacy (platforms) to support, is fully open source (Rust) to the level RMS would approve of.
If anything, it could be great if Rust is used to implement a faster Emacs Lisp compiler/interpreter, thus preserving all of Emacs Lisp source (and saving some implementation time.)
And make it fully multitasking so there are no more problems of key input getting stuck because of a process.
So opened - https://github.com/Wilfred/remacs/issues/238
God knows how many GPLed lines of code has not been submitted back upstream to them over this minor technicality, and that's such a shame.
I honestly think this is for the FSF to fix, not everyone else.
How many legal disputes have the FSF had recently, in practice, where this turned out crucial?
More specifically: how much have they gained by this vs how many contributions have they lost?
Until proven otherwise, I'd expect them to clearly be in the negative here.
If there was evidence that some third party could enforce GPL compliance on behalf of the copyright holder(s) then maybe the FSF should consider relaxing the requirements.
Do you recall that Sony failed to comply with the GPL when they used LAME? The LAME community wasn't prepared to actually do anything about it. That's the kind of shit that would happen, especially to smaller projects. There are big companies out there that want to use your hard work without compensation and shit all over the GPL. Never forget that.
I'll rather have code which is harder to defend than defense for code I never got.
Copyright assignments allow the FSF to enforce the GPL on Emacs.
I know plenty of people who won't bother on plain principle, and that's both people and code which the FSF is missing out on.
New times. New standards. Get with the program FSF.
It may have useful security properties over time. People use emacs for a great deal of arbitrary text reading and transform from things directly from email and the web. Elisp would largely be at fault for security bugs but perhaps this would help in areas.
Perhaps a binding to use Alacritty directly for a port to Wayland? Emacs has a pretty terminal centric view of its UI output as is.
A tight binding to Servo allowing a serious upgrade to Emacs visual and ui capabilities.
I'm open to be surprised. But I really do want Emacs to have full access to my computer.
I guess if new management at Mozilla relaunched Firefox OS as BootToServo then the killer app from day one could be Emacs.
None of those ports ever see even the tiniest beginning of something actually usable (heck, even Climacs faltered), but I'm still all for it!
Also, 90+% of Emacs is written in Lisp which doesn't need to be ported at all.
I just built it, and it's fully functional. It'll be a perfect replacement for Emacs at every stage of the port, which is really cool.
But of course, that safety only exists to the extent that the Lisp interpreter/compiler is bug-free. The C side of Emacs's code base is alternatively great and terrible, depending on how you look at it. On the one hand, 30 years of continuous development has resulted in code that works really well. On the other hand, it is very hard to extend this code to support new features (good multi-threading support has been in the works for ages, moving to lexical scoping and away from dynamic scoping has taken years, etc). This is because it's difficult to maintain quality and compatibility while also navigating such an old project.
Once enough of this old C code is rewritten in Rust, it'll be possible to leverage the memory safety features (and the type system) of Rust to innovate much more quickly. Not as quickly as you can innovate on code that's written in Lisp, but much more quickly than you can innovate on the C code.
Besides, the number of libraries that a program uses is only a detriment if it means that the developers have to spend more time installing and maintaining those libraries. Cargo makes this comparatively painless.
I thought the lisp was written in C and so moving it to Rust would be useful. Perhaps it's written in itself. If that's the case then my bad. I did not mean rewrite the list parts in rust.
I am unclear about what you mean by this? The lisp in emacs is elisp ( emacs lisp). More generally, Lisp is the name given to a class of languages including Guile, Racket, Scheme, Common Lisp that generally feature homomorphism, functional style and parentheses to enable the homomorphism.
The LISP interpreter is written in C. I was trying to suggest that it may be useful to rewrite that in Rust. That is a necessary step in getting rid of the C dependencies for emacs.
Question to the devs: are you planning on redoing how the gui subsystem works, to work with (for example) the gtk event loop? I've always wanted to try to put a qt frontend on emacs, but have always been horrified away by that subsystem.
That makes it something less than a drop-in replacement from the overall CEmacs project perspective. I'm sure Rust devs are working on more platform support, and it's possible Emacs devs are open to simply dropping some of the more obscure targets, but it might be a long time before alignment happens on that requirement alone.
There are many unrelated editors that have cloned aspects of Sublime to varying degrees in the 2010s.
It's about having useful aspects. And ST3 is the best example we have of a native, fast, modern, cross platform, plugin-able editor.
Though I guess it's not actually a port of Vim, just a vi-like editor.