Hacker News new | comments | show | ask | jobs | submit login
Announcing Remacs: Porting Emacs to Rust (wilfred.me.uk)
206 points by smg on Jan 11, 2017 | hide | past | web | favorite | 89 comments

The example seems to be wrong https://github.com/Wilfred/remacs#porting-c-functions-to-rus...

fn Fnumberp(object: LispObject) -> LispObject { if lisp::SYMBOLP(object) { unsafe { Qt } } else { Qnil } }

It uses SYMBOLP instead of NUMBERP. Highlighting the risk of introducing new bugs.

Which leads to the question, why Remacs can't just auto-wrap the lisp::* functions, which I assume are the Rust versions of the C Macros. If you look at Emacs' C code there are a lot of functions and macros that implement Elisp primitives in a C way. E.g., NUMBERP(x) will return 1 if x is a number or else 0. So you can use this function to deal with lisp objects in C code. The function that exports this primitive to elisp is Fnumberp. Rust has a better type system than C and supports meta-programming. So why not have a simple wrapper that can take a (LispObject) -> bool and turn it into a (LispObject) -> LispObject. Similar for other Elisp<->Rust types.

However unless GNU Emacs is willing to accept the Rust replacement code I don't think this will succeed. It is a lot of work and it takes quite some time until it actually pays off. It seems simpler to do what GCC and GDB have done and switch from C to (a strict subset of) C++ to simplify at least some of the more painful C hackeries.

And the reasoning given in the announcement are rather weak:

* "We can leverage the rapidly-growing crate ecosystem." Emacs recently added module support allowing leveraging all kinds of ecosystems

* "We can drop support legacy compilers and platforms (looking at you, MS-DOS)." how is that an opportunity when it effectively removes support for platforms.

> GCC and GDB have ... switch[ed] from C to (a strict subset of) C++ to simplify at least some of the more painful C hackeries.

Interesting! Where can I read more about this? Is it true that gcc compiles using g++? That's rather amusing. :)

> "We can drop support legacy compilers and platforms (looking at you, MS-DOS)."

> how is that an opportunity when it effectively removes support for platforms.

Supporting MS-DOS is something of a challenge now. I've not done any research but I have vague notions that GCC ports aren't really being maintained anymore. Besides that, many platforms (for example BeOS) are stuck on GCC 2.95.3, a rather famous last version using a particular ABI. A lot of stuff is stuck on that GCC version. (I don't know many details, although I'm very interested to learn more if anyone else has any insight.)

With the above said, I do disagree with wholesale willingness to summarily sweep legacy platforms off the table. Rust has saved itself some maintenance nightmares because it doesn't support DOS, but it means quite a large number of people are still stuck on C for industrial control system tooling. (Granted, I can't deny that I'm talking about a really tiny niche here...)

Yes it is true, all major C compilers are now written in C++.

You can find everything related to GCC's C++ transition here:


On Windows even the C runtime, MSVCRT.dll got re-written in C++ and the C functions are actually extern "C" ones.


So a kind of small victory for us on the C++ side of the fence on the endless C vs C++ discussions.

Now the joke "my compiler compiles yours" has been reversed.

Wow, that's a surprise.

I'm really considering learning C++ now... I'm just hugely put off by the long compile times :( (I have 10+ year old hardware)

If you are on Windows, VS 2015 and the upcoming VS 2016 have lots of improvements regarding compile times.




Apple is also doing a similar work regarding incremental C and C++ builds, based on the work they did for Swift


C++ builds can be made bearable if you don't go crazy with template meta-programming, do forward declarations, only expose actually public data on the headers, instantiate most use templates.

But specially do use binary libraries, even across application modules. Don't spend endless time recompiling code that doesn't change.

> C++ builds can be made bearable if you don't go crazy with template meta-programming, do forward declarations, only expose actually public data on the headers, instantiate most use templates.

Exactly. E.g. QT based applications (which rely on object orientation, inheritance and virtual methods) compile reasonably fast. However the template oriented style (e.g. found in boost libraries) can increase compile times A LOT. I had more than factor 10 for one of my libraries that I ported from having boost as a dependency to QT. Although the template oriented style might really work faster at runtime, because there's less need for virtual function calls and other indirections.

I suspect with 10+ year old hardware he will not be on Windows.

You're right: Slackware :)

This ThinkPad T43 turned 10 this year :P

(In all fairness, my T60 is 9 years old now.)

I do have 10+ year old hardware on Windows, namely 7 and 10 versions of it.

IT only changes workstations when they break down or they cannot fulfill project requirements.

Before you invest time learning C++, have you considered Rust? There are pros and cons for and against each language, but as they're both operating in the same space it's something to keep in mind.

If you use C++ and believe Rust has the potential to reach the same marketshare / level of adoption C++ has now, then you should strongly consider learning Rust in the future. It has a compelling list of safety features and a more modern and ergonomic design.

I personally think Rust is going to go very far in the not too distant future, but I'm a slightly biased Rust fanboy. Do your own homework on the matter. You might decide your next language is Rust.

Well, the OP is a pro-Rust topic, so yeah.

As I said on the top-level comment that opened this thread, I'm tentatively considering Rust. I'd love for it to take off and become a serious and reliable contender. I have a tendency to be conservative and cautious (generally), so I'm seeing how things pan out.

The compile times are not that long unless your program is very big. I also have 10 year old hardware.

I guess it depends on what you're used to. Each time I switch from C to C++, compile times take some adjusting.

If you're not careful about how you organise your C++ code (be careful what you put in headers, don't go crazy with templates), compiling can get really slow, even for relatively small projects.

Just #include <iostream> and the compiler needs to process 37,799(!) lines and more than a MB of data:

  echo '#include <iostream>' | g++ -E  -x c++ - | wc
     37799  104779 1280834
To contrast in C:

  echo '#include <stdio.h>' | gcc -E  -x c - | wc
       456    1638   20880
Comparing the compile times of PostgreSQL (C, minutes on an old laptop) and MySQL (C++, 30+ minutes) is also instructive.

Maybe when modules are finally introduced, things will improve.

The numbers seem to depend heavily on your environment, by the way:

  $ uname -m
  $ source /etc/os-release && echo $NAME
  Arch Linux
  $ echo '#include <iostream>' | g++ -E  -x c++ - | wc
    27336   59448  643240

Sure. Mine was macOS Sierra.

If your <stdio> is comparable to mine, that's still about 30 times more bytes that the compiler sees for C++ compared to C.

I'd edit this into the original comment if I could; here seems to be a good spot to put it.

I think I should add a bit of context about what I mean by slow compile times: I feel comfortable if I can hit CTRL+S* and have my code rerun or restart within about half a second. At one second, it should either be executing or have output something and finished, and if it takes more than about 10 seconds to get to whatever point I'm prototyping or making a decision on, I get jittery.

It's partly because I have a ridiculously short feedback loop, but also because I am (for some reason) sensitive to interactivity latency and I prioritize responsiveness extremely highly. So long compile times tend to lean on those buttons and make me uncomfortable.

With this in mind, the only C++ experience I've had yet was doing some modifications to the Dillo web browser, which was the main web browser I was running on the 800MHz AMD Duron box I used between 2012-2014 (the same one I described in https://news.ycombinator.com/item?id=13344332).

It was a lot of fun - I modified the tab bar so tabs could be dragged around, moved between windows, etc :D - but the compile times were around 3 seconds! This was just for the test harness I built which had an absolute minimum of code in it; I was also using GCC precompiled headers for literally everything (my prototype code was a ~300 line .cpp file, no headers or anything). This was a really big adjustment for someone used to tcc (~10ms compile time) or gcc (~300ms compile time) who felt that gcc took juuuust a bit too long to do its thing.

I've seen C and C++ code compile on friends' i3 boxes (haven't seen what an i7 can do yet) and been amazed. I think when I have a machine like that I'll probably be able to seriously get into the language. (I actually have an i3 here but it had a memory failure several months ago, hopefully I can get some new DIMMs for it soon and see if the motherboard actually still works.)


* About ^S - I'm yet to play with Emacs which I understand has two keystrokes to save files; I mostly use Vim because it's installed everywhere. I far prefer ^S over ESC (Press)Shift ; (Release)Shift w Return i/a.

With the failure of Lisp Machines, Lucid pivoted into doing a Lisp like interactive environment for C++, with compilation at function/method level. There is an old VHS video that some uploaded to YouTube demoing it. Search for Lucid C++.

The last version of Visual C++ (v4) used a database repository as binary representation of C++ code, also offering an interactive experience.

The problem with both environments was that they required powerful hardware that most people weren't wiling to pay for and consequently died.

Microsoft is now having a smarter linker with db support for VSC++ 2017.

Both Apple and Google were discussing at last LLVM conference how to add similar capabilities, IDE friendly, to clang, partially based on the Swift Playgrounds development experience.

I think I vaguely recall learning about database-incorporating C++ compilers recently, and now I actually see them. This is really neat.

I'm definitely looking forward to LLVM getting these capabilities. It'll move C++ development forward by light years, I reckon. (For example, Chrome takes an hour to fully rebuild in CI on what I suspect is fairly modern server hardware.)

Here are the Lucid and IBM compilers I talked about.

Lucid Energize Demo VHS 1993


IBM VisualAge C++ Professional 4.0


Sadly there isn't much information online, as this was on the days of BBS, Compuserve and other early online services.

Wow, that's really awesome.

If only the source code at https://archive.org/details/archiveteam-lucid-collection was complete :( that would be one source to getting a copy I could play with.

I'm definitely checking out VA C++ at some point though.


Its all relative.

> However unless GNU Emacs is willing to accept the Rust replacement code I don't think this will succeed.

There are many kinds of success. :-)

Emacs needs its stuff moved into Elisp as much as possible, and its Elisp to be improved, not a port of the C stuff to some random language. Rust is not a standardised language and is mostly driven by a single entity. Totally not the right tool. Also I find it very cryptic.

Totally agree.

The part of Emacs which needs a ramp-up is the Lisp layer, not the C layer.

When I read this post I just had a big 'why' in my mind.

What about it makes it the wrong tool for the job?


As a long time Emacs user, the only way I'd even consider switching off of GNU Emacs would be if there was a port written in Common Lisp that replaced ELisp with Common Lisp. Even then, I'd only switch if it had replacements for the ELisp modules I use (ERC, Slime, C++-mode, etc.).

As an end user, I don't see any advantages in switching to yet another Emacs rewrite in yet another low level language.

Isn't the point of this port to work out-of-the-box with existing elisp?

Right. But as an end user there's no advantage to using Remacs over normal GNU Emacs. "Written in Rust instead of C" isn't a useful feature to me.

Using Common Lisp instead of ELisp would be useful enough that I might switch.

I think that the idea behind this project is to simplify Emacs' code, e.g., by dropping support for ancient systems (they mention MS-DOS) and using pre-existing crates instead of in-house solutions. A simpler codebase might provide significant advantages for end users as well: (1) there will be less bugs in the code, (2) finding and fixing these bugs will be simpler, and (3) new features and structural improvements will be easier to implement. Of course, users will actually switch to Rustmacs only if they will be 100% compatibile, i.e., if Rustmacs can run every Elisp code in the same way as Emacs does. It's surely a very ambitious project!

Have you tried to build GNU Emacs on windows? I'd take a Rust port in a heartbeat.

Yes, I use it every day, at work and at home between different computers. I have a few windows-nt specific lines in my .emacs, mostly to set paths to grep and the like but otherwise have no problems and it is pretty indistinguishable from using it on linux systems.

Could you perhaps elaborate on the problems you are experiencing, since otherwise I really have no idea what you're talking about.

He said "build", not "use".

No. Sounds like a Windows problem.

I wrote this Common Lisp script to pull the tip of the official Git repo, and then build and install. I run it every few months if I need to restart Emacs for some reason. Works (for me) on Linux and OSX without out a problem. I've been using it long enough that I had to update it to use Git instead of whatever they used before, and I don't recall ever having a build error once I installed the dependencies the first time.


I know it's cool to hate on Windows but some of us have to work with operating systems that support the tools we use.

Rust has phenomenal cross-platform support so there's no need for scripts or multiple dependencies. It's just a simple "cargo install <foo>" and away you go.

Emacs, on the other hand, has some pretty big system dependencies (like gtk3, alsa and imagemagick on a modern linux) and one step of compilation where it actually loads up all the elisp stuff (which is really slow to load), then dumps the memory holding the loaded elisp stuff, and on next startup it just reloads the dumped state into memory. Somehow I strongly doubt that's going to be a simple "cargo install foo".

How do they intend to handle GTK no longer caring about being cross platform?

Who emacs? I think it can be built against half a dozen gui toolkits.

Well that's kind of silly. Unless you're contributing code to the Windows port of Emacs itself then there's no reason I can think of to build from source instead of downloading the prebuilt Windows binary from the GNU ftp site. They have pre-built 32 and 64 bit versions, each with a variety of options http://ftp.gnu.org/gnu/emacs/windows/

Even if that's not feasible for some reason, setting up the build environment is a one time sunk cost, so who cares?

I build from source at work because I'm still using an ancient version of Ubuntu there, and "apt-get install emacs" installs version 24.something.

At home on a modern Debian system I use "apt-get install emacs" and it works just fine. I did test that build script on OSX, but the one I actually use is from https://emacsformacosx.com/

Why bother when there are binaries available?

Even on Linux it's not as quick as before 24. I may be will wrong I stopped running my own build since a year.

Yes but the commenter that you replied to was indicating that he would only care if a rewrite was done in Common Lisp so that Common Lisp could replace ELisp.

I thought the plan was to port emacs lisp to the Guile interpreter and, in the process, make Scheme a first-class citizen for scripting.


Doesn't guile have atrocious performance, even compared to elisp?

Maybe it used to, but Guile 2.0 is quite a bit faster than Emacs Lisp. Guile 2.2 is faster still.

Au contraire. Guile avoids the locks currently Emacs has.

Weren't they working on adding the ability to use Guile beside ELisp?

I really love the idea and style of this blog post and the project's README. I recognized the style from somewhere and then realized it was the same author as "Example Driven Development"[1], another one that showed up on HN a while back.

[1]: http://www.wilfred.me.uk/blog/2016/07/30/example-driven-deve...

People should stop rewriting huge codebases that work quite well into the next hype-language. There's absolutely nothing wrong to develop it in C or C++. Stop the hype-train.

Counterpoint: having a solid, existing and mature codebase can serve as a useful tool to learn where and how best to apply a new language's features. As a bonus, perhaps some improvement in the original system will be recognized!

Seriously. Next thing you know there will be a Rust-based BLAS.

Rust is becoming such a huge systems development language, that's so good to hear that finally the developers are getting better deal for performant applications instead of C/C++

Only in the HN bubble. Meanwhile, "in the wild," it's hard to find a person who does systems programming and has even heard of Rust, or if they have, that has given it serious consideration.

It's too early and too immature a language for all the fawning it gets on HN.

I think if the developers do it right they can snag enough mindshare among new developers that Rust will be a serious competitor to C and C++ in 10 years or so. Which is great, because I think it's a better language. But it isn't there yet.

I was programming in Python in 1996 and struggled to find anyone to pay me to work in it. Meanwhile it's now 2017 and it's only now a fully mainstream language for the last 5 years or so.

It takes a while.

(I lost interest in Python in early 2000s, and then it got popular. So if I lose interest in Rust it's bound to get popular.)

Curious what language you generally turn to - or in the likely case of multiple languages, what you use each for.

In particular, what do you turn to for the kinds of things others would use Python for?

Are you saying that Python wasn't a mainstream language in 2012?

I appreciate this sentiment; I've been very hesitant to explore Rust myself, it seems too new, too overhyped, and lacking in foundation (or, as you said, too immature) at this point for me to have absolute confidence in it, or at least the same level of absolute confidence I'd have in C.

I'm definitely keeping an eye on it though, because if it keeps going at the same rate it has the potential to be great in 10 years.

But at the same time, flurries of activity aren't universally positive: it becomes harder to filter out the low-quality contributions that introduce subtle bugs, and smaller/more strict teams have an easier time maintaining stringent quality controls.

PHP and JavaScript very rarely segfault either. Sure, they use virtual machines (PHP's is non-JITed too!); and yes, their CPU and memory usage are comparatively astronomical. But I can only hope that Rust's memory-safety doesn't wind up attracting the same kind of unskilled language architecture "contributions" that PHP has, or wind up with an ecosystem as fragmented as Node's.

> But I can only hope that Rust's memory-safety doesn't wind up attracting the same kind of unskilled language architecture "contributions" that PHP has, or wind up with an ecosystem as fragmented as Node's.

Going from "PHP has memory safety" to "memory safety causes the quality of code to decrease" is quite a leap.

Haskell also has memory safety.

You're right. Let me qualify.

Haskell as an excellent example of a language that, by design, filters out probably many well-meaning individuals who don't have the requisite level of high-level skill.

Languages like Java, BASIC, PHP, etc are easy to get started with, which means those languages attract all types. Perl is somewhat in this class too, along with C++ due to that language's widespread popularity.

In PHP's case, the language's lack of inherent default-to-paranoia security settings has bitten it hard, because no real consideration was given to these phenomena (or apparently anything else, but that's a different story).

It's a nuanced (social) issue, but solving the type safety problem does make it generally easier not to shoot your foot when you still don't know what you're doing. The hope is that the programmer steadily advances beyond the baby-steps/worsethanfailure.com stage - and doesn't build any large systems that are too big to rewrite before they're decent.

Languages like Haskell, Lisp/Scheme, Erlang, C, assembly language, etc are generally difficult to do large-scale useful things with unless you've either studied the language a lot and/or already have programming experience.

I'm not sure what "memory safety" has to do with skill regarding "language architecture." In fact, I'm not quite sure what you mean by that whole bit.

Memory safety isn't a bad thing. But maybe pushing it as the thing will end up making it harder for newer systems programmers to be comfortable dropping into "unsafe-Rust" when it's necessary out of fear, or harder for them to learn the lessons that C and C++ folks have had to when they do have to use "unsafe-Rust". We just don't know what the impact will be in that regard. It's part of being a young language--it just needs time to develop.

Your second paragraph pretty much nails my thinking exactly. Memory safety is a very good thing, but having to manage memory/the stack/etc is kind of an IQ test that filters out contributions of generally low quality.

At least that's the theory. Thinking about it, there are probably people out there who can nail things like memory management but are terrible at eg higher-level application architecture to such an extent that they're ultimately not competent engineers anyway. It's possible at least.

> PHP's is non-JITed too!

It depends on which PHP you mean.



Oh, that's awesome to hear, that stock PHP will finally be JITted. I wonder how good it'll be...

(My primary dev platform is 32-bit, so I can't take HHVM seriously.)

Meanwhile, "in the wild," it's hard to find a person who does systems programming and has even heard of Rust

I agree that it's hard to find people who do systems programming, but those I know have mostly heard of Rust.

There isn't much else of anything happening in that space aside from C++ iterations.

If you base your insight on HN articles, then yes it would appear that way.

While I am a big fan of Rust, that is not quite true.

Rust still needs to improve its compile times and IDE related tooling, to make it appelative to the enterprise coder used to the comfort of Java / .NET tooling + C++ for lower level stuff.

I like the language but this amount of hype can only backfire.

So it has about 10 rust files so far?

Isn't there already another project, creating a OS with rust?

The problem, of course, being that Emacs has become a horrible codebase to maintain 20 years of compatibility, and rewriting is unlikely to change that.

That's not true. Like any old codebase it's got its sketchy corners, but it's certainly not "horrible".

Emacs is one of the few longstanding C codebases I know that compiles almost entirely without warnings. Which is quite an achievement and speaks somewhat to it's quality.

Emacs has some design decisions that are the result of 20 years of concessions on a ton of different platforms, and some of those are questionable anno 2017. Emacs' actual codebase definitely isn't bad.

To clarify: I'm not saying that locally the code is bad, I'm more talking about the kind of macro issues outlined in Buttery Smooth Emacs. But it's these kind of things that will be harder to "fix" in a port.

What happened in 1997? There's stuff from (GNU Emacs) 1985 / (Emacs) 1976.

What a waste of time.

Over the years I have see a number of posts about revamping emacs. Is there anything that actually works?

Atom is the “spirit” of emacs, taken into a world where your language of choice is Javascript & the runtime is the web browser.

Otherwise, not that I’m aware of. Everyone knows that Elisp is horribly crufty, but getting everyone to shift to a new language requires carrying all the extensions over which is a huge task.

To the point that it makes users of a modern PC in 2017 feel like they have traveled back in time and are using Emacs on a VAX/VMS system.

Yeah, the size of the binary distribution alone is kind of eye-watering. Google Chrome as a runtime is not a small dependency.


Practical, however, I have no idea whatsoever about.

How long before someone decides to port LLVM and jemalloc to rust? :)

Why use a GPL license ? This kind of port could have been the occasion to get rid of this type of license.

Why port Emacs to Rust ?, it adds no value to users. Why not develop a modern GUI like Intellij IDEA with a Common Lisp implementation underneath like SBCL ?

If you changed the license you'd have to do a clean room implementation, and even then deal with potential legal issues from the... personalities... associated with the GPL.

You mean the author who wrote his own license to specify how his work may be distributed?

Any person who write their own license to specify in fine details how the work may be used and distributed is going to have a very strong (and likely legal) issue with people who try to circumvent that license. Even a clean room implementation would need source material as instructions, and writing such specification without any copyrighted elements of the source would be quite a challenge when dealing with a very large program that has compatibility to a large ecosystem as its primary feature.

Because GPL license protects from stealing of code without contributed back.

Apple chose FreeBSD as base for it OS, Google chose Linux. Look at sad state of FreeBSD now.

Good luck!

Sounds like fun, I'm in.

Emacs is already struggling with a lack of core developers; porting it to a relatively obscure language won't alleviate that. I'd rather people help out with Emacs proper than doing silly stuff like this.

I know Rust fans are passionate, but we don't really need "X, but in Rust" for every value of X.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact