
Design of a Vim-like text editor - ProfDreamer
http://lists.suckless.org/dev/1409/23497.html
======
_b
Right now, using vim is better than using this new editor (vis), although vis
is close to being substantially better than vim.

I use the suckless window manager (dwm) and terminal (st), so gave vis a try
for a couple days. Three vim plugins made me return: git-gutters, you-
complete-me (ycm), and clang-format.

It is hard to extend vim's functionality. Its plugin interface has issues.
Because of that, ycm and git-gutters can not be used at the same time, and
something that ought to be straightforward like exposing clang's autocomplete
information requires a considerable engineering effort (ycm) spanning three
languages and about as much code as vis is total.

Giving vis good extensibility would be easier than fixed vim's plugin
interface (sorry neovim). My personal preference for doing this would be a
patch to vis that adds vim like gutters and dropdowns, and then patches that
require this one and add specific things like ycm and git-gutters. A
reasonable programmer could disagree and prefer out-of-process plugins. Either
approach being implemented well would be very exciting.

~~~
JackMorgan
With all sincerity, these plugins are available in Emacs, along with a Vim
emulator called Evil. I used to fight with Vim so much, but for several years
have been just using it inside Emacs, a decision I've never regretted.

~~~
exDM69
evil-mode is not without problems either. In particular, it doesn't work very
well outside of "regular" code/text buffers. Using it in GDB/scratch/other
buffers is very inconvenient.

So if migrating from Vim to Emacs + evil, you still have to learn how vanilla
Emacs works. At least this was my experience when trying it out.

~~~
gnuvince
You could add a hook to use evil only in certain modes, for instance:

    
    
        (add-hook 'prog-mode-hook 'evil-local-mode)
    

This activates evil only for modes that derive from prog-mode.

~~~
exDM69
Yes, that's the only reasonable way to go.

But you still need to learn how vanilla emacs works.

------
ridiculous_fish
This piqued my interest:

> handle arbitrary files (this includes large ones, think >100M SQL-dumps)

My old hex editor Hex Fiend was a serious attempt to handle arbitrary-sized
files correctly. It's hard! In particular, operations which are usually
instantaneous (e.g. Find) now may take a long time: they need progress
reporting and cancellation, and ideally should not be modal.

A text editor makes that even harder, because now simple operations like "go
to beginning of line" may take a long time if you have to find the beginning
of the line. There's probably some conditions that you could impose (e.g.
handles large files but not long lines); it would be interesting to see what
those are.

> Loading a file from disk is as simple as mmap

This is a sketchy decision. For one thing, it means you cannot work with files
larger than maybe 3 GB, or even 3 1 GB files, in a 32 bit process. I'd also be
uncomfortable relying on mmap over NFS.

Hex Fiend handled this by not mapping files, but reading them (via pread) on
demand.

> Since the buffers are append only and the spans/pieces are never destroyed
> undo/redo functionality is implemented by swapping the required spans/pieces
> back in.

Saving is what makes this tricky. Say I open a 10 GB file, delete it all, and
save it. Can I now undo that delete? (Hex Fiend initially could not, and users
were unhappy). If so, where does that 10 GB data live?

For that matter, how DO files get saved? Say I append 1 byte to the end of a
file: is the entire file rewritten? Say I delete 1 byte from the front of the
file: does it require twice the disk space to save it?

How about copy and paste? Say I open a 1 GB file, copy it, and paste it into
another. Does that 1 GB of data get copied into memory, or is it just
referenced in the original file? If it's referenced, what happens if I now
edit that original file - does the data get copied at that point?

Anyways this is really tricky (but fun) stuff, and I hope the author succeeds
since I do want a fast text editor that can operate on arbitrarily sized
files.

~~~
rtpg
>For one thing, it means you cannot work with files larger than maybe 3 GB, or
even 3 1 GB files, in a 32 bit process.

Serious question : is it bad to assume a 64-bit capable running environment
for developer's computers nowadays? I feel like it's better to run with
something simple (and way less error prone) if the cost is some 32-bit issues
like stated.

Granted, vim is a special case, since we're running this through ssh on
servers or whatnot.

Lots of really good cases mentioned in this post, though, bet you make a good
debugger

~~~
ridiculous_fish
There's definitely still new Linux systems that are 32 bit (Raspberry Pi, etc)
though probably not many as a developer computer. Maybe Crouton on a
Chromebook?

Anyways you have a point about the simplicity of mmap. Reading up on the
suckless philosophy, it does seem to fit.

~~~
mjcohen
I think Crouton installs 64-bit Linux.

------
gchp
I'm currently building a similar project in Rust called iota:
[https://github.com/gchp/iota](https://github.com/gchp/iota)

I don't intend it to be a vim clone, however I'm currently adding some
features which I've borrowed from vim. The main one being modal editing. Its
much earlier on than this project, though.

As a side note, building a text editor is great fun, one of the most
interesting projects I've worked on!

~~~
kzrdude
It looks cool, and I love Rust.

A question, which are your frustrations with other text editors? Can it really
be the case that you can't find an existing text editor that you like? :)

~~~
gchp
Another part of it which I didn't mention is that I've always wanted to build
a text editor, they've always intrigued me. So this was me scratching the itch
if you like. I also used it as an opportunity to learn rust, which I really
like, incidentally.

To answer your question, I actually hadn't found one that suited me, fully.
There are many that I like, but non which did it for me. It sounds a bit
weird, but it needs to feel right, which none of then did for me. So that's
why I started it. It seems others are interested in it too which is cool!

~~~
kzrdude
I was curious if you could describe the specific frustrations, and which of
these you can fix.

~~~
gchp
Right, here goes...

\- ability to handle large files without freezing or without a noticible drop
in performance

\- make all aspects of the editor scriptable. Emacs does this well, but I
really don't like elisp. It will also be possible to script it in a number of
languages, not limited to a single one.

\- fast, like really fast. My Emacs config takes ten seconds to load, using
IDEs takes much more than that to open. I want to make use of modern processor
capabilities and minimise load time. Also to parallelise as much of the
editing functionality as possible, so the user is never blocked by a task or
operation.

\- cross platform, easily installable. Some editors do this well, others
don't. With iota you'll be able yo just download a binary and run it. No
installation process, no dependencies. Same process to install on all
machines.

I guess those are the main ones. Am on mobile right now so its hard to give
more detail at the moment. I'll try update the motivations part of the readme
with information like this today.

------
Myk267
I'm always interested in the suckless projects. I like emacs and vim, too, but
it's interesting to see projects made which can just grab the fork, a sharp
knife, and a good pan and make do instead of necessarily including the kitchen
sink, if I may get the analogies out.

Also, no tests? Phew!

------
ryandvm
I've been using Vim for over a decade. I love it. But I feel more and more
anachronistic every time I have to hunt down some Vim plug-in for whatever IDE
I'm setting up.

I'd love to see a fresh take on hyper-efficient text editing in modern GUI
environments.

~~~
Slackwise
Well, with Neovim, we'll have a legit fully-featured plugin for any and all
IDEs that do plugins. I sincerely can't wait to have Neovim in
IntelliJ/RubyMine/CursiveClojure.

~~~
sigzero
IF the project survives.

~~~
jaredmcateer
It may be still too early to tell but it seems there is enough interest to
keep it alive. It's been almost a year since the project started and it has
over 120 contributors, over 2500 commits and doesn't seem very top loaded on
the project owner (who is responsible for only 18% of the commits).

~~~
MetaCosm
Which was one of the core goals. Spread the load, avoid single points of
failure, create a vibrant community of contributors, allow people in.

~~~
sigzero
I wasn't trying to be negative. There have been a lot of editor projects come
and go is all. I like the idea of neovim. I just with Bram was more flexible
so that neovim wasn't necessary.

------
lobster_johnson
The article mentions the Project Oberon book and the piece list algorithm,
which goes back to Xerox PARC's Bravo editor. Coincidentally I came across
this paper recently about different data structures for text editors, which
covers the piece list algorithm but also several others: "Data Structures for
Text Sequences" (1998), Charles Crowley
([https://www.cs.unm.edu/~crowley/papers/sds.pdf](https://www.cs.unm.edu/~crowley/papers/sds.pdf)).

~~~
edwintorok
There is also this old book about Emacs that describes data structures for
text editing:
[http://www.finseth.com/craft/#c6](http://www.finseth.com/craft/#c6)

------
mikejholly
So weird to see this at #1 since I was researching this exact topic last
night. My idea is a fully scriptable (Racket or CL) editor with modern package
management system. A tiny C core would handle rendering (curses, etc.). Very
cool write up though!

~~~
asprintf
As an alternative to ncurses, I encourage you to look at termbox[1]. It's
small, minimal, and very robust in my experience. (As a side note, the author
also wrote a text editor in Go that uses it.)

[1] [https://github.com/nsf/termbox](https://github.com/nsf/termbox)

~~~
mikejholly
I'll check it out. Thanks!

------
oscargrouch
Instructions to compile the source on Freebsd:

in config.mk, change the line :

CFLAGS += -std=c99 -Os ${INCS} -DVERSION=\"${VERSION}\" -DNDEBUG
-D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700

to:

CFLAGS += -std=c99 -Os ${INCS} -DVERSION=\"${VERSION}\" -DNDEBUG

The flags (the removed ones), trigger the "__BSD_VISIBLE 0" macro that makes
the signal "SIGWINCH" unavailable, as they are used in vis.c

------
faragon
The source code of vis looks great, and the executable takes about 100KiB
(x86-64). The editor does not behave exactly like vi/vim, but it is quite
acceptable.

I like a lot the suckless manifesto, recommended projects, etc.
[http://suckless.org/](http://suckless.org/)

------
spain
How about an embedded Lisp interpreter?

~~~
agumonkey
At one point, REPL first editors (ipython notebooks, etc) will meet edition
based IDE (them) in an essential cosmic spark and nothing will ever be the
same ever again.

~~~
mej10
IntelliJ sort of has this for Scala. Scala Worksheets. They are more useful
than a REPL, but not quite to iPython in terms of features (mainly because
they aren't browser based). But they also have some advantages over iPython
(like integrating many of the IDE features).

~~~
Fede_V
PyCharm now offers the ability to open iPython notebooks inside itself - which
gives you the nice IDE goodies, plus the notebook goodies.

The downside of course is that Python is very dynamic - so IDEs are of
relatively limited use.

~~~
mdaniel
_The downside of course is that Python is very dynamic - so IDEs are of
relatively limited use._

I am surprised to hear you say that with "PyCharm" being the first word in
your comment. We have had very good experiences with PyCharm +
reStructuredText annotations to help it when the return type or arg type is
not inferable. Just like its IJ friend, PyCharm has caught quite a few bugs
and helps the team edit _Python_ and not _text_.

PyCharm's usage is not optional on my team; we tried the "use whatever text
editor you want" approach and it turns out that people are not as good at
memorizing the type signatures of a complicated codebase as computers are.

------
PythonicAlpha
I would like to have a vim-clone, that also can handle embedded images.

Just a wish.

------
marktangotango
Very interesting. I also found quiet a lot of inspiration from reading about
the Oberon system. This author was inspired by the text editor, I was more
interested in the language. The author uses 'peice chains' as opposed to a gap
buffer which was new to me.

I'm sure many will say, why another editor? I say why not? Looks like a fun
project.

------
flavioribeiro
Great idea! I wonder why you aren't using github (or bitbucket, etc) for
hosting the project. I personally think that early projects can benefit from
tools like that, since people can follow the progress (issues, bugfixes, etc)
and access the code without cloning the repo.

~~~
SmileyKeith
It actually is on GitHub[0]. I'm not sure why it wasn't mentioned in the
mailing list.

[0]: [https://github.com/martanne/vis](https://github.com/martanne/vis)

------
shurcooL
That was a very interesting read. I might want to implement some of those
things in my text editing component.

I have one question. How would they implement "go to line number"
functionality? How fast would it be?

~~~
martanne
At the moment only the line number corresponding to the start of the viewable
area is cached. Further requests are served based on that. Since the
underlying pieces/buffers remain unchanged, one could also maintain a more
complex data structure such as a balanced binary search tree.

~~~
shurcooL
What about uncached? Imagine the user opens a file and immediately issues a
command "go to line 12345". Or is go to line not implemented at this time?

~~~
martanne
Then the underlying file is scanned. The performance of this depends on the
number of non contiguous changes made to the document. At load time i.e. with
no changes this will be relatively fast since it is a single memory mapped
piece/buffer.

Further request are then either served from line 12345 or the start of the
file, depending on which is nearer.

------
oneeyedpigeon
How do I submit a patch? Needs a couple of tiny mods to compile on OSX.

~~~
shurcooL
I would imagine you can submit a PR at
[https://github.com/martanne/vis/pulls](https://github.com/martanne/vis/pulls).
He says "As always, comments and patches welcome!" in the README.

Edit: Maybe not, the mailing list mail says "git clone
git://repo.or.cz/vis.git" instead. I'm not sure where the canonical source
lives.

------
ponyous
I think that explaining core of the editor to the people is whole point of how
to get people to use/extend/modify it. And this post really hit the nail on
the head.

------
joelthelion
I didn't see plug-in support in that list. Excellent, first class plug-in
support is an absolute must have for a good modern text editor.

------
thu
The piece table/chain description makes me think to an interactive xdelta
(which is for instance used in Git).

------
Zardoz84
And neovim ??

------
0xdeadbeefbabe
You don't have to use posix to suck less you know, or do you?

------
ExpiredLink
Most of the "main goals" are supported by ... Eclipse!

~~~
Crito
Consider the following: multi-tools, mess kits, and sporks exist, yet many
people prefer using the specialized equivalents.

~~~
coldtea
The analogy breaks very fast, as people who prefer the specialized equivalents
don't go sticking tons of smaller pieces together to try to make them resemble
a multi-tool functionality, whereas Vim and Emacs users do add tons of plugins
to get IDE like behavior.

FWIW, I think the aversion to IDEs is bogus -- unless it comes from people who
don't install plugins on their Vim/Emacs. If they do, then their problem is
not IDEs per se, but how popular IDEs are implemented.

The gist of an IDE is "having all the functionality in one place": edit,
search, refactor, browse code, scm, REPL, build, etc. People loading Vim/Emacs
with plugin try to build the same thing essentially.

The problem with IDEs is not their "IDE-ness" but: 1) they lack a decent,
fast, editor component. 2) they are slow to load and memory hogs.

Nothing of the two is inevitable. Just an issue with current implementations.
For (1) we can image an ide with a full Vim component, for example, as it's
editor part. For (2), the problem is that both 2 popular IDEs (Eclipse, Idea)
are built with either Java (which is crappy for GUIs), or have a messed up GUI
like Visual Studio.

------
amelius
Why not write a rich-text editor instead? Or a webbrowser?

Seriously. They seem like more interesting endeavours to me. Especially if you
document every step of the way. In fact, you could write a book about it.

~~~
lunixbochs
[http://surf.suckless.org/](http://surf.suckless.org/) is already a thing.

~~~
agumonkey
Never noticed the logo, elegantly regular and cryptic.

