

Color_coded: Real-time semantic highlighting in vim with libclang - Jeaye
https://github.com/jeaye/color_coded

======
unfamiliar
This is one thing I would love to see in sublime text.

Isn't the name a bit vague though? "color_coded" could refer to anything,
surely something that references clang directly would be more appropriate?

~~~
jokoon
Syntax highlighting is a complex problem, because it needs to be fast enough,
and the more token types you have, the longer it takes.

Each time you type a character, you have to parse the code (clang is used
here), select the according color, and draw the text (often the OS will deal
with this, and I don't know if you can tell an OS to just update one line of
drawn code).

This is a vim plugin, and it's way easier to colorize text in a console
environment than in a cross platform GUI window (not to mention UI text
editors use a dedicated data structure, the rope:
[http://en.wikipedia.org/wiki/Rope_%28data_structure%29](http://en.wikipedia.org/wiki/Rope_%28data_structure%29)
to make text editing responsive when using the mouse).

My guess is that sublime text completely shortcuts this problem by using
webkit to draw its own text surface.

Anyway, it's another example of how competing windowing systems are inflexible
and make things harder on the programmers, which often overcome the problem by
avoiding the OS. Cross platform software programming is more than a philosophy
or an art, it's a war against OS manufacturers. It's won by finding the
easiest solution to make it work on all platforms.

In this case, you can clearly understand how making an editor with syntax
highlighting requires many different things, so editing the colors and adding
a language will greatly improve complexity.

My take is that one day someone will have the decency to just release an OS
that just offer developers to use the hardware directly (or something like
cairo) to draw even a simple window, without anything like Xorg, android,
wayland, gnome, KDE, etc. The cost of coding something to display the results
of your software is way too high when you want to do it the "mainstream way".
I'm a little angry at Torvalds, not having tried to introduce something to
build simpler GUI stuff.

So, in short, don't expect it to work on sublime text if the dev don't get
involved. I guess it's possible since sublime text support scripting, but I
wonder if it's possible to do it with decent performance. Even Sublime text
start to hang a little when you have a large html file.

I was once very frustrated with text editors (always some missing feature, and
I never accepted to switch to vim or emacs), so I tried to make my own, and I
failed miserably and settled for what is already there. I still wonder if I
can be more productive with vim, but if it becomes necessary to use it because
a 11" laptop can't run an IDE, maybe I'll do the switch.

~~~
awalton
> Syntax highlighting is a complex problem, because it needs to be fast
> enough, and the more token types you have, the longer it takes.

Yes, it scales with the number of tokens. Why wouldn't it? This isn't a
problem. It's 2015, we have ~4GHz umptheenth core computers, we can do this in
real-time.

> Each time you type a character, you have to parse the code (clang is used
> here), select the according color, and draw the text

You seem to have no idea what is actually happening here. You don't need to
rerun the whole parsing step at each character if your tool understands the
language, only when you hit a token that changes the parser's state (so when
you enter a paren, you're no longer typing an identifier, so now we need to
reparse.) There are a huge number of ways to optimize this step from there,
but I don't think anyone is actually employing any of them yet, either because
they're too complicated or computers are just so fast we don't care (like
maintaining a document AST and simply updating the nodes as they change around
so you can get away with vastly simpler "reparses").

The actual layout and rendering of text is then done - each OS has their own
text layout engine, and there's several open source ones (e.g. Mozilla and
Blink/Webkit have their own text layout kits, Pango does it on GNOME, etc).
Color highlighting is a part of text rendering, which happens next, which also
every platform has it's own solution for (like, e.g. Cairo, an image
compositing library, which uses FreeType, the Windows or Mac font rendering
systems for rendering glyphs).

What should be colorized/bolded/treated as different is usually tracked by a
set of span data structures (e.g. line number X, character Y to character Z
should have this set of attributes applied).

> My guess is that sublime text completely shortcuts this problem by using
> webkit to draw its own text surface.

Sublime Text implements its own editor widget. Here is the author noting some
bits about it:
[https://news.ycombinator.com/item?id=2822114](https://news.ycombinator.com/item?id=2822114)
. Shipping Webkit is not an easy task, nor is it a small one in disk
footprint. I'm not sure how you randomly came to the conclusion Webkit had to
be involved.

The rest of your post seems as if it was generated by some kind of buzzword
Markov Chain, so I'm ignoring it.

~~~
jokoon
okay

------
adrusi
Doesn't seem to work in neovim currently, as lua support is being completely
changed.

~~~
Jeaye
Right. This hasn't been tested in neovim at all. I can add that to the readme
to save people some time.

------
hboon
On the off chance that Jeaye is still reading. How do i get it to work with
Objective C? I tried:

-x objective-c

Nothing for CCerror. `make run` with the .cpp sample works.

------
en4bz
Really wish projects like this and YCM would stop including most of Boost and
provide an option to use the installed system version instead.

~~~
Jeaye
With the annoying number of systems that don't have boost installed, such
projects are forced to accommodate (by shipping with their own). After the
accommodations are made, there's little reason to allow for the usage of the
system's boost.

Similarly, color_coded downloads its own clang. Why?

1) People may not have clang installed (like boost) 2) (more importantly)
color_coded is tested and proven to work with a specific version; it's more
portable this way

If we suddenly allow users to use system-level boost and clang and foobar
implementations instead of that which we ship, we're also opening a can of
worms for incompatibilities. The amount of coverage that needs to be tested
grows surprisingly quickly.

------
farresito
Thank you for your work on this! I will surely give it a try.

------
fourier
Will be great to have something like this in Emacs.

~~~
Jeaye
After looking, I'm astonished that this doesn't exist in Emacs. There is a
completion plugin using libclang; maybe the highlighting could just be added
into it.

On vim's side, YouCompleteMe explicitly denied my approach to integrate
highlighting.

~~~
captainmuon
I hacked together something to add at least #ifdef highlighting* to
YouCompleteMe. It's not quite ready to share due to some bugs, but I'm
planning to release it as a patch / friendly fork when I find some time.

I believe there's no reason to have two plugins each running an instance of
clang.

* i.e. showing parts of code that are ifdef'ed out in grey

~~~
lsiebert
it would be nice if it could hook into YCM's config file instead of needing a
separate one, when available.

~~~
Jeaye
Yeah, that's something I considered from the start. The issue is that, in
order for that to work reliably, color_coded would basically need an embedded
Python interpreter.

YCM's config file can contain arbitrary python functions and doesn't really
need to follow any standards. This makes it hard on color_coded and ultimately
didn't seem worth the trouble.

------
gcb0
sigh. yet another plugin that uses more files then a simple plugins/x.vim that
has no manual installation instructions.

~~~
Jeaye
Did you look at the 200 line readme, specifically the part labeled
"Installation"?

~~~
gcb0
yes i did. the 'generic' install is using pathogen or vundle.

------
slashnull
Isn't that like HDevTools for C++?

Neat.

------
bowlofstew
That is awesome. I am going to give that a shot today.

