
Apple's investment into Clangd and refactoring tooling - artisdom
http://lists.llvm.org/pipermail/cfe-dev/2018-April/057668.html
======
DannyBee
So, just because there seems to be confusion:

Clangd already exists. It already supports LSP. It is already part of the llvm
project. It was written by !Apple, and used by a wide number of people.

(I want to make sure these people still get credit, i already see people in
this thread thanking apple for supporting LSP, which is not actually what
happened).

What apple has proposed contributing is: Making clangd work with xcode, and
extensions to LSP.

This is great (as long as the second ends up standardized over time).

But the benefits people listed here so far already exist in clangd. The main
benefit to work is "xcode support" and "better refactoring" which is awesome
:)

~~~
oflannabhra
The proposal is to split the LSP implementation into a transport layer and a
logical layer. The current LSP implementation is directly tied to JSON-RPC.

Apple's end goal is, as you say, to get clangd to work with Xcode. However,
having a replaceable transport layer for LSP could benefit lots of people
other than Apple.

~~~
DannyBee
Sure, it's nice, but again, being pragmatic, nobody else has desired such a
thing that much.

Ie so far the "lots of people" have not materialized, even outside clangd.

I can certainly think of use cases for such a thing, but most people seem
happy enough with json-rpc at the moment.

Part of that is likely that trying to speak LSP over a non local connection
may be a fools errand anyway. It's a chatty protocol no matter what the
transport.

~~~
oflannabhra
Just because Apple is the first doesn't mean there won't be others. Maybe
others have not materialized because the upfront cost of updating clangd was
judged too high.

JSON-RPC has several alternatives that very large software companies have
vested interests in: gRPC and XPC.

You seem more knowledgeable of these technologies than I am, though. It just
seems to me a good thing that 1) Apple, and consequently, all iOS and macOS
developers will be using clangd, and 2) Apple is putting significant effort
into improving clangd. I'm not sure why any of that news or optimism needs to
be quenched.

~~~
DannyBee
My point was precisely that plenty of languages have LSP implementations at
this point, an f even in those, nobody has cared much to try to find an
alternative to json rpc.

As for 2, all I'll say is based on history in llvm projects, I'm not anywhere
near as optimistic as you. The history here is simply not one of all great
things. The PR is, but the reality is much more mixed (some really great
things, some really not so great). So while I certainly am hopeful, it's
definitely not the unbridled optimism everyone displays here.

~~~
kenferry
You don't think apple's support of llvm and clang has been important to the
project? That's REALLY pushing it.

~~~
DannyBee
That's not what i said at all. I said some things have been really great, some
really not so great. This isn't even a controversial view among long-time
llvmers.

Heck, last time i asked him (a few months ago), even Chris Lattner agreed with
my view, and he used to run his stuff at Apple, so i don't think it's
unreasonable.

I think it's perfectly okay to be cautiously optimistic instead of fawningly
happy.

Do you actually have data or a real argument you'd like to use to suggest
otherwise (IE that it's been all overwhelmingly positive?)

------
saagarjha
I'm glad to see that Apple is putting work into making their tooling better
and easier to work with, but I'm still not clear on the benefits of this
change. Would someone care to enlighten me? Currently, the benefits of
libclang that I see are that I can easily drop it into a project without
requiring a lot of dependencies, and I don't need to figure out how to get
permission to run different processes and manage interprocess communication
between them–if anyone doubts that this will be an issue, on iOS neither of
these is really supported at all. In addition, I don't require a daemon to
constantly be running to analyze my code. (Why is this bad? Take a look at
Swift's sourcekitd.)

~~~
vhbit
> if anyone doubts that this will be an issue, on iOS neither of these is
> really supported at all

What for you might need it on iOS? Server is meant to run on a development
machine.

~~~
JDevlieghere
> What for you might need it on iOS? Server is meant to run on a development
> machine.

While I totally agree, I can see the author's point. Think for example about
Swift Playgrounds.

------
CodeArtisan
There is an article from 2010 titled "Emacs is dead" that is being reposted on
HN from time to time. The author args that the greatness of Emacs is to rely
on external, editor agnostic tools where text act as an universal
interface/medium but that the practice is fading out, and thus the dead of
Emacs. Now, ten years later, Apple announces this. Emacs, the undead?

[https://tkf.github.io/2013/06/04/Emacs-is-
dead.html](https://tkf.github.io/2013/06/04/Emacs-is-dead.html)

~~~
oblio
I think you might be right, but the greater context is highly ironic.

The company resurrecting the Emacs concepts is Microsoft. They're the ones
that created OmniSharp, bringing the idea of language servers to the
programming mainstream. They then created the Language Server Protocol for
what now is Emacs's biggest long term rival in the flexible editor arena:
Visual Studio Code.

Visual Studio Code is for the moment on a somewhat bloated and shaky
foundation because of Electron. But otherwise its design is quite solid. And
even that foundation will probably become a lot stronger in the next few years
as WebAssembly gains wide adoption. Visual Studio Code itself is written in
Typescript and it's not hard to imagine Microsoft adding a WASM backend to it.

~~~
terminalcommand
I think Emacs's greatest strength comes from the ability to use a single
scripting language throughout the whole editor. I can quickly write up some
ELISP statement to do anything, if I like it I can put it in a file and load
it. If I want to share it, I put it in an extension.

I've just looked up the official tutorial on making an extension in VS Code.
It seemed cumbersome. You need a code generator to lay the foundations, you
need to consult the API docs etc.

With Emacs you can learn as you go and the editor itself guides you if you
ever get stuck. You can read the source of any part of Emacs you like, change
it on the fly, evaluate and try it (except for some core modules written in
C). And I think that's the beauty of Emacs. Emacs is easy to tinker with.

~~~
oblio
Yeah, but I have a million monkeys^Wdevelopers at my back for
Javascript/Typescript. How many divisions does Lisp have?

~~~
fourthark
Out of curiosity I looked it up. It's a factor of 200: 3788 packages on MELPA,
vs 650000 packages on NPM.

------
pjmlp
Looking forward to it.

It is really nice to finally start having C++ environments close to the
promise of Energize C++ and Visual Age C++ 4.

~~~
codetrotter
[https://www.reddit.com/r/programming/comments/25r6pw/a_demo_...](https://www.reddit.com/r/programming/comments/25r6pw/a_demo_of_lucids_1993_graphical_cc_programming/)

They mentioned incremental compilation in the first few seconds of the video.
Didn’t watch the rest of the video yet.

I’ve always wished for incremental compilation. Imagine incremental
compilation that was so fine grained it would recompile only the functions you
changed. Of course this makes full-program optimization impossible but I think
most of us don’t compile at the highest optimization levels during development
anyway because doing so slows down builds without any benefit for development
builds. For production builds we use the highest optimization levels of
course.

Additionally I would like to have hot patching. I know some people have this,
personally I’ve never had that.

If anyone knows about any systems for hot patching Rust code please let me
know :)

~~~
pjmlp
While not perfect, Visual C++ makes it a nice experience.

[https://blogs.msdn.microsoft.com/vcblog/2016/10/05/faster-c-...](https://blogs.msdn.microsoft.com/vcblog/2016/10/05/faster-
c-build-cycle-in-vs-15-with-debugfastlink/)

[https://msdn.microsoft.com/en-
us/library/4khtbfyf.aspx](https://msdn.microsoft.com/en-
us/library/4khtbfyf.aspx)

------
fowl2
I'm assuming this is the "LSP" discussed?
[https://microsoft.github.io/language-server-
protocol/](https://microsoft.github.io/language-server-protocol/)

Mapping json-rpc onto xpc... Sure why not

~~~
fokinsean
Woah I had never seen this before. That's really cool. So in theory you can
make your own editor and provide a ton of IDE-like features by communicating
with this server?

Are there any other implementations of LSP by other orgs?

~~~
evmar
[https://langserver.org/](https://langserver.org/)

------
jupp0r
I have mixed feelings about this.

On one hand it's really nice to see Apple jump on the LSP train and use
standard tooling for their IDE to interact with language tools.

On the other hand, instead of having a consistent strategy of using proper LSP
(which by specification is JSONRPC), they are shoehorning LSP on top of their
in-house XPC transport. To do that they're planning to introduce additional
complexity to clangd (a transport abstraction), while actually defeating one
of the main purposes of LSP, which is reducing the m-times-n complexity
problem of matching compilers with tools to an m-plus-n complexity problem. No
other tools (unless they support LSP-over-XPC as well) will be able to talk to
XCode. XCode won't be able to talk to other tools. I hope they rethink that
decision, keep clangd simple and instead adopt proper (JSONRPC) LSP in all of
their other tools for Swift, etc instead. That way they'd not only open up
XCode to clangd, but also all other editors to their refactoring tools for
Swift, etc.

------
raverbashing
Could someone explain what's the difference between both approaches and what
are the advantages of Clangd?

~~~
ninkendo
Moving language parsing/refactoring/etc into a separate daemon is a step
towards unifying tooling around specific protocols for interacting with
compilers. Microsoft's effort towards a single language server protocol is
worth mentioning: [https://github.com/Microsoft/language-server-
protocol](https://github.com/Microsoft/language-server-protocol)

I'd love to see a future when new languages/compilers can implement this
protocol for their external compiling daemons and just get support in all the
various text editors/IDEs automatically.

------
vhbit
I hope that it also means there will be Apple-provided LSP server for Swift
soon.

~~~
solarexplorer
There is one already: SourceKit [http://www.jpsim.com/uncovering-
sourcekit/](http://www.jpsim.com/uncovering-sourcekit/)

------
woolvalley
This is great news! More parts of xcode are being exposed as open source and
clangd is getting more engineering resources.

------
Game_Ender
Does anyone have any details about Apples cross language indexer they mention?
A good indexer is the key to and efficient language server and I think
clangd’s is a little weak right now.

------
flamedoge
I tried building Clangd in trunk but failed to build.

~~~
DannyBee
We build and use it every day, so ... report it on the list?

