Obviously cdp only applies to the communication mechanisms relevant to control a browser — i guess what I’m musing about is wanting to Suss out - is there anything that will be lost (dx feature support, performance) if one attempts to implement ide features but using debug adapter protocol to adapt CDP? Is CDP one of the target integrations that debug adapter protocol will aim to support?
I think CDP is ridiculously important and likely to become more so ...
Has anyone used some of their abstractions elsewhere? How good are these protocols they are pumping out?
Adoption wise, I'd say it's a success. As for how good the protocol is, I'd personally have preferred something binary instead of json.
Documentation can be found:
Seems to work fine. Functionally it seems the same, maybe a bit faster. Haven't used it a whole lot.
Really tempted to try out the Microsoft PLS just so that I can say that I am using Microsoft in Emacs. Which is pretty close to being technically correct.
Brave new world.
Nowadays if I was forced to give up my Linux desktop and choose between Windows and OS X for work... Windows 100% of the time. Because I can then use WSL and get Linux nicely integrated into the base OS. Especially since Macbook Pros are not that nice anymore.
I didnt see how it works on Sublime Text yet. Feels like plugins on Sublime dont get the love and quality as plugins in VS Code or Atom.
Wonder if Spacemacs / Spacevim will natively support LSP and DAP. Would be really nice.
But what I would care about is that it's easy to create a protocol specification and to generate most of the boilerplate code out of it - so that the schema gets automatically verified and that developers can focus on building their application logic instead of getting the serialization right. That can be done with JSON too (e.g. with json-schema or by generating code out of typescript definitions). However I haven't yet checked whether LSP and the debug protocol make use of this.
Binary protocols are easier than JSON, to me, and to most people I would say, who have actually implemented a binary protocol or wrote code to read or write binary files.
Especially not with broad vagaries like "Binary protocols are easier than JSON".
A computer program (an IDE in this scenario) talking to another computer program (the debugger) simply does not need to use JSON! By choosing to use JSON, a choice was made to make the protocol far more inefficient than it needed to be.
The result will be a slower than necessary debugging experience for all who use it. Debuggers already consume enough developer time per day! Choices should be made to make developer lives better, not worse.
You're speaking super hand wavy though. What specific protocol is better than JSON? I'm sure there are plenty, but that's not the point.
The point is, if it's not as common as JSON then it can hardly be considered easier, can it? If I have to write a implementation of X protocol just to work with this DAP or LSP, is that really easier?
Imo, commonality is hugely important when you're trying to work with every editor and every language (that editors are written in) out there.
I'm not saying there isn't a binary protocol that would be in every language out there. However JSON basically is, at the very least. So lets talk apples to apples about what you think is better, because "Binary protocols" is the vague sense are not even remotely as common and widely supported as JSON.
Don't get me wrong, I sort of hate JSON. Yet for many applications, especially those seeking compliance with everything under the sun, it has merit.
The format needed to solve the problem you're working on.
> The point is, if it's not as common as JSON then it can hardly be considered easier, can it? If I have to write a implementation of X protocol just to work with this DAP or LSP, is that really easier?
If it's easier to understand and to write, then yes. That's what "easier" means.
> Imo, commonality is hugely important when you're trying to work with every editor and every language (that editors are written in) out there.
"commonality?" Do you mean "usage?" If it's easy to write code to handle a given binary format that you need for your own problems, writing libraries for other languages is essentially trivial, and you avoid sitting underneath all of those JSON libraries, which is a huge win!
> Don't get me wrong, I sort of hate JSON. Yet for many applications, especially those seeking compliance with everything under the sun, it has merit.
It does have merit, you're right; and I am not saying that a binary file/protocol should be used in all cases, and JSON in none. I am saying that the performance lost by using JSON is significant, even though it feels like it is not. I am saying that when you choose to use JSON when a binary format suited to the problem would suffice, that you are necessarily wasting time and effort that could be solved actually solving the problem you need to solve. I am saying that JSON, in most problems that I have seen it applied to, is not the best solution, not the fastest solution, not the most effective solution, and not the easiest solution to fix when something breaks. When those four things are true is when JSON seems to be used the most often, and that is what I have a problem with.
Are we software engineers or are we not? I argue that we're not, despite demanding that we be called engineers, in many cases. We don't measure what we do to see what is most efficient or has the best performance or the lowest cost. We don't start with a budget of some kind then write our software so that we do not exceed it. We rationalize our software development decisions away as being "good enough" without doing ANY measurement against our own imaginary acceptance criteria or really any solid justification of that position at all. We count on users not noticing delays or not caring enough about them so we can be lazy and/or save time in development. I think those things are a problem.
Programming is not an art. Everything about programming and the programs created, including developers and their time, can be measured and improved. Virtually no one does this. Virtually no one seeks to improve their skills solely so they can become better and produce better results; everyone is concerned about the newest technology coming down the pipe because it is "cool" and not even considering its applicability to a given problem.
Obviously I have a lot to say about this. I think I've said enough, and I'm sorry I dumped this all in a reply to you. It certainly is not all directed at you, specifically.
It seems like the issue is understood: people prefer string-based protocols because they can read and interpret them at a glance with no ramp up time, whereas a binary protocol requires extra tooling and manual translation (at least until the programmer memorizes it).
Whether or not programmer comfort/ease should be the primary driver of development decisions, it's hard to pretend like it's not at this point. Probably competes only with "buzzword needed for my resume" in terms of the true rationale behind system design, regardless of the flimsy justifications that people may try to make up.
Taking that into consideration, the reason MS may release a debug protocol based on JSON is that they want developers to use it, and developers will always use the thing that looks simplest at a superficial glance.
I've found, for example, that most devs will cram arbitrarily large quantities of production-critical data into anything that vaguely resembles a data hole and then yell and scream when you try to explain the meaning of "cache", "ramdisk", or even "temporary" to them. Expecting more from the developing public at large is just setting oneself up for failure.
I was going to point out how disgustingly condescending this was and how it reeks of someone who overinflates their own sense of ability via ego... but then I read your profile which brags about getting muted and whatnot and realized that's exactly what you take pride in so...
It's a conclusion that's been burned into me after many years of attempting to recover from data loss events precipitated by people who just want to get their boxes ticked in the most convenient way possible and get home, with no regard for whether what they're doing is sustainable or appropriate. If you haven't been exposed to that in the course of your career, then I'm happy for you.
As for "bragging" about HN's censorship, the snippet was added after people emailed to ask why I had stopped responding to them. I can see where you're coming from on the tone and I'll try to make a few adjustments and see where that gets us. Thanks for sharing your POV -- honest feedback is hard to come by. :)
You're describing easier for a computer, which isn't really relevant. For a developer, parsing JSON is clearly easier than parsing a binary format. For effectively every major language nowadays, it's a single function call, and you're done, it's hard to beat that.
In practice, sending & parsing JSON instead of a binary format is not going to make your debugging experience here any _noticeably_ slower (if you have a DAP case where it does, I'd love to see it) and it is going to make development quicker & easier, and make the traffic inflight easier to debug too.
It's a tradeoff of course, but for most cases it's worthwhile.
If you're talking about the ability of the human eye and brain to see the binary format and understand it, then yes. JSON is pre-humanized. I would say that it's pretty easy to write a visualizer for binary data if you know the format, and I would add that the time taken to write that visualizer is trivial compared to the execution time saved by using a binary file or protocol over JSON over even a short period of time.
> In practice, sending & parsing JSON instead of a binary format is not going to make your debugging experience here any _noticeably_ slower (if you have a DAP case where it does, I'd love to see it) and it is going to make development quicker & easier, and make the traffic inflight easier to debug too.
Debugging performance of a debugger is orders of magnitude larger than the time saved by using a binary file/format, you're right. That's a problem with debuggers, not with binary data storage.
> It's a tradeoff of course, but for most cases it's worthwhile.
Yeah for many cases JSON is a fine choice. For a protocol that goes between a computer program and another computer program (such as the IDE and the debugger process, as in the use case of the Debug Adapter Protocol) it is a bad choice. It's necessarily slower and more wasteful of bandwidth and CPU time than a binary protocol would have been. Debuggers are slow enough, they don't need to be made slower by using JSON.
Does not mean anything.
Who defines the "difficulty" of a protocol based on serialization?!
What is that even supposed to mean?
The protocol can be hard to understand, it can make unreasonable demands of users, it can have a ton of bloat or complex backwards compatibility rules.
But the serialization method literally orthogonal to complexity of the protocol itself
Implementing a protobuf serializer or implementing a JSON serializer would be completely 100% outside of the scope of a protocol that uses either.
Doubly so when you're talking about JSON, which has serializers just about everywhere, so you can't even say "oh well the protocol is easy to understand but the serialization method is hard to implement".
If anything using a custom binary serialization implementation would be the worst of both worlds, now consumers have to implement custom serialization AND you have to solve all the issues other solutions did like endianness, backwards compatibility... not having gaping security vulnerabilities...
This is like saying a protocol is "difficult" because you used it over HTTP so now somehow it inherited the entire complexity of HTTP.
Their protocol's complexity isn't defined by it's serialization any more than it is by the transport they're using.
I won't even bother replying to the rest of that, just very vague handwavy rhetoric that's not saying anything much.
Can I use the protocol with simple command line tools or do I have to spend several hours coding up a protocol handler?
Will implementers be more or less likely to fuck up their implementation of a binary protocol?
How will you version the binary protocol? How will you maintain backwards compatibility?
Of course you do, something has to actually read the strings and parse them into JS data structures.
(yes, there are 3rd party libraries that provide some or all of the above, but they may not be applicable for a variety of reasons, including API stability, maintainability, compiler support, host/target OS support, tooling support, C language version support and a bunch of others)
You don't need protobuf, you don't need json, you don't need any of these things that just add bloat and cost you performance.
Writing and reading binary is easy, and is FAR easier than JSON if you are using something like C.
Most languages have convenience methods for strings, yes, and those can be stacked on top of each other so that JSON libraries can be written, of course. Those libraries are FAR slower than just reading and writing bytes, and deserializing those bytes into structs or serializing those structs into a byte array, and the binary data is much smaller than a JSON document containing the same data would be.
People these days are acting like binary protocols are hard, or that they are suboptimal for one reason or another. I guarantee you that someone who was trying to find a bug in a JSON library will spend as much time or more than someone trying to find a similar bug in a method that reads or writes binary data, and at the end of it, the person who looked at the methods that handle binary data will FULLY understand what is happening in that code. The person debugging the JSON library will almost certainly not have a full understanding of the JSON library after finding and fixing a bug unless they wrote it to begin with.
It comes down to this - all computer programs do is process data. All UX problems, all usability problems, all computer problems boil down to problems transforming data. Transforming data is literally the only thing a computer can do. Any work spent writing code that does not directly solve the data transformation problem at hand is wasted time and work on the part of the developer. Any performance lost using code that does not directly solve the data transformation problem at hand is time away from the CPU for other tasks, time away from a consumer of that data, and so on. It adds up, and it adds up quickly.
I call them the peanut gallery and the best way to deal when them is to pat them on their head for being so smart, thank them for their opinion and ignore everything they said. Failure to mute them will derail any project lead by a junior dev who doesn’t get know how to deal with pot shots from the peanut gallery. The poor junior dev will actually listen to the worthless feedback and stall the project.
I’m sure the folks who worked on this protocol are rolling their eyes at this comment thread...
Yeah more or less that's true, but in typed languages especially it's pretty inconvenient (unless you circumvent the type system or dump directly into a hash table, but that's usually bad).
> It's easy to debug
There are tons of corner cases re: parsing JSON . Plus things like canonicalization and duplicate keys are pretty confounding.
Given all the other constraints on the product, how does making a binary protocol fit in? What trade offs will be made if they used a binary protocol?
This is a move toward being able to debug programs on Azure with VSCode remotely.
There's a lot of talk about fast C++ JSON libraries and how they can parse 1GB of JSON per second, and that's a lot of data, for sure, if you only count the data that's wrapped up in JSON. 1GB of JSON data contains a lot of waste that is not even usable data, and that's before you consider the textual encoding of what may better be represented by binary data, which inflates the size of a JSON file even further than the JSON syntax itself does. A binary file that is read at 1GB per second would contain far, far more actual data, and binary files can be read and written much more quickly than 1GB/s.
I'm really starting to lose patience with all the JSON and web technologies that are finding their way into non-web use cases. using binary formats and simple programs to handle them is really, truly, easier to rationalize about, easier to maintain, easier to write, and easier to replace when the time comes.
1) Write a JSON library from scratch.
2) Write a method to handle a binary format from scratch.
3) Compare them.
I understand that you are likely relying on some JSON library or language feature, and I would argue that you will bear the weight of that all of that code the instant it has any problem, and that you bear the performance cost every time you use it. Maybe that library or language feature doesn't have any functionality bugs that you encounter; you will still pay for the loss in performance for as long as you use it.
Maybe this is acceptable. For me, if I'm writing a tool that either delivers a result that a human relies on, or is a tool that a human uses, I consider it immoral to waste that human's time without a very good reason.
I always try to avoid wasting my own time. Reasoning about code that reads and writes binary files is much easier for me than reasoning about an entire JSON library.
Desktop, laptop, and tablet CPUs are not getting faster overall. In fact, they're getting slower as cores are added to the silicon dies.
We've lived as developers for far too long under the performance gains provided by hardware advancement. Software has generally improved performance only when put on newer and faster hardware platforms. As a general rule, faster hardware platforms aren't a guarantee anymore. We may have more cores to run on, or more CPUs thrown at us to improve performance, yes. We can improve performance in other ways, with zero hardware cost. Avoiding JSON unless it is actually needed is one way we can reclaim performance from the debt of our previous decisions.
1. Write your own C compiler.
2. Implement your binary encoding/decoding.
Further popular counterexamples besides the NT kernel (cf. ):
- Joilet file system (for CD-ROMs)
As you can see in the examples, they are all (except perhaps the Joilet file system) still very actively used. I see no reason that they will disappear soon.
Also the decision to use UTF-8 under GNU/Linux is a decision that was similarly made about 20 years ago.
First of all, there's graphemes are inherently not one-to-one with code points, e.g. Á = A + `. There's simply no Unicode encoding that will let you safely index into an array without paying attention to the meaning of the underlying codepoints. (and no, using NFC won't solve this either, because there are combinations for which there's no composed equivalent)
Secondly, general formatting info won't fit into 11 bits (italic, bold, underline, strikethrough - that's already 4 bits, and we haven't talked about color, font weights other than bold, etc.), so why bother baking in a limited, intentionally gimped version into your character encoding?
It is not a "very bad idea" it is "an idea you do not like." Those are different things.
The way you're describing UTF-32, it can't work at all, and it definitely does.
Trying to save space by using UTF-8 over UTF-32 seems like a very small gain to me, is all. UTF-32 is simpler, for text created in that encoding.
We can discuss how Emacs and VSCode compare to each other (and we can discuss the advantages and disadvantages of VSCode taking the electron approach until we're blue in the face), but VSCode is definitely not trying to compete with Emacs, rather aiming for a very different user base.
However, this article isn't even about VS Code proper. It's about Debug Adapter Protocol. If DAP achieves the same adoption as LSP, this will be a huge move forward in developer tooling across all sorts of tech stacks and domains. In between LSP and DAP, just about any editor is capable of becoming a lightweight IDE for an enormous variety of languages, and this is a Good Thing for everybody.
They just haven't been friendly much towards developers outside their ecosystem, as soon as they have monopolised the given ecosystem.
The thing is, it's the idea of Emacs that's important; you could argue that Emacs itself is an antiquated and idiosyncratic implementation of that idea. I'm fine with parallel attempts at programmable editors; seems like a win.
Both do not scale up to the needs of today (Windows support, Remote Support, solving the n x m problem, ...).