Code generation and strong contracts are good (and C#/Java developers have been doing this forever with SOAP/XML), but they do place some serious restrictions on flexibility.
I’m not sure how gRPC handles this, but adding an additional field to a SOAP interface meant regenerating code across all the clients else they would fail at runtime while deserializing payloads.
A plus for GraphQL is that because each client request is a custom query, new fields added to the server have no impact on existing client code. Facebook famously said in one of their earlier talks on GraphQL that they didn’t version their API, and have never had a breaking change.
Really, I don’t gRPC and GraphQL should even be compared since they support radically different use cases.
> I’m not sure how gRPC handles this, but adding an additional field to a SOAP interface meant regenerating code across all the clients else they would fail at runtime while deserializing payloads.
This is basically the reason every field in proto2 will be marked optional and proto3 is “optional” by default. IIRC the spec will just ignore these fields if they aren’t set or if they are present but it doesn’t know how to use them (but won’t delete them, if it needs to be forwarded. Of course this only works if you don’t reserialize it. Edit: this is not true see below).
Even if a process re-serializes a message, unknown fields will be preserved, if using the official protobuf libraries proto2 or 3.5 and later. Only in 3.0 did they drop unknown fields, which was complete lunacy. That decision was reverted for proto 3.5.
Some of the APIs (C++ for example) provide methods to access unknown fields, in case they were only mostly unknown.
I do remember some debate around required/optional in proto2, and how the consensus was to not have required in proto3 - parties had good arguments on both sides, but I think ultimately the backward-forward compatibility argument won. With required, you can never retire a field, and older service/client would not work correctly. I haven't used proto2 since then, been using proto3 - but was not aware of what another poster here mentioned about proto 3.5 - so now have to read...
Learned that at Google, for each service method, introduce individual Request and Response messages, even if you can reuse. This way you can extend without breaking.
Also never reuse old/deleted field (number), and be very careful if you change the type (better not).
These were basic principles I put in place at a previous company that had a number of API-only customers. Request/Response messages. Point releases could only contain additive changes. Any changes to existing behaviors, removal of fields, or type changes required incrementing the API version, with support for current and previous major versions.
Protobuf was designed to keep backwards/forwards compatibility, you can easily add any fields to your messages without breaking old clients at the network layer (unless you intend to break them at the application layer);
Also, gRPC's human readability challenges are overblown. A naked protocol buffer datagram, divorced from all context, is difficult to interpret. But gRPC isn't just protocol buffers.
There's a simple, universal two-step process to render it more-or-less a non-issue. First, enable the introspection API on all servers. This is another spot where I find gRPC beats Swagger at its own game: any server can be made self-describing, for free, with one line of code. Second, use tools that understand gRPC's introspection API. For example, grpcurl is a command-line tool that automatically translates protobuf messages to/from JSON.
There's solutions like that for GraphQL [1] and REST too.
For REST OpenAPI/Swagger has a very large ecosystem, but does depend on the API author making one.
In C# one can have a decent documentation just based on a few attributes so you don't even have to write that much to have that schema available to your clients.
We are trying opeapi-generator and the experience is that the generated code for server stubs is either non existent, requires certain frameworks or is just not working.
So we had to write our own code generator templates. It was a pain compared to GRPC.
I can't speak to GraphQL, but, when I was doing a detailed comparison, I found that OpenAPI's code generation facilities weren't even in in the same league as gRPC's.
Buckle up, this is going to be a long comparison. Also, disclaimer, this was OpenAPI 2 I was looking at. I don't know what has changed in 3.
gRPC has its own dedicated specification language, and it's vastly superior to OpenAPI's JSON-based format. Being not JSON means you can include comments, which opens up the possibility of using .proto files as a one stop shop for completely documenting your protocols. And the gRPC code generators I played with even automatically incorporate these comments into the docstrings/javadoc/whatever of the generated client libraries, so people developing against gRPC APIs can even get the documentation in their editor's pop-up help.
Speaking of editor support, using its own language means that the IDEs I tried (vscode and intellij) offer much better editor assistance for .proto files than they do for OpenAPI specs. And it's just more concise and readable.
Finally, gRPC's proto files can import each other, which offers a great code reuse story. And it has a standard library for handling a lot of common patterns, which helps to eliminate a lot of uncertainty around things like, "How do we represent dates?"
Next is the code generation itself. gRPC spits out a library that you import, and it's a single library for both clients and servers. The server side stuff is typically an abstract class that you extend with your own implementation. I personally like that, since it helps keep a cleaner separation between "my code" and "generated code", and also makes life easier if you want to more than one service publishing some of the same APIs.
OpenAPI doesn't really have one way of doing it, because all of the code generators are community contributions (with varying levels of documentation), but the two most common ways to do it are to generate only client code, or to generate an entire server stub application whose implementation you fill in. Both are terrible, IMO. The first option means you need to manually ensure that the client and server remain 100% in sync, which eliminates one of the major potential benefits of using code generation in the first place. And the second makes API evolution more awkward, and renders the code generation all but useless for adding an API to an existing application.
gRPC's core team rules the code generation with an iron fist, which is both a pro and a con. On the upside, it means that things tend to behave very consistently, and all the official code generators meet a very high standard for maturity. On the downside, they tend to all be coded to the gRPC core team's standards, which are very enterprisey, and designed to try and be as consistent as possible across target languages. Meaning they tend to feel awkward and unidiomatic for every single target platform. For example, they place a high premium on minimizing breaking changes, which means that the Java edition, which has been around for a long time, continues to have a very Java 7 feel to it. That seems to rub most people (including me) the wrong way nowadays.
OpenAPI is much more, well, open. Some target platforms even have multiple code generators representing different people's vision for what the code should look like. Levels of completeness and documentation vary wildly. So, you're more likely to find a library that meets your own aesthetic standards, possibly at the cost of it being less-than-perfect from a technical perspective.
For my part, I came away with the impression that, at least if you're already using Envoy, anyway, gRPC + gRPC-web may be the least-fuss and most maintainable way to get a REST-y (no HATEOAS) API, too.
I don't entirely disagree that gRPC tooling is nicer and more complete in some areas, but there's some misconceptions here.
You can specify openapi v2/3 as YAML and get comments that way. However, the idea is that you add descriptions to the properties, models, etc. It's almost self-documenting in v3, and looks about the same in v2, although I've used v2 less so can't be sure.
I can't speak to editor support, but openapi has an online editor that has linting and error checking. Not sure if that's available as a plugin somewhere, but it's a likely a little more awkward if it is by virtue of being a YAML or JSON file rather that a bespoke file extension.
I've seen v3 share at least models across multiple files - it can definitely be done. String formats for dates and uuids are available, but likely not as rich as the protobuf ecosystem as you mention.
And I wholeheartedly agree that the lack of consistent implementation is a problem in openapi. I tried to use v3 for Rust recently and gave up due to it's many rough edges for my use case. It's a shame - the client generation would have been a nice feature to get for free.
I had forgotten about the YAML format; I probably skipped over it because I am not a fan of YAML. As far as the description features go, they're something, but the lack of ability to stick extra information just anywhere in the file for the JSON format severely hampers the story for high-level documentation. I'm not a fan of the "the whole is just the sum of the parts" approach to documentation; not every important thing to know can sensibly be attached to just one property or resource.
The flip side (IMHO, at least), is that simple build-chains are underrated.
As much as I love a well-designed IDL (I'm a Cap'n Proto user, myself), the first thing I reach for is ReST. It most cases, it's sufficient, and in all cases it keeps builds simple and dependencies few.
> The flip side (IMHO, at least), is that simple build-chains are underrated.
Wow, yes! Yes!
IDLs represent substantial complexity, and complexity always needs to be justified. Plain, "optimistically-schema'd" ;) REST, or even just JSON-over-HTTP, should be your default choice.
In my personal experience the other huge benefit is you now have a source of truth for documentation. Since your API definition is code you can leave comments in it and code review it all while having a language that is very approachable for anyone who is familiar with the C family of languages.
For a newcomer having `message Thing {}` and `service ThingChanger {}` is very approachable because it maps directly into the beginner's native programming language. If they started out with Python/C++/Java you can say "It's like a class that lives on another computer" and they instantly get it.
You don't get much more human readable than GraphQL syntax. There are now oodles of code generation tools available for GraphQL schemas which takes most of the heavy lifting out of the equation.
I use gqlgen for a Go backend and the combination of our schema design and the default codegen results in it opening one database connection for every row that ends up in the output. You can manually adjust it to not do that, but it doesn't seem like a good design to me.
(It also does each of these fetches in a separate goroutine, leading me to believe that it's really designed to be a proxy in front of a bunch of microservices, not an API server for a relational database. Even in that case, I'm not convinced it's an entirely perfect design -- for a large resultset you're probably going to pop the circuit breaker on that backend when you make 1000 requests to it in parallel all at the exact same instant.
Because our "microservice" was Postgres, we very quickly determined where to set our max database connection limit, because Postgres is particularly picky about not letting you open 1000 connections to it.
I had the pleasure of reading the generated code and noticing the goroutine-per-slice-element design when our code wrapped the entire request in a database transaction. Transactions aren't thread-safe, so multiple goroutines would be consuming the bytes out of the network buffer in parallel, and this resulted in very obvious breakages as the protocol failed to be decoded. Fun stuff! I'll point out that if you are a Go library, you should not assume the code you're calling is thread-safe... but they did the opposite. Random un-asked-for parallelism is why I will always prefer dumb RPCs to a query language on top of a query language. Sometimes you really want to be explicit rather than implicit, even if being explicit is kind of boring.)
Depends on the implementation. I've been using PostGraphile which says "PostGraphile compiles a query tree of any depth into a single SQL statement, resulting in extremely efficient execution".
I adore gRPC but figuring out how to use it from browser JavaScript is painful. The official grpc-web [1] client requires envoy on the server which I don't want. The improbable-eng grpc-web [2] implementation has a native Go proxy you can integrate into a server, but seems riddled with caveats and feels a bit immature overall.
Does grpc-web work well? Is there a way to skip the proxy layer and use protobufs directly if you use websockets?
On the GraphQL side you can use gqless[0] (or the improved fork I helped sponsor, here[1]). It's by far the best DX I've had for any data fetching library: fully typed calls, no strings at all, no duplication of code or weird importing, no compiler, and it resolves the entire tree and creates a single fetch call.
Pretty cool! Graphql-code-generator is great, but this goes a whole lot further. I no longer have to even write queries.
I guess the big advantage is that when you write a manual query you can still pull down more data than you need by accident. Whereas this approach only pulls down what you need.
Wait, thrift interoperates poorly with Java? We are stuck using it in go because another (Java-heavy) team exposes their data via it, and the experience has been awful even for a simple service. So what is it good in?
This was a long way back. Specifically, the Scrooge (or Finagle) generator for Scala and Java supported different versions of Thrift libraries. Maybe the problem was related to having a project with both Java and Scala and over the wire interop would have been fine--don't remember the exact details.
Not for me. I tried this with Javascript a couple years ago, and it was painful mapping to and from gRPC types everywhere. There's no "undefined" for example if you have a union type. You have to come up with a "EMPTY" value. On top of that, we had all kinds of weird networking issues that we just weren't ready to tackle the same way we could with good ol' HTTP.
"Generates client and server code in your programming language. This can save engineering time from writing service calling code"
It saves around 30% development time on features with lots of API calls. And it grows better since there is a strict contract.
Human readability is over-rated for API's.