
Living APIs and the Case for GraphQL - craigkerstiens
https://brandur.org/graphql
======
bretthopper
I've been building GraphQL APIs @ Shopify for almost 2 years now so I'm a
little biased but I mostly agree with this post. Here's a few random thoughts
from my experience with GraphQL:

* GraphQL definitely seems to be more popular in the context of the client. A lot of the conversation is around client-side libraries like Apollo and Relay and how to consume APIs.

* Because of this, the actual GraphQL servers get less attention and there's less resources on building them.

* GraphQL on the client is a huge win in my opinion and I've never really heard anyone who's used it disagree with that.

* BUT, GraphQL on the server is absolutely much harder to write than a REST API. I've found two main reasons for this though:
    
    
      1. GraphQL is more powerful and you don't get all that for free.
    
      2. You end up paying more attention to designing a good GraphQL schema/API because of the type system.
    

Yes you can absolutely design great RESTful APIs with many of the same
features as GraphQL. Even at Shopify we offer a `fields` query param in REST
to only select a subset of fields. Guess what? Barely anyone uses this which
shouldn't be surprising.

JSON API and companies like Stripe with their `expand` feature offer
equivalents to GraphQL but it's non-standard and way harder to do since you're
inventing it each time. There's also no standard universal clients for these.

Greenspun's rule applies here in my opinion (slightly exaggerated):

> Any sufficiently complicated RESTful API contains an ad-hoc, informally-
> specified, bug-ridden, slow implementation of half of GraphQL.

I don't know where this will all end up. I do know that GraphQL isn't perfect
and it's still early. Public APIs are tough right now because developers are
so used to RESTful APIs, but at least at Shopify, we believe that will shift
over time.

~~~
adamkl

      1. GraphQL is more powerful and you don't get all that for free.
    

Personally, one of the things I think is most powerful, and missed in every
discussion about GraphQL, is that it’s more than just an alternative to a REST
API (understandably so, given that is it’s primary use case). At its heart,
it’s a query language specification with no mention of transport protocols. I
can’t speak to other implementations, but _graphql-js_ gives you the tools to
parse queries and schemes into abstract syntax trees, and once you have that,
there’s a lot you can do.

Take a look at Apollo’s work on merging schemas together for a good
example.[1]

Personally, I’m working on a proxy layer that will sit in front of a GraphQL
API and act as a sort of business rule engine. It analyses the incoming
queries to figure out which sets of rules need to be run, and then actually
alters the incoming query to add in any additional data points required before
forwarding the request to the backend API. Once the combined result comes
back, it executes the appropriate rules and filters the output before sending
back to the client.

These sorts of solutions don’t have anything to do with APIs. Heck, you could
create a GraphQL schema to act as a classic data access layer, and call it
internally from your own code.

    
    
      2. You end up paying more attention to designing a good GraphQL schema/API because of the type system.
    

Yes indeed. We have been working hand-in-hand with our business SMEs to build
out a schema that is essentially our business object model. It has made
communication between technology and business much smoother when we can point
to a visualization of our API [2] and have our business partners understand
exactly what they are seeing.

I’m not sure where GraphQL will end up either, but I can see a place for it
just about anywhere. (I suppose I’m a bit of a kool-aid drinker.. I wasn’t
when I first started using GraphQL, but it has grown on me)

[1] [https://www.apollographql.com/docs/graphql-tools/schema-
stit...](https://www.apollographql.com/docs/graphql-tools/schema-
stitching.html) [2] [https://apis.guru/graphql-
voyager/](https://apis.guru/graphql-voyager/)

------
languagehacker
GraphQL is good for some things, but often pretty dangerous for many of the
use cases it was originally introduced for. Without aggressive validation, it
does introduce significant blocking and overloading concerns for irresponsible
clients or bad actors. I like having a mix of a highly restricted GraphQL API
on the read end and a strictly enforced protocol buffer RPC. This enforces
good schemas, and encourages extremely slim requests over the wire where
possible. It's one very good solution in your toolbelt, and one that
integrates really well with a lot of the frontend frameworks out there today.
It's just worth being judicious about.

~~~
i_s
"Aggressive" validation seems to be a part of most libraries for helping
people implement GraphQL, so I don't think this is a real issue. I'm assuming
you mean stuff like max nodes, max depth, max page size, etc.

~~~
arkadiyt
It could also be referencing authentication and authorization controls on
fields - I think GraphQL makes it too easy for developers to expose private
fields without realizing it. Just last week I found a financial institution
exposing bank account numbers and routing numbers (not mine) through their
GraphQL endpoint - they still haven't fixed it.

~~~
RussianCow
This is just as easy to do with REST, though, and not something unique to
GraphQL. Most of the time, the problem has to do with specifying fields to
exclude rather than fields to expose, which makes it easy to add a private
field to a model and have it automatically, unintentionally exposed by the
API.

------
markjspivey
Interview with Christian Schwendtner “GraphQL is good, but it’s not an
alternative to ‘real’ REST services“

[https://jaxenter.com/graphql-good-no-alternative-rest-
servic...](https://jaxenter.com/graphql-good-no-alternative-rest-
services-142814.html)

~~~
derefr
I wouldn't say GraphQL is an alternative to REST.

GraphQL is (a DSL syntax for) a restricted subset of SQL—one that's restricted
_enough_ that it's okay to allow arbitrary clients from the public Internet to
request of you.

Like SQL, though, its proper place is "inside" the API abstraction-layer, not
"outside" it. GraphQL queries, like SQL queries, are things your backend app-
layer logic can generate and send to your DB layer (presuming your DB layer
supports GraphQL.)

There are certain APIs where you might want to expose the ability to use
GraphQL to modify the composition of the tree of data-structures returned by a
particular controller-route. But this doesn't mean you should be using GraphQL
"as" your API. GraphQL here is just a parameterization of your API, like
result-set pagination. The API itself still needs to exist regardless of
GraphQL, and needs to speak something that allows for all the standard CRUD
operations.

~~~
headcanon
Respectfully, this is false. First, GraphQL is not a subset of SQL, nor is it
in any way related to SQL. Second, GraphQL is designed for safe client-server
interaction. GraphQL is not a query language in a traditional sense, it is
simply a syntax for a consumer to describe how the data it receives should be
structured. This does not preclude server-side usage as well, however.

While a real-life application may want to expose a REST interface, there is no
inherent reason why a service cannot expose a GraphQL-only API, and each
client only consume that API.

~~~
derefr
> First, GraphQL is not a subset of SQL, nor is it in any way related to SQL.

I didn't mean to imply the language syntaxes are related. But they're both
syntaxes _for relational algebra_ , and GraphQL has a strict subset of SQL's
_capabilities_ in terms of what relational-algebra queries/mutations it can
express. (I.e. GraphQL can express joins, but only inner joins with an
aggregate projection on the left side; and GraphQL can express projections,
but only trivial projections applying the identity function to a subset of the
available tuple fields.)

(Note that other, more powerful graph manipulation languages, e.g. Datalog or
SPARQL, are _also_ syntaxes for relational algebra. They're similarly limited
in some ways compared to SQL—and yet the types of queries they express easily
would balloon out to hundreds of lines of SQL.)

> there is no inherent reason why a service cannot expose a GraphQL-only API,
> and each client only consume that API.

There is no inherent reason why a service cannot expose an SQL API, either,
provided you lock down the restrictions on the role the DB is acting as (both
in terms of ACLs on DML query types, and in terms of caps on compute time,
memory, etc.)

But people don't _do_ that, and they avoid it for more than just the
challenging ops story it implies. You don't expose SQL-based APIs to the
public internet, because you tend to build APIs to service the needs of
particular clients with particular use-cases, and those use-cases can be
_optimized for_ by adding things like caching middleware, but only if you can
filter out and tag the particular use-case each request is for. REST lets you
do that; the use-cases are directly named by the (method + origin + path), and
the semantics are already there, built into the protocol, for caching them,
substituting them, access-controlling them, etc.

GraphQL, like SQL, doesn't have any place for naming the use-case of a
request, in the request. So, if you want any of the useful properties like
cacheability or access-control or whatever else to apply, you probably want
your GraphQL request to appear "inside" a RESTful request, such that you still
have individual REST endpoints for particular use-cases.

~~~
joshribakoff
You wrote..

> GraphQL, like SQL, doesn't have any place for naming the use-case of a
> request

This is not true, see the docs:

> The operation name is a meaningful and explicit name for your operation. It
> can be very useful for debugging and server-side logging reasons

[https://graphql.org/learn/queries/](https://graphql.org/learn/queries/)

You wrote...

> There is no inherent reason why a service cannot expose an SQL API

GraphQL let's you query across backends, such as relational (SQL), graph
databases, REST APIs, it is backend agnostic. You're comparing apples to
oranges, SQL & GraphQL are different abstractions.

~~~
derefr
SQL is backend agnostic. And I'm constantly surprised that people aren't aware
of this, especially the people who think that a DB needs to be "NoSQL" just
because it necessarily can't support all the _features_ that SQL offers. No
RDBMS supports 100% of the features in the SQL standard, either! It's okay to
"support SQL" in a way where you just throw errors if 90% of SQL's syntax is
used! Many systems do that! Clients are Required by the SQL standard to cope
with that!

Every modern DB labelled "NewSQL" is fundamentally just a "NoSQL" database
that has been made to _speak SQL_. Sometimes it is, in fact, literally a
previously-built NoSQL database, with an SQL adapter layer like
[https://en.wikipedia.org/wiki/Presto_(SQL_query_engine)](https://en.wikipedia.org/wiki/Presto_\(SQL_query_engine\))
used as middleware.

And your application can speak SQL, too! It's not _that_ hard to:

1\. grab an SQL2016.y grammar file and generate a parser for it in your
language-of-choice;

2\. expose a port on your daemon that speaks a protocol wire-compatible with
some particular RDBMS's wire protocol, e.g. the
[https://www.postgresql.org/docs/current/static/protocol.html](https://www.postgresql.org/docs/current/static/protocol.html)
(like CockroachDB has done.)

If you do these two things, then other programs can connect to your app _as a
database_ , and use their ORM tools to handle your data model. (Consider this
option if you're ever developing e.g. an MMORPG server; it can save hundreds
of hours of work in exposing new APIs and deprecating old ones, for the cost
of a little front-loaded work.)

\---

But I digress.

You can make any application layer, or database layer, speak GraphQL. Because
it's _a syntax for relational algebra_ , and relational algebra is just "How
You Describe Relationships When Making Requests." It has nothing to do with
the fundamental underlying schema of the data; it's a language for
communicating your _intent_ —the things you want to get, or the things you
want to change—by naming them through their relationships to other things.

Now replace "GraphQL" with "SQL" in the above and notice that nothing changes.

~~~
joshribakoff
1 - I don't have to fork my database to use graphQL... it was designed to be
an API gateway.

2 - forking databases to speak the same language still doesn't solve the issue
of how to coalesce data from multiple backends into one response. You need an
API gateway for that.

3 - forking a db would take months or years for me. I added graphQL to my
stack in a few hours without modifying my backends, again because it's
designed as an API gateway.

Yes they both declare data requirements. But no they aren't both relational
and only one of these two technologies was designed as an API gateway.

------
emilsedgh
I maintain a relatively big REST API. There are hundreds of different entities
with very deep relationships.

Our solution is exactly like Stripe's. Clients ask for the relationships they
want to fetch using the query string.

It's not ideal. Seems like a hack. But it works.

On the bright side, I can query the system using curl.

What should be my incentive to consider switching to GraphQL for this project?

What should be my incentive to consider GraphQL for my next project?

~~~
tango12
I think the big (only?) advantage that GraphQL has in this case is you get
"API documentation" out of the box as a frontend developer.

The tradeoff is that I need to use GraphiQL or similar tooling to leverage
this auto-complete, type-safe magic and that using curl becomes more painful.

I'm curious: Suppose you converted your REST endpoints into GraphQL endpoints
in the simplest way possible by changing the query-params to root GraphQL
nodes. So there's no "graph" really. Just lots of top level nodes. This
reduces the API docs overhead and nothing else.

Would you switch? Is your existing tooling to extract API docs already good
enough that this isn't a problem?

~~~
emilsedgh
[https://docs.rechat.com](https://docs.rechat.com)

This is our API Documentation and it's generated automatically from our E2E
tests. It's almost free for us to maintain.

Can you tell me if this is OK enough comparing to a GraphQL that wouldn't
justify a rewrite?

------
faitswulff
One critique I haven't seen about GraphQL is that it's less human-readable. I
know that "GraphQL is unapologetically driven by the requirements of views and
the front‐end engineers that write them"[0], but if everything switched to
GraphQL endpoints, I'd miss having URLs you could change by hand :/

[0]: [https://facebook.github.io/graphql/October2016/#sec-
Overview](https://facebook.github.io/graphql/October2016/#sec-Overview)

~~~
masklinn
> I'd miss having URLs you could change by hand :/

URLs you can change by hand in _API calls_? When I'm at that level, I can just
as easily edit the call body.

> One critique I haven't seen about GraphQL is that it's less human-readable.

My experience is the opposite, reading a complete graphql query is way easier
than reading a mix of URL, query parameters and entity body, even more so when
I need multiple such queries to replace a single graphql query.

And the query tells me exactly what's in the result, to boot, so it acts as
documentation for the data-extraction code.

~~~
faitswulff
> URLs you can change by hand _in API calls_?

No, that's not what I meant. I meant that RESTful APIs relate the URL to the
data in a way that isn't true in GraphQL APIs.

Take reddit, for instance. I can clearly see a username in the url:
[https://www.reddit.com/user/masklinn](https://www.reddit.com/user/masklinn)

Swapping one out behaves as I'd expect:
[https://www.reddit.com/user/faitswulff](https://www.reddit.com/user/faitswulff)

And tacking on a format type to the end exposes the content in the manner I'd
expect:
[https://www.reddit.com/user/masklinn.json](https://www.reddit.com/user/masklinn.json)

The conflation of route and content is frustrating for developing single page
apps, but it maps better to my understanding of disk files and folders, thus
making the URL more human readable and easier to toggle and change by hand.

If everything was an SPA, you'd _have_ to interact with the data through the
frontend, which speaks to some singular GraphQL endpoint, hence my mentioning
that GraphQL itself mentions that it is "unapologetic" about its front-end
orientation.

~~~
jazoom
Did you think about masklinn's comment for 5 minutes before replying?

~~~
faitswulff
I came back to edit the snark out.

~~~
jazoom
That's why pencils have erasers. :-)

------
andrewingram
GraphQL has seen a lot of adoption, but the majority will be private APIs
rather than public APIs like Github's.

There's no rules about how you have to use it, but the problem it was built to
solve was communication between first-party applications and backends.

------
ericcholis
I think GraphQL found it's place between backend and frontend as a data layer.
Airbnb just wrote an article detailing this very use-case [0]. Perhaps it
isn't suitable for public APIs.

[0]: [https://medium.com/airbnb-engineering/reconciling-graphql-
an...](https://medium.com/airbnb-engineering/reconciling-graphql-and-thrift-
at-airbnb-a97e8d290712)

------
dopeboy
Good "state of the state" esque writeup. Thanks, Brandur.

I've spent the past three years using Django Rest Framework. I switched over
to using a GraphQL (Graphene as the provider, Apollo as the client) and it's
been fantastic thus far. Easy to pick up, great documentation. Will learn more
as we grow.

~~~
_hardwaregeek
I'm not sure I agree with "great documentation". It's a pet peeve of mine, but
I don't like it when the tutorial for a server side program uses ES6 modules
and decorators. This means I need to set up a build system for my back end.
Which is just annoying. I can forgive ES6 modules, since they're experimental
in Node 10, but decorators are still on stage 2. Don't write your tutorials
with experimental features.

------
lima
One very interesting project is rejoiner:
[https://github.com/google/rejoiner](https://github.com/google/rejoiner)

It translates GraphQL calls to gRPC queries.

~~~
cmjqol
Has no activity since March , doubt it's still used.

~~~
lima
It's a proof of concept. What are you expecting?

------
dmitriid
> You should find yourself being able to build a query that delves 4+
> relations deep without much trouble.

How is that a good thing from the _server_ ’s point of view?

As mentioned elsewhere, almost no one talks avout the _server_ shen talking
about GraphQL.

When you can construct an ad-hoc query 4+ relations deep on the client, how
can anyone efficiently handle this on the server? You can’t even properly
cache GraphQL requests.

~~~
geogeim
This is the problem I had with graphql. Indeed it simplifies the interface
between the backend and the frontend of your application (that is when the api
can be cleanly mapped to a graph) but it doesn't really solve the problem of
minimizing roundtrips, it just moves it between the backend and the database.
Because the database I was querying was not on the same network I was losing
seconds (!) doing roundtrips on the network gathering all the data necessary
for a single complex graphql query. You may say: well use something like
postgraphile but what if you did not designed the database and so it doesn't
map cleanly to your graphql object tree? I had to waste a week implementing a
postgres query generator so I can remove the latency. Not fun, especially when
you have deadlines.

------
__s
Went with graphql where I'm at, but ended up mostly embedding another query
language on a method that most querying happens through. The DB is an entity
component system, so Entity table has an id column & the rest of the tables
are keyed off that, makes it that any foreign key can reference any entity. So
the query language is a JSON blob of { id: "asdf", allOf: { ComponentName: {
field: "hasvalue" } }, optional: { OtherComponent: { reffield: { optional: {
AnotherComponent: {} } } } } } where it's going off of viewing entities as a
bag of components

Initially we were just exposing a big Entity type in GraphQL which had a field
for each component. Didn't feel like there was enough control over caching &
prefetching. Current system is kind of nice because one reads/writes to redux,
calls commit on ids to submit to server, querying just populates the redux
store

------
pm90
That blog is just beautiful. I really like this simple style which still looks
really well polished.

~~~
dvdhnt
I definitely agree. Here's the source if you're interested:
[https://github.com/brandur/sorg](https://github.com/brandur/sorg)

------
geekjock
Great write-up. One aspect of GraphQL that I've found difficult is
communicating errors to consumers in a developer-friendly way. What is the
best way for GraphQL server to communicate to the client the equivalent of
HTTP status codes 500, 404, etc?

The approach I recently implemented was to add a "type" or "code" property
with standardized values to error objects in the GraphQL response (similar to
error codes in Stripes REST API [0]). The client can then check those "errors"
objects for specific codes and behave accordingly.

[0] [https://stripe.com/docs/error-codes](https://stripe.com/docs/error-codes)

~~~
joshribakoff
This is in the docs.

[https://graphql.org/learn/serving-over-
http/](https://graphql.org/learn/serving-over-http/)

> [a] query might result in some data and some errors, and those should be
> returned in a JSON object of the form {data: {}, errors: [] }

Also checkout [https://github.com/kadirahq/graphql-
errors](https://github.com/kadirahq/graphql-errors) which exposes a standard
`path` field for each error, so the client has a standard way to know which
part of the query resulted in an error.

If a mutation fails, for example a bank withdrawal fails because of
insufficient funds, that is not necessarily an "error". To quote "code
complete":

> Exceptions should be reserved for what's truly exceptional.

I'd send a mutation like this:

    
    
      withdrawFunds($amount: 100, $currency: "USD") {
        didSucceed
        didOverdraft
        canRetry
      }
    

The client can control exactly which field(s) to request, or simply fire &
forget if it does not care if the transaction went through. If you change your
mind later, you can add & deprecate fields, in the true spirit of graphQL.

In the case you shove arbitrary error codes in strings, you lose the benefits
graphQL brings, like tracking usage of a field, self documenting &
introspection, solving for over/under fetching, and the ability to revise
these codes without versioning your mutations.

------
ORioN63
Those were actually very good points, specially the interaction between client
/ server being better understood due to the queries being made per purpose and
not just trying to obey one of many REST conventions that the server provides.

------
wnevets
graphql feels like I'm going back to the days of SOAP/RPC calls, maybe I'm
just getting old.

------
11235813213455
GraphQL is so vastly superior for validation, filtering, nested queries,
relations, documentation, discoverability...

You can even have a mutation save a nested object (fragmenting it and passing
it down to other handlers), which is so great.

You can make a GraphQL endpoint quite fast with
[https://github.com/graphql/graphql-js](https://github.com/graphql/graphql-js)

------
sbr464
I’ve been doing work with GraphQL for about 1.5 years now. It took a while to
wrap my head around but now it’s amazing to work with. I’ve been working with
dynamic schema generation, multi-tenant, completely dynamic gql servers,
building dynamic gql queries on the client. It’s enabled some pretty amazing
functionality that would be more difficult to reproduce with RESTful
technologies.

~~~
jlu
What you described seems very interesting, is there some place to see your
work or any writeups to read?

------
gandutraveler
Most of the backend engineers I worked with don't really get or see advantages
on why GraphQl is good. Also it's not easy to build GraphQL API on server side
compared to REST. There are tons of evolved libraries for REST. I personally
think that transition from REST will be incremental and via JSON route.

~~~
pjmlp
As someone doing full stack development, I also don't see any benift,
specially now that REST tooling is finally catching up with SOAP and I don't
do hype driven development.

------
pm90
What are the benefits of using GraphQL visavis grpc?

~~~
masklinn
grpc is a bog-standard "shallow" RPC: client calls an endpoint, client gets a
result, if client needs sub-results it calls more endpoints.

GraphQL queries have depth, when you call a toplevel endpoint which returns a
non-trivial type you can select fields of the sub-resources to fetch, in fact
you can also select just a slice of the top-level resource as well.

Let's say HN had a grpc API and you wanted the top comment of each article of
the main page, unless it provided a very specific endpoint doing that you'd
probably have to list the front page (and get a whole host of crap you don't
care for e.g. the title, the destination URL, the score, the author, …), and
for each article get its comments (again with all sorts of metadata you may
not care for). In GraphQL, you'd select all but exactly what you need at once.

~~~
hnaccy
How does the backend ensure complex queries are efficient and that the client
can't make a costly query without realizing it?

~~~
masklinn
graphql does not provide for more complex queries than a regular RPC, the API
designers decides specifically what they want to provide at each level.

What they do is coalesce all those separate queries into one, and actually
allow for sub-query optimisations if you feel like it.

So the answer is that it does not, any more than e.g. grpc does not.

------
tonetheman
No one in their right mind would ever choose to implement GraphQL rather than
a REST api. The complexity difference is just too great. Maybe have your
primary API as REST then as some kind of janky caching thing use GraphQL.

The only experience I have had calling GraphQL came from github and it was not
great. The API was responsive but when I was trying to figure out syntax I was
longing for a much clearer REST end point that would give me the data I was
after.

~~~
zachrip
I use it everyday and love it. I write both sides of it and it's paradise
compared to working with rest. I'd pick my stack again any day: typescript,
react, react-apollo, graphql-tools, apollo-express. I'm able to move so damn
fast, it's a breath of fresh air.

~~~
dopeboy
Curious about your stack - what's your backend in and what GraphQL provider do
you use?

~~~
zachrip
The stack is already listed in the first comment but it's Apollo on the front
and backend. Typescript on both sides as well. Interfaces a MySQL DB and
various grpc services.

~~~
dopeboy
Ah ok, I thought Apollo was only a frontend client. My mistake.

