
Airbnb Is Moving Faster at Scale with GraphQL and Apollo - saranshk
https://medium.com/airbnb-engineering/how-airbnb-is-moving-10x-faster-at-scale-with-graphql-and-apollo-aa4ec92d69e2
======
tinyvm
The state of the industry sometimes deeply sadden me.

Creating a stack that complex just to render Static Content... Seriously ?

I'm fairly confident that only the author of this project can do something
with the codebase.

It must be an absolute mess between Zeplin , Storybook , Apollo , GraphQL ,
Next, Yeoman etc...

Just why ?

Can't Airbnb invest in one good CMS and tooling solution ? Don't they have a
CTO that define the company tech governance and tech stack ?

Isn't building landing page for "Luxury Destination" one of their core
businesses ?

Do each Airbnb engineer create their own stack for a tiny part of the website
?

It just buggers me to see something like this and remind me of their 'React
Native Fiasco' where they decided to use React Native but their mobile
engineers didn't like JS , so the engineers of each platform just wrote the
app using binding to use Java or Objective-C.

Sometimes I really tell myself that working for a FAANGS must be awesome, but
then this type of content pops up and it just remind me I should either stay
at my current job or create my own business to avoid all this.

~~~
venantius
I used to work at Airbnb.

Airbnb does not, in fact, have a CTO who dictates the company tech governance
and/or stack. They prefer to run things in a federated manner, with individual
teams making the decisions that they feel are best for them. While they're
encouraged/required to draw up design docs and have them reviewed by an
architecture review group, the group's recommendations are non-binding.

This model has advantages and disadvantages. On the upside, it creates an
environment where people can take risks and do things that haven't been done
inside of the company before. On the other hand, it means people sometimes go
out on a limb and push the company into supporting something that turns out
not to be sustainable in the long term.

As a matter of personal preference, I like to have a set toolchain that a
company is built around. But it would be unwise to suggest that Airbnb's
strategy hasn't worked out pretty well for them overall.

~~~
stareatgoats
FWIW, I regularly use Airbnb for my travelling lodging needs. I usually find
what I'm looking for (an apartment with washer, WiFi and many good reviews in
the lower price-bracket).

But I always open the site with some apprehension, since I know I'm in for a
bad user experience. It's sluggish, and I can't bring myself to appreciate how
the layout and even menus differ (or even disappear) depending on which area
on the site I visit. It makes navigation cumbersome, and hard to remember how
to navigate between visits.

I would switch to a similar service in a jiffy if it had solved these
problems.

I always suspected a lack of a top-down coordination to be the reason for
these issues, thanks for confirming. I've come to believe that this kind of
loose federation strategy mostly suits junior devs (on which a startup might
be deeply dependent by all means), hardly a serious long haul business. I
expect they will change policy in due course, or perish.

~~~
tvanantwerp
I've had the exact same problems using AirBnb. I can never rely on things to
be in the same place one visit to the next, if they continue to exist at all.

I don't think AirBnb is alone in this. There are many sites and apps that
change UI's radically with disturbing regularity. I've begun to wonder if
there is a glut of UX/UI people in tech right now, and that this endless cycle
of zero-value-added change is just an attempt to justify their continued
employment.

~~~
untog
I actually think the problem isn't a glut of designers or UXers, it's an
obsession with A/B testing and multi-armed bandit scenarios. I'm sure that at
any one time there are dozens of A/B tests running on the site, and the
inconsistency we all see is a direct result. Netflix is a similar offender in
this area, I never have any idea where "continue watching" is going to be when
I open up the app.

------
bsbechtel
I recently did a POC for my company on a mature Angular project to see what it
would take to switch over to the Apollo stack. I ended up reducing LOC by 50%,
adding caching, full offline support, and optimistic UI. I’m convinced if we
had used the Apollo stack at the start of the project, our front end resource
requirements might’ve been 30-40% less for the project. The things the Apollo
team have done are really making a big impact on engineering teams that adopt
their tools.

~~~
hdra
I recently tried Apollo for a project as well, for the most part, its almost
magic, doing most of the usually tedious things for me. But I often run into
unexpected blocker on the most basic functionalities that I need.

One of them that I haven't been able to really solve till now is around cache
invalidation / removing deleted items from cache.

How have you been finding Apollo around that area?

~~~
sbacic
I'm currently working on a full-stack app centered around GraphQL and my
experience is pretty similar to yours - when it works, it's like magic. The
problem is, it doesn't always work the way you expect it to. Here are some of
the things I've noticed:

While the main technologies are pretty mature and well documented, gluing them
together is often a lot trickier and left entirely to the user. If you need a
way for Apollo to communicate with the database, you have to handle that
yourself. Same thing for authentication and access control. File uploads also
require a separate library.

I think the whole stack has a lot of promise, but it's going to have trouble
gaining steam as long as it forces the user to think about these sort of
things.

Something else I've noticed is that it shifts a lot of the work to the back-
end while at the same time introducing new problems you never had with REST.
With REST, you had endpoints with clearly defined inputs and outputs, which
were easy to reason about and secure. This is not the case with GraphQL -
because it is so expressive, it's hard to cover every possible use case,
especially if you're doing the entire security and access control yourself.
The other problem GraphQL has that REST does not is recursive queries - it's
entirely possible to request something like this:

authors -> posts -> comments -> authors -> posts -> comments ...

Another thing I should mention - while Prisma and Apollo are pretty stable and
well documented, various smaller libraries are often not. This is not an issue
specific to GraphQL, but because it has a smaller ecosystem then say, React,
you're a lot more likely to run into it.

Lastly - the pace of the development is bonkers. I've run into situations
where an 8 month old post was already obsolete. Or major versions a year and a
half apart.

~~~
bsbechtel
I lot of the issues you mentioned really are meant to address problems faced
by teams working on a project at scale. I really didn't understand the need
for GraphQL until I was put on a project where we had multiple REST endpoints
we needed to query data from, new people were joining every week, and
communication + data requests across different teams required a significant
portion of everyone's time.

The issues you mentioned are issues if you want one (or a few) framework or
library to handle all of your web development needs, but rarely does one
library work for web development at scale.

~~~
sbacic
It sounds like your issue was with poor documentation and not some fault
inherent to REST itself. I don't think that just switching to GraphQL is going
to fix that problem (though Playground is a pretty nice way of getting to know
the API).

I guess my main qualm was that, because GraphQL does a relatively poor job of
explaining _why_ you should use it, people like me get the wrong idea about
what kind of problems it solves and why you should use it instead of REST.

~~~
bsbechtel
It's not really an issue with documentation within our team. It's an issue of
having hundreds of REST endpoints (all which need to be learned by new devs),
multiple local dev servers running on your machine to mimic those endpoints,
and an excess of work done on the front end to merge data from across those
REST endpoints once it all arrives on the client. Additionally we would get
requests from other teams wanting to access data, so instead of them just
writing a query to the same GQL endpoint we all share, we have to write a new
endpoint for them, manage access control on that endpoint, etc. These are the
issues GQL solves.

------
EduardoBautista
As primarily an Ember.js user (with an interest in Elm), just looking at how
many hoops people in the React ecosystem go through _just to send a request_
just boggles the mind.

Maybe I am in the minority, but writing boilerplate code all the time just
isn't something that interests me.

~~~
aboutruby
I also much prefer simple REST APIs, just much easier to understand among
other things. (e.g. fetch("/api/users").then(...) and it's good to go)

~~~
exogen
It's certainly easier to start out that way. But there are also a lot of
pitfalls, that systems like what's described in the article are designed to
solve.

For example: let's say when component A or component B get rendered, they need
to fetch users in order to show something about them. You only want to make
the request if either of these components is actually rendered on the screen.
What if they both get rendered? Is something going to know to reuse the
inflight request so you don't have two identical requests? Or is it just
naively going to make unnecessary requests? What if some other thing already
fetched the user list earlier, are either of them still going to make the
request anyway? Or use the data that's already there?

Another example: you've got your result from the users endpoint. Now you fetch
some user information from the "blog author" endpoint. Information about the
same user is in both responses. What's the authoritative client-side source of
information about that user now? Are you now going to potentially be
displaying stale/conflicting info about that user in 2+ different places?

etc.

You can always just keep things simple if you want, and people in React and
GraphQL land are happy to do that, too. But a lot of people are also focused
on building complex applications. This is just to give you some perspective on
the "why" here.

~~~
shaunpersad
> Is something going to know to reuse the inflight request so you don't have
> two identical requests?

I really don't see what GraphQL has to do with this problem. What you're
describing are issues that pop up with any external datastore. GraphQL is just
a protocol. If there are clients that take care of that type of caching,
that's an implementation detail. You can also have REST clients that do the
same.

> Are you now going to potentially be displaying stale/conflicting info about
> that user in 2+ different places?

If you knew you needed info about the user as a blog author as well, you
could've included that in the users request. Isn't that what you would've done
in GraphQL anyway?

I'm not trying to detract from GraphQL's usefulness, but you can accomplish
the same things in REST pretty easily too, especially if you control both
server and client code.

IMO, GraphQL really shines when you're implementing clients for APIs that you
don't already control. In that case, the flexibility is great. But if you're
building both the APIs and the clients, REST works (and has worked) pretty
easily.

~~~
exogen
Indeed! I wasn't talking about REST vs. GraphQL at all, but rather the benefit
that fancy _clients_ get you (notice I said "systems like what's described in
the article", not GraphQL in general). I think that's what the grandparents
were talking about, since the boilerplate they were discussing presumably
comes from Apollo, not GraphQL. Apollo is a fancy client like you describe.

------
fro0116
I'd personally be interested in some details on how they're _generating_ the
mock data for their server.

As they're using that mock data for testing, the data returned has to be
deterministic, so I'm guessing they're either making up mock data manually by
hard-coding that data in resolvers on the mock server implementation, or using
some kind of fake-data generating library to generate that data dynamically
based on graphql type in the schema, with a fixed seed so that the data
doesn't change between runs.

I'm hoping to explore the latter approach a bit more as I feel it would be
great for testing productivity to have mock data generated from the schema
instead of having to manually write them for every new thing added.

~~~
adamrneary
It's actually neither. :)

We have a shared development environment with a persistent (and thus
deterministic) dataset. So when one developer runs a query against that
dataset, another developer could run the same query and get the same result.

The best thing about this, of course, is that if your Storybook data looks at
listing 112358, you can also open that same listing in development and see the
same result in the product. Very powerful.

~~~
fro0116
Thank you for the quick reply!

My question was more about how that persistent dataset in your shared
development environment is created though. As that dataset has to first exist
for people to query against it.

Curious if that creation process is manual or automated somehow through
inference on the types in the schema.

~~~
adamrneary
For sure. Unrelated to GraphQL in this case. Very manual (in a good way!).
Sometimes it's achieved manually via the UI. Sometimes manually via scripts,
etc. Imagine you're creating a new field for Business Travel, and you've added
some new fields via the API. One way or another you're grabbing 1-2 listings
in development, modifying those listings to reflect the desired change, and
then in the dev setup, you're saying "grab listings x and y and update their
data for use in Storybook and unit tests."

In my experience, if the system is working properly, there's not a lot of room
for type-driven inference. We often get a design with very explicit data
present, and we want to bring that data in rather than calling on Casual or

~~~
fro0116
Thanks! Yeah that approach makes a lot of sense for the use cases presented in
the post, and may be the most pragmatic path for enabling this workflow.

Although I feel the automated data generation approach still has value in that
it can introduce some variance in the dataset to better represent real-world
data, potentially uncovering issues in handling of edge cases in the
design/implementation of the UI, that the original conveniently customized
dataset that came with the design wouldn't. Though such an approach will
likely also need to offer the ability for users to override/customize the
generated data on a case by case basis in order to be useful for real world
applications, so we'll probably end up with a bit of a hybrid approach at the
end of the day regardless.

------
munchbunny
With posts like this, I think it's important to remember that projects like
this get created for one primary reason: it solves problems AirBnB has. If
you're lucky, it solves problems you have too.

I work on some tech giant cyber security stuff, and it's obvious if you look
at it that what my team designs solves a tech giant scale problem in the
context of past engineering decisions made to solve other tech giant scale
problems this company had. I probably wouldn't recommend my team's approaches
to anyone who has less than ~5000 engineers.

From this experience I've adopted a heuristic: first consider the scale the
system was built to address. If you are not in the ballpark of that scale,
give it a long hard look before you adopted it.

------
or113
When looking at the network tab, I couldn't find anything related to graphql.
Nor by searching overall the code. Does anyway know where exactly Airbnb use
graphql?

~~~
rayshan
Adam talked about server-side rendering in the video. You won't see the
graphql queries in network activities.

------
stockkid
It is unclear from the article how exactly Airbnb is moving "10x faster" as
the title claims. Unless backed by a benchmark, this kind of hyperbole
shouldn't be an engineering article.

------
gaius
Is moving faster to evade taxes and regulations really something to encourage?

