
When REST isn't Good Enough - williamdix
https://www.braintreepayments.com/braintrust/when-rest-isnt-good-enough
======
mikeknoop
We (Zapier) are in a particularly good place to comment on this. We have
implemented about 70 very diverse web APIs in-house. Early on, we used client
libraries because they were easy and convenient. That decision bit us, hard.
(Most) client libraries are un-maintained, opaque, and difficult to extend.

We spent many months ripping every client library out down to the raw
requests. We are in a unique position because of how many APIs we have to
support, but the overarching advise we give to people designing APIs:

If you are deviating from the norm, you're doing it wrong.

REST APIs are nice because you can infer how to communicate with the API out
of the gate. APIs are a pain and you only introduce headaches by not doing
things in a sane, conventional fashion.

~~~
edanm
Is there a good book/resource on learning REST API ideas? I'm looking for a
beginner's resource for someone who doesn't really grok the ideas behind REST
and the motivation for REST. But also something that gets technical very
quickly - I've been programming for long enough that I pick ideas up very
quickly.

~~~
sopooneo
I'm sort of with you, though I've read quite a bit at this point and
interacted with a number of APIs of varying degrees of "RESTfulness".

The biggest question I wish the various tutorials would address is: why is
REST a good idea? Why is this particular way of doing things better than
others?

And here's one thing I believe is true, that I have literally never seen in a
REST tutorial: sometimes REST is _not_ the best way to go. Sometimes an RPC
architecture is better.

~~~
BerislavLopac
"sometimes REST is not the best way to go. Sometimes an RPC architecture is
better."

I think that the full versions of there acronyms make a pretty good job of
explaining what is best when.

"REpresentational State Transfer": obviously, it transfers _state_ , i.e.
information about a certain resource at a given moment. "Remote Procedure
Call": obviously, it calls a _procedure_ , which may involve several state
changes and other activities. Of course, a procedure may be masked behind a
REST endpoint (e.g. when you POST some data and do some procedure before you
end up with a certain state) and vice versa (e.g. a simple getter), but you
may pretty much view REST as the SQL of the Web, and RPC as stored procedures.

~~~
SilasX
>but you may pretty much view REST as the SQL of the Web, and RPC as stored
procedures.

Indeed, but that just raises the question: what would possess you to write an
app out of SQL calls rather than general functions? So then why do the Web
equivalent thereof?

~~~
BerislavLopac
You don't, of course. What you do is build an app which is using REST to store
data somewhere online, and the app itself may be in the user's browser, on a
mobile phone or anywhere else for that matter.

~~~
SilasX
Definitely. Just pass that memo along to the "apps should work purely through
REST API calls" crowd.

------
bguthrie
Note that this isn't a critique of the REST architectural style; rather, it
documents their decision to prefer supplying clients with prebuilt libraries
rather than a public API that their respective language communities can use to
build their own. You could read the source to any one of their client libs and
infer the rough structure of their services, so it's not all that private;
they're simply choosing to leave what's there undocumented. It's an
interesting approach, but not without its drawbacks.

~~~
MatthewPhillips
I have no knowledge of what BrainTree does, but if I'm picking a service with
an API, I'm picking the one that is easiest to learn (preferably there is
little that needs to be learned), I'm not picking one that requires me to
switch languages -- or just as bad, locks me into one of a few languages I'm
currently using.

~~~
elarkin
BrainTree does payments. They have concerns about security that most apps do
not. Who really cares if bugged timeout/retry logic makes your app post twice
to facebook? Probably no one. Who cares if bugged timeout/retry logic makes
your app bill them twice? Probably everyone.

BrainTree and you may not make the same decision, and you both could be right.

------
KrisJordan
Braintree isn't the only HTTPS API a developer will consume. Rather than bury
the knowledge of HTTPS best practice in the black box of a library, why not
exemplify it in language-specific examples of plain-old REST API
documentation?

If the boilerplate for manually getting an SSL connection right in a given
language is _that_ obtuse, the "just use our library" pitch is even more
compelling.

All this said, I have used Braintree's libraries and they are extremely well
executed.

Not exposing REST API documentation externally (it surely exists _internally_
, right?) just feels like a cop out.

(An argument I'm surprised was left out: it's easier for support staff to work
with customers integrating with a good library than some home rolled REST
client. I can imagine this to be true, but maybe a case of premature
optimization if the majority of big API players expose and document their REST
APIs anyway?)

------
pixie_
I don't like REST. Making a call to a server should be like calling a function
anywhere else - you have your parameters and return value. In REST the
parameters are spread over 3 places, the action, the url and the post
parameters themselves. It's a much better design to combine all your
parameters in one place and keep things simple, for example -

REST version: UPDATE /course/324234 { description: "This is a level 1 course"
}

Improved version: POST /course/UpdateDescription { courseId: 324234,
description: "This is a level 2 course" }

In this case POST is always used for api calls. Course is the namespace,
UpdateDesciption is the method name, and parameters are kept all together as
JSON.

~~~
enjo
The REST version makes it clear that you are dealing with operations on a
resource. The URL identifies a _thing_ in a consistent way, while the method
and corresponding data allow you to manipulate it.

The biggest benefit comes when building a cache architecture. RESTful API's
(when properly implemented) come with built-in assumptions about idempotency.
You end up in a place where you can easily cache GET requests while reasoning
in a very consistent way about where changes to a resource will be made.

It's a pattern that comes with a lot of really useful benefits.

Now all of those same qualities can be built into other architectural styles
(such as the one you propose). However, I find that it becomes much harder to
reason about the role of operations and what they're actually doing when you
lose that clear distinction that REST enforces.

I do want to quibble with one thing as well:

 _In REST the parameters are spread over 3 places, the action, the url and the
post parameters themselves_

That's true for any modular system isn't it? You have the _object_ you are
operating on, you have the _operation_ , and you have the _data_. Object-
oriented systems and even most structured languages (like Javascript for
instance) make this distinction at some level. Why is that not appropriate for
web architecture?

~~~
SilasX
>It's a pattern that comes with a lot of really useful benefits.

Sure, but people can implement that pattern without buying into the whole
clumsy REST edifice.

For example, a strict reading of REST (and RESTers support such readings!)
would tell me that if I have an API that takes two cities and returns the
distance between them, then I must expose a URI pointing to _every combination
of cities_.

See here: [http://roy.gbiv.com/untangled/2008/rest-apis-must-be-
hyperte...](http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-
driven)

>>A REST API should be entered with _no prior knowledge_ beyond the initial
URI (bookmark) ... From that point on, all application state transitions must
be driven by client selection of _server-provided choices that are present in
the received representations_ or implied by the user’s manipulation of those
representations. ... [Failure here implies that out-of-band information is
driving interaction instead of hypertext.] [emphasis mine]

So, 1000 cities, you must send a million links over the network rather than
1000 possible parameters plus the formatting.

Now, some RESTers assure me that, no, you can somehow communicate to the user
that [prefix] / [first city] / [second city] will get you that answer; that a
server can "instruct clients on how to construct appropriate URIs".

Fine, but then we're right back to custom, author-documented APIs and RPCs:
"to do that, format the call this way". Right back where we were before trying
to force-fit everything into a CRUD mold with increasingly bizarre tables just
to make all the calls work.

Or maybe a REST purist would tell me that I'm supposed to link a URI for the
starting city: [root]/distances/[start city]/, and then from there link them
to possible choices of the second city: [root]/distances/[start city]/[end
city]/ .

Fine, but why jump through two pointless hoops, when I know what I want, and I
prefer to just send _one_ request rather than pointlessly spend bandwidth
navigating a path I don't care for?

~~~
bct
Nobody ever said that you should use REST if it doesn't suit the problem
you're trying to solve.

~~~
SilasX
But there are people that believe REST suits every API problem, even when
accumulated wisdom says it doesn't suit the problem ( _cough_ Roy Fielding).

~~~
bct
No, he makes it quite clear that it's a solution designed for a particular
problem space. You're debating a straw man.

~~~
SilasX
Indeed: the problem space being APIs. Feel free to point me to an example of
fielding pointing to an API problem that shouldn't be RESTful.

~~~
bct
No, the problem space being "scalability of component interactions, generality
of interfaces, independent deployment of components, and intermediary
components".

This merely happens to overlap significantly with the requirements of public
APIs.

I have difficulty imagining your distance calculator needing any of those
things, though.

~~~
SilasX
>No, the problem space being "scalability of component interactions, ...

>I have difficulty imagining your distance calculator needing any of those
things, though.

Right, because no one's stupid enough to use (or stick to the use of) REST for
a distance calculator or any other algorithmically generated information. But
make no mistake, scalability of component interaction _is_ an issue, just a
solved one (for those that see REST for what it is).

The solution is: specify an input scheme (for a hand calculator: put the first
number, then plus, then the second number, then equals) and let the user
choose the inputs. This saves you from the (intercomponent-unscalable)
combinatorial explosion in which you have to give the user a link to every
possible computation as they navigate the interface, and which is the REST
method.

So, any exposed function in which you can't feasibly blast every possible
input set over the network is REST-incompatible, so I guess the _serious_
RESTers don't think you should do it. Which kinda makes it little more than a
footnote.

~~~
jon_moore
I'm pretty sure Google's home page doesn't include every possible search term
for you to select from, and that's a perfectly RESTful example of an exposed
function (search). Forms are a very powerful hypermedia construct.

~~~
SilasX
In the context of a website, maybe you can have a helper like that to avoid
the REST bloat. But the architecture is for arbitrary APIs, existing outside
of web pages, in which I don't have a neatly visible form. All I'm allowed to
do is give the user URIs to choose from.

Some kinds of apps (esp those that can't tolerate the overhead of REST, like
for mobile) need to know how to format a Google search request without
navigating through a session on Google site, but just knowing what it should
look like, and formatting it that way. REST would restrict you to pointing
them to google.com and following links; it prohibits you from saying, "hey,
you can have your app just point to google.com, then '?q=', then your search
terms connected by +'s".

------
scanr
Braintree have made an interesting choice in keeping their REST API private
and requiring customers use their client libraries. I think they could both
have client libraries that they promote as well as a public REST API. Breaking
down their reasons:

Security - agree with them on this, the more they can help their users make
their systems secure the better. Not sure if it should preclude a public REST
API but certainly motivates for having a good client library.

Platform Support - another good reason to have the client library, essentially
encoding best practice in the client. I've certainly seen customers abuse
features of our APIs. Again, not sure if it should mean keeping the REST API
private. Certainly it's a good idea to be defensive on both the client and
server (e.g. for queries that request too much data, rate limiting etc.).

Backwards compatibility - Here I disagree with Braintree. I think it should be
just as easy to manage backwards compatibility purely on the server side.

~~~
dan_manges
I'm a developer at Braintree. The challenge with maintaining backwards
compatibility on the server side is handling the wide variety of possible
inputs into the system. It's hard to write tests that account for all of them.

I've seen a couple of cases where backwards compatibility can be broken in
unexpected ways. For one application that we built at Braintree, we had a
client that was sending us an application/x-www-form-urlencoded POST body
without the Content-Type request header. We upgraded the version of Rails that
this app was using, and it broke that integration because Rails made a change
where it wouldn't parse the POST body without the Content-Type header.
Unfortunately, we didn't have any test cases in our test suite that made POSTs
without a Content-Type. We were able to identify the issue and resolve it
quickly, but it was a surprising bug. With client libraries, we can test every
version against the upgraded app and know that all clients will continue to
work.

Are there interesting request profiling techniques that can be executed on
production traffic to analyze requests? I think the challenging part of
backwards compatibility is making sure unintentional use cases, that were
never intended to be supported, continue to work.

~~~
scanr
Thanks for responding! And I'll update my response to the backwards
compatibility to say that I see the value in having a closed set of clients
that you need to test for backwards compatibility.

That said, these are some of the things that I've done to help with backwards
compatibility through only the serverside API:

* Track production requests and use them as test cases

* Build a large suite of test cases against the API

* Build the API in a statically typed language (yeah, I know, contentious, I love Rails but there's something about an externally facing API that makes me want to use a statically typed language).

* And then the ultimate -- build the API as an app that talks back to the business logic... essentially, it's your 'client library' but deployed on the server between the actual code and the client. Then, never change the part of it that faces the client, only the mappings on the the real business logic.

------
davetron5000
This has been my approach to internal service design as well. It's far easier
to make sure people integrate properly by providing a rich, well-documented
API client library than it is to rely solely on the REST-based one. REST APIs
are hard to document, and require a lot of orthogonal boilerplate to even make
the requests properly.

Making a client library is also a great way to "dogfood" your REST API - you
know a developer will write code, not HTTP calls, to interact with your
service. The client API allows you to test out that code and make sure your
API is well designed.

~~~
arethuza
"require a lot of orthogonal boilerplate to even make the requests properly"

What kind of boilerplate would that be? For me one of the big advantages of a
RESTful interface is that they are really easy to call from cURL _without_
much need for building up a complex context.

~~~
e28eta
I found the translation between native code and a REST API to be what I'd call
boilerplate. You need to build a URL, build the request (usually converting
some model object into JSON), make the request in a non-blocking fashion,
receive and parse the response into either a successful response, probably
converting JSON back into a model object, or a failed response, which needs to
be mapped to some native representation and passed back into the business
logic.

Contrast that with a client library which can do all of that for you, for
example: taking a native model object and calling one of two callback
functions provided, letting you concentrate on the business logic of your app.

I'm not arguing that a client library is the right way to go, but I certainly
understand where the grandparent is coming from RE: boilerplate code. Also,
this is going to be heavily influenced by the language and libraries you're
using.

~~~
MatthewPhillips
Except company B uses a single callback, returning an object that contains an
error code when not successful. Company C uses a blocking library, and
requires you to pass in your model object as a dictionary. See where I'm going
with this?

------
jeremyjh
Why not provide a C api? This would make it easy for community language
bindings to capture the benefits you describe in languages you aren't natively
supporting (such as Go, Erlang or Haskell).

------
tlrobinson
It's a shame there isn't some simple standard (de-facto or otherwise) way of
documenting REST APIs such that an idiomatic client library for each language
could be generated automatically.

The OPTIONS method is a really underused part of HTTP and would be great for
this purpose: <http://zacstewart.com/2012/04/14/http-options-method.html>

~~~
olalonde
Yes, there should be a WSDL for REST APIs.

~~~
cryowaffle
have you heard of the WADL?
[http://en.wikipedia.org/wiki/Web_Application_Description_Lan...](http://en.wikipedia.org/wiki/Web_Application_Description_Language)

------
pelle
There are several good points in the article. Particular the one about https.

That said as a clojure dev I'd much rather work with a nice clojure http
library like clj-http than have to use their provided java library.

------
outdooricon
The platform support argument is not very persuasive here. The beauty of open
source and the community is that when you combine a popular service with
common platforms that are likely to be used, it is likely someone in the
community will release the needed api. Why not provide some community support
for the platforms, but allow the users to release an api where they know their
needs better than you do?

On the other hand, the security argument is quite strong. Improperly
implemented SSL handshaking, especially when dealing with payment transactions
like they do, has the potential to be devastating. But this is where
interacting with the community that is making the api's would be key. There's
a lot of value to be had when company and community development work together.
They could contribute the ssl code themselves to the open source client
projects.

------
pbreit
I don't really care for this. The promise of web technologies, which has
finally become reality over he past few hears, is obviating the need for
client libraries. This benefits. Oth providers and consumers alike. The
benefits cited by the OP are not that compelling. Most (all?) other APIs
allow/encourage direct access so for a large number of developers, that's a
strong preference.

------
waffle_ss
The title should add "by itself" at the end, as they are still using REST -
just adding client libraries.

------
drivebyacct2
This headline is atrocious, this article has so incredibly little to do with
REST mechanics.

