
The Future is Hypermedia APIs - Milagre
http://www.emergentone.com/blog/the-future-is-hypermedia/
======
thibauts
_> Hint: Your browser is an API client_

Apart from parsing, _I am_ the API client in this instance. I'm the one making
decisions from understanding what services are about.

The universal API client seems to be a kind of silver-bullet-IA in many
people's mind. I feel it's both impossible and not necessary to accomplish
that. The complexity we have to handle to use APIs is already minimal.

What makes hypermedia interesting IMHO is (minor point) easier intra-linking
within a webservice, and (major point) _the ability within a service to point
to resources from another one_.

 _This_ will enable the distributed web we are all waiting for.

~~~
Milagre
> Apart from parsing, I am the API client in this instance. I'm the one making
> decisions from understanding what services are about.

Your browser parses, yes, but then it goes and fetches images, css,
javascript, iframes, etc. Following the analogy, these are 'api calls'. It
knows to make these calls because the HTML defined the links, and for the HTML
to be rendered the contents of the links are required.

I maintain the browser is the client - it responds to your actions, but it
does so with a lot more activity than just parsing.

>What makes hypermedia interesting IMHO is (minor point) easier intra-linking
within a webservice, and (major point) the ability within a service to point
to resources from another one. > >This will enable the distributed web we are
all waiting for.

Couldn't agree more. But without some standards, those links to other services
may not be as useful as they could be.

~~~
thibauts
Sure css, js etc are fetched, but this is hard-coded behaviour, there is
nothing dynamic or discovered about it. Or I fail to understand it.

This may well be the problem then, as we can see here and elsewhere. Many
people like me don't understand. The claims are too vague. We need real-world
exemples of how this can be applied with clear benefits. Mere transposition of
the hypertext qualities to a medium intended for machine consumption will not
do it for most of us.

~~~
Milagre
> Sure css, js etc are fetched, but this is hard-coded behaviour, there is
> nothing dynamic or discovered about it. Or I fail to understand it.

It grabs the links from the content of the document it retrieved, using the
rules of HTMLs structure to find them. That's both dynamic and discovered.

------
jsankey
_“Proper” use of a REST or hypermedia API generally requires you only ever
initiate communications with the API from the root (<https://my.api.example/>
versus <https://my.api.example/some/other/url>). From there, every response
from the API will then contains links to other resources, often with a
relation to explain how each resource relates to the next. In this way, you
never guess a URL - you are always given it._

Speaking from practical experience using an API like this: it can result in a
lot of extra round trips to get to your final destination. The indirection is
nice purity-wise, but in practice do you really want to make 3-4 times the
round trips to the server for every API interaction? Especially when you might
be on a mobile network? All for the sake of flexibility you may never need?

I'd say supporting the discoverability for learning, and consistency etc is
nice. But in practice concerns like this (and the DHH example, and others
already in comments here) will crop up and clients will start building their
own direct URLs. So the idea that you'll be able to change the URL structure
without affecting clients is a pipe dream.

~~~
steveklabnik
> it can result in a lot of extra round trips to get to your final
> destination.

You can usually get around this. Many people make their resources have too
much hierarchy; there's nothing inherently unRESTful about having a flat one.

You should _not_ be having 3-4 round trips for every API interaction. Each
request is an interaction. Hypermedia APIs expose a workflow, not a data
model.

~~~
Milagre
> Hypermedia APIs expose a workflow, not a data model.

I would say this is true of REST APIs. Not necessarily hypermedia APIs. An API
that directly exposed a graph database as an HTTP-based API using
hypertext/links within resources would still need to be classified as a
hypermedia API, but would fail to adhere to the HATEOAS principle.

Edit: More specifically - a graph database that exposed itself with links for
every edge and a resource for every node.

~~~
steveklabnik
I personally draw a distinction between "Hypermedia APIs" and 'hypermedia
APIs.' The former is 'orthodox REST' and the latter is 'an API that serves up
hypermedia.'

------
pdonis
I'm confused about something. The author's "simple and easy solution" to the
problem of people saving API URLs is to have a "bookmark relation" link; but
all that means is assigning a unique, permanent name to the resource so people
can find it again. Why wouldn't you just design your API so the API URL _is_
that unique, permanent name? The "bookmark" link "identifier" ends up being a
URL anyway; why not make _it_ the API URL to start with?

~~~
Milagre
This is a fair point!

Mostly, I think that would produce an ugly API.

The reason I've been leaning towards that solution is that it assigns the
responsibility of locating any other resource, in one request, with full
backwards compatibility, to exactly one location, freeing other locations to
change in a more fluid and graceful way without the cruft.

Edit: Oh and because other URLs can have the context of their parent paths
deliver some value or information. But then again URLs should be opaque =).
Definitely worth thinking more about!

~~~
pdonis
_Mostly, I think that would produce an ugly API._

I don't see why the API would have to be ugly. The examples of "bookmark" link
URLs given in the article are perhaps ugly (using query strings, etc.), but
there's no reason why such URLs couldn't be "nice" ones. The article talks
about things like "what if I want a URL structure like /User/projects/5
instead of /projects/5?", but that's a red herring: you could just as easily
say "what if I want a bookmark query string that looks like
?user=User&project=5 instead of ?project=5?".

The bottom line is that as soon as you have a public API, you have identifiers
that can't be easily changed because people need to be able to bookmark them;
and those identifiers can't be too ugly because they are visible to users. So
they should work OK as API URLs.

------
sopooneo
I am having deja-vu. When I was just starting, everyone screamed about pure
"semantic" markup and how all css class names should describe _what_ was
contained in a node, and have nothing to do with how it was presented. People
bent over backwards to comply. I couldn't make it work, but assumed I was just
far behind the curve. A few years later and that roar is gone. Bootstrap.css
for everything, with class names that describe structure. I can't say I'm hurt
by this as it is what I had resorted to all along. But the absolute 180
without even a hint of the old battle cry in the air is startling.

I feel like that is again happening here. I _can not_ image how fully
compliant REST api's would work in the real world. And when I ask for an
example? _Twitter, check the Twitter API!_ But no, that breaks the rules all
over the place in the name of practicality. _Oh yeah? You want a real example?
How about the whole web? The whole web is your example!_. That example is so
far outside of instructive or helpful I don't even know where to start. I
stand agape, unable to even speak. The way clients access my blog system
should be based on the whole web? Even though it itself is on the web?

I'm done chasing prophets. In five years I do not believe this REST mania will
exist anymore, and not because it is so ubiquitous as to go unstated. It will
have died or changed drastically in the name of actually working.
Bootstrap.rest. Either I am too dumb, or the movement is crazy, or both. But
either way, I'm done trying to make things in a way I don't understand. The
great and holy can thump their Fielding Bibles with faith that they will be
taken home soon. Me, I've given up on salvation, and find the road to hell a
much less exhausting one.

~~~
steveklabnik
> I can not image how fully compliant REST api's would work in the real world.

Have you ever used GitHub's API?

    
    
        $ curl https://api.github.com/

~~~
mjs
Oh! If that's the sort of thing we're talking about, then I'm all for it. The
proponents of Hypermedia APIs have been saying a lot of vague things about how
it will enable automated clients, and I assumed that the clients they had in
mind would be able to do something useful with arbitrary APIs.

But it's clear from GitHub's API that it will do nothing of the sort: those
link relations (and even concepts) are very much GitHub specific, and no
machine will ever be able to figure out what to do with the them, what methods
or media types the endpoints support, and so on.

(I'm not sure that it adds very much over a text document that that gives the
relations and how to generate endpoints, but at least it's cheap to produce,
and not actively harmful.)

~~~
steveklabnik
> The proponents of Hypermedia APIs have been saying a lot of vague things
> about how it will enable automated clients, and I assumed that the clients
> they had in mind would be able to do something useful with arbitrary APIs.

Consider a web browser, or an RSS reader. These are generic hypermedia clients
that consumer standardized media types.

> But it's clear from GitHub's API that it will do nothing of the sort: those
> link relations (and even concepts) are very much GitHub specific, and no
> machine will ever be able to figure out what to do with the them, what
> methods or media types the endpoints support, and so on.

Right. We're not talking about AI, we're talking about generic re-usable
components.

> (I'm not sure that it adds very much over a text document that that gives
> the relations and how to generate endpoints, but at least it's cheap to
> produce, and not actively harmful.)

The point is that by forcing you to define the communications protocol up
front, you de-couple clients and servers. This means that they can evolve
separately. Think about the RSS example: new versions of RSS clients can be
made, and the servers don't need to be updated, and servers can update
themselves and their responses and you don't need a new version of the client
to handle it.

The other advantage is that if there are multiple services in the same product
domain, and they use the same type, you get generic re-use of clients across
services. Everyone spits out RSS, everyone consumes RSS.

~~~
mjs
RSS is a great example, because about the most sophisticated link relation in
RSS is a link. But there seems to be a big difference in kind between what's
being suggested for hypermedia APIs and the example of RSS.

With RSS, the client doesn't need to know very much--only one method is
supported (GET), it corresponds to a click, and you'll almost always get one
media type back (text/html). The IANA define some other generic link
relations, but there seems to be a very big gap between generic link relations
("self", "back", and so forth), and application-specific link relations that
are necessary for non-trivial apps.

How is GutHub's API useful to hypermedia libraries? A "library" that could be
shared between GitHub and a different endpoint would be maybe 10 lines of
code? HAL has a bit more structure, but we're talking maybe 100 lines of code.
(Versus URI templates, which must be hundreds of lines, and with much more
complicated rules.)

~~~
steveklabnik
> But there seems to be a big difference in kind between what's being
> suggested for hypermedia APIs and the example of RSS.

Well, RSS is just a simple example: its domain is small. Which is why it's
nice to talk about. If you need bigger and more powerful things, than stuff
gets complicated.

> With RSS, the client doesn't need to know very much--only one method is
> supported (GET),

I should have said ATOM and ATOMpub; it has a slightly more complex
edit/delete workflow as well.

> The IANA define some other generic link relations, but there seems to be a
> very big gap between generic link relations ("self", "back", and so forth),
> and application-specific link relations that are necessary for non-trivial
> apps.

Right. The idea is that you use app specific ones at first, then, as you find
they're useful and stable, you make them a generic one.

> How is GutHub's API useful to hypermedia libraries?

This is backwards. It's more like "how are hypermedia libraries useful for
implementing the GitHub API."

~~~
mjs
> Right. The idea is that you use app specific ones at first, then, as you
> find they're useful and stable, you make them a generic one.

Ok, but do you think there will will ever be a generic definition of "user" or
"product" or "photo" that can access the full experience of e.g. Amazon,
Google, Facebook, Flickr and GitHub?

Regarding Atom, Google adopted a extended version of Atom in for some (but not
all) of their products, but no-one else adopted it in almost a decade, and
Google are now deprecating it. (I'm sort of blurring the difference between
link relations and media types, but they seem entwined.)

URIs and shared media types are great for interoperability, but my guess
(we'll see if it comes true) is that pretty much anything beyond that isn't
going to get traction. Hypertext was around before the web, but I think a big
reason it took off is because had very loose interoperability demands. (e.g.
links can break.)

------
paul_odin
Is there a list of Hypermedia-esque APIs in use today (apart from sitemap)? I
can find lots of explanations of the concept but it's hard to find examples.

The closest I can think of is the way some APIs handle pagination (GitHub and
Recurly for example) by using Link: rel=next/prev/start headers rather than
through GET parameters, which in principle would allow a generic REST client
to iterate through results on its own.

~~~
steveklabnik
GitHub does it for more than pagination:
<http://developer.github.com/v3/pulls/>

Please see my comment here: <http://news.ycombinator.com/item?id=4949311>

Also, Balanced (YC W11):
[https://www.balancedpayments.com/docs/overview#storing-
the-u...](https://www.balancedpayments.com/docs/overview#storing-the-uri-vs-
id)

~~~
technoweenie
$ curl <https://api.github.com>

We're experimenting with more :)

~~~
steveklabnik
<3 <3 <3.

I really need to buy you a beer sometime.

------
mjs
A few thoughts:

* Mike Kelly (the author of the HAL spec) is not aiming for a universal API client <http://news.ycombinator.com/item?id=4949357> (and also seems to be under the impression that no-one else is either).

* A universal client seems impossible because APIs will typically need to use service-specific link relations for all "interesting" relations (e.g. product, author). You might be able to share generic link relations (like those in IANA's table: [http://www.iana.org/assignments/link-relations/link-relation...](http://www.iana.org/assignments/link-relations/link-relations.xml)), but sharing only these doesn't seem to be worth the effort.

* Hypermedia-powered experimentation is kinda cool, but you're still going to need something to drive it (see e.g. Kelly's HAL browser <http://haltalk.herokuapp.com/explorer/hal_browser.html#/>), at which point you might as well write a full-fledged API dashboard.

~~~
steveklabnik
We already have a 'universal API client' called the browser. Therefore,
implementing a NEW one isn't very interesting.

~~~
mjs
Sorry, I don't get what you're responding to. Browsers don't understand
hypermedia APIs (or even HAL)--they only grok HTML.

~~~
steveklabnik
> Browsers don't understand hypermedia APIs (or even HAL)--they only grok
> HTML.

HTML is a hypermedia media type. A server spits it out, the client (a browser)
consumes it.

A browser is a generic hypermedia API client, for services that expose HTML.

~~~
AffableSpatula
please can we not stretch the definition of hypermedia "API" to include HTML.
HTML is a hypermedia type, but it's primarily for representing information for
human consumption, so it's not an API in the sense most people understand..
why confuse things?

~~~
steveklabnik
It's not stretching the definition: HTML and the web are where RESTful
principles are defined from. They're the OG hypermedia API.

> it's not an API in the sense most people understand

Neither are hypermedia APIs in general.

~~~
AffableSpatula
I know where REST came from, my point is that - as far as normal people are
concerned - HTML web apps are.. apps.. not APIs.

------
notJim
The HATEOAS wonks are crazy, IMO. Most people who consume your API will have
no idea what HATEOAS is and they sure as hell won't care. What will they care
about? The URL to hit to get the data they want. They will hardcode that URL
in their app, too, so expect any changes to it to require a great deal of
fanfare to re-educate your users.

In theory, the URN solution could work, but it's far more complex and requires
more maintenance than simply having URLs that can be relied upon. His solution
to the problem of breaking bookmarks is also a lot of work for consumers of
his API--they have to two codepaths: one for when the URL works like it
should, and another for when it changes and they have to use hypermedia to
find it again. Again, most consumers of the API aren't going to give a shit.
They are going to hardcode URLs and complain when you break them.

Not only that, but consider the implications of an API that requires you to
start from the beginning each time for mobile traffic. Each request over a
mobile connection takes an eternity due to high latency. You're seriously
telling me that you want to force users of your API to make N * depth of
resource requests, instead of just going straight there, when each request
takes 100 ms or more? Okay, so maybe you can cache it, but that just means the
user's first impression of the app is that it takes 500 ms just to do the
first thing. Pretty shitty first impression.

The comparison to a browser doesn't hold much water for me either. Users (and
search engines) will save your URLs, so if you break them, you will lose
traffic. If I try to go to my bookmark and get a 404, you really think I'm
going to take the time to find your page again? Maybe, but it's much more
likely I'm going to shake my head and close the tab. Of course Google will
eventually find your page again, but it will have lost the SEO points it
gained, which means you'll have to start rebuilding your SEO from scratch.

The other issue with the browse as hypermedia engine that he leaves out is the
human brain. The human brain is excellent at extracting information from text
data, so there doesn't need to be any kind of common standard or format to
browse a webpage. The brain just figures it out that when you say click:
[here], [here] is a link that goes to whatever you were just talking about.
That dramatically reduces the burden on webpage authors to conform to any kind
format. His theoretical (well, implementations certainly exist) universal REST
client only works when every API you want to consume standardizes to
presenting the information in the same format.

I drank the HATEOAS Kool-Aid when I first learned about REST too, and then I
built a beautiful API that used it. What did the developers ask for? A list of
URLs.

The future is almost never $elegantly_designed_complex_system. It's almost
always a pile of garbage, with a few humans sitting around sorting through the
garbage to find the thing they want.

~~~
steveklabnik
> Most people who consume your API will have no idea what HATEOAS is and they
> sure as hell won't care. What will they care about? The URL to hit to get
> the data they want.

A few years ago, before Rails made "REST" popular, this exact same statement
was made. "Nobody is going to want to learn about PUT and DELETE. They just
want to do everything over POST."

> They will hardcode that URL in their app, too,

This is an education problem. We're still in the early days of this stuff.

> One for when the URL works like it should, and another for when it changes
> and they have to use hypermedia to find it again.

This is a tooling problem.

> You're seriously telling me that you want to force users of your API to make
> N * depth of resource requests, instead of just going straight there, when
> each request takes 100 ms or more?

One thing that's nice about hypermedia is that you can lead clients to
wherever you want them to go. You're free to change things at any time. Have a
mobile app? Make less features, and make the request paths shorter. Have a
desktop app? Make them longer, add more stuff. It's up to you. Nobody says
that you have to complete N steps to do anything unless your server does.

> Users (and search engines) will save your URLs, so if you break them, you
> will lose traffic.

Right. So what do we do on the web? We 301 redirect until we don't have enough
traffic to care anymore. Same thing with your API.

> The future is almost never $elegantly_designed_complex_system. It's almost
> always a pile of garbage, with a few humans sitting around sorting through
> the garbage to find the thing they want.

Luckily, Hypermedia is the _opposite_ of Big Design Up Front. It lets you
evolve over time. Partial application still yields benefits.

Please see my comment over here: <http://news.ycombinator.com/item?id=4949311>
GitHub, BalancedPayments(YC W11), and others are finding real benefits today.

~~~
notJim
You might be right, as you've clearly thought about this more than I have. I
have a couple of thoughts though.

> A few years ago, before Rails made "REST" popular, this exact same statement
> was made. "Nobody is going to want to learn about PUT and DELETE. They just
> want to do everything over POST."

> This is an education problem. We're still in the early days of this stuff.

The developers I was working with also had a lot of trouble with REST. I had
to explain repeatedly what PUT, DELETE, GET and POST were (they weren't really
familiar with HTTP at all), and that they weren't doing RPC calls. On the one
hand, _I_ feel REST is a simple and elegant concept and its not much to ask
someone to understand, but on the other hand, when I build an API, I want it
to be easy to use and accessible to developers of wide skillsets and levels of
experience. To me, sacrificing some of my desire for elegance is worth it if
it reduces the level of “education” required.

> One thing that's nice about hypermedia is that you can lead clients to
> wherever you want them to go. You're free to change things at any time. Have
> a mobile app? Make less features, and make the request paths shorter. Have a
> desktop app? Make them longer, add more stuff. It's up to you. Nobody says
> that you have to complete N steps to do anything unless your server does.

Either I'm not understanding you, or this seems totally backwards to the way
APIs should work. You build an API so that users can do things with their
data/your service that you haven't thought about, or cover edge cases that
aren't interesting to you. I feel like my API should enable people to build
the things they want, not limit and encumber them so that I can have my pretty
hypermedia. In other words, I'm not interested in leading clients places. I'm
interested in where they lead me.

~~~
BerislavLopac
"The developers I was working with also had a lot of trouble with REST. I had
to explain repeatedly what PUT, DELETE, GET and POST were (they weren't really
familiar with HTTP at all), and that they weren't doing RPC calls."

You should try using the SQL analogy instead of RPC, as this is much closer to
what a "traditional" developer might expect. With REST, one is not doing a
_procedure_ call, but one is querying to create/retrieve/update/delete data.
Of course, there might be some procedures behind it, but to a client it's just
a data query (just like SQL has stored procedures of its own).

~~~
steveklabnik
HTTP verbs do not map exactly to CRUD.

~~~
BerislavLopac
Of course they don't; but that's good enough for giving an intro to a
"traditional" developer.

------
robconery
I dunno. It seems like the future is _arguing_ about Hypermedia. If the idea
doesn't take hold ... is that _our_ fault as developers? It reminds me of the
OpenID hype - engineers solving a problem that doesn't exist or, worse yet,
solving a future problem that has yet to materialize.

If the answer to this is "educate people" then that's the wrong answer. Or
maybe it's the right answer if you're an IT type that loathes their users. I
know I might sound crusty about all of this but how many years are we going to
chase an idea that only a few people seem to understand?

Can we just admit it, right here, right now: _all of this is academic_. We
should be focused on extracting patterns and practices that provide value and
_cycling on that_.

The rest is noise.

------
breckinloggins
I think these ideas would get a lot more traction if they were presented in
terms of something like "Semantic REST" (S-REST?).

Basically, I'm thinking that the ultimate benefit to having somewhat
standardized interactions with our web APIs (with more included metadata) is
that they can be mined, discovered, and manipulated intelligently by
algorithms.

It helps people, sure, but just like using contact microformats helps give the
user a phone number they can click to call, it's equally if not more important
that it gives a crawler an easy way to say "hey look, a phone number!"

This is the main criticism I have with DHHs comments on hypermedia: they seem
to focus too much on the human consumer (and often the programmer) and not
enough on the algorithmic potential.

~~~
steveklabnik
"Hypermedia" was chosen at a term because it gets right to the core of what
"REST" (made popular by Rails and other frameworks) misses from REST: the
hypermedia constraint.

