
Getting hyper about hypermedia APIs - thisduck
http://37signals.com/svn/posts/3373-getting-hyper-about-hypermedia-apis
======
AffableSpatula
I wrote the spec for application/hal+json that got compared to WS-star ,
here's where I'm coming from:

JSON doesn’t have links. Establishing some basic conventions for that makes
complete sense. Defining those conventions is called a spec. Giving a payload
that follows conventions a name also makes sense. Establishing that is called
registering a media type identifier.

It makes no sense to keep reinventing the linking wheel in every API .
Pretending like a very minimal media type like hal+json is akin to WS-* is
incredibly disingenuous and/or stupid.

Establishing a standard media type like hal+json with a bunch of conventions
allows us to build generic tooling that can help with both serving and
consuming payloads that contain links.

Being pragmatic is great, but misrepresenting a genuine effort to improve the
status quo and improve the API ecosystem in a reasonable, non-complicated
fashion as ‘hand-waving’ or 'unnecessary' is not very constructive.

~~~
mjs
Are there any examples of what, for example, the Flickr or Twitter API would
look like under hal+json?

I'm struggling to see how application/hal+json would help me write a client to
upload a photo. I imagine that with or without hal I'm ultimately going to
POST the data to some endpoint. The question is, how do I figure out what
endpoint to use?

Without hal+json I need to read the API docs to discover that "/photos/" is
the correct endpoint to post to. But with hal, it seems I still need to read
the docs to discover that within the hypermedia file, the key "photos" holds
the value for the URI template. So either way, I need to read the docs, and
either way, my client breaks if the meaning of the string "photos" changes.

~~~
AffableSpatula
Your applicaiton's link relations should be URLs. This means that you can
expose each bit of documentation for each rel at their URL. So every time you
see a link in a hal+json document, you can follow the _rel's_ URL, and fetch
the documentation for it.. that turns out to make the API very easily
discoverable, and gives you a way to model/manage your API's documentation in
a consistent way.

You can actually see what that looks like in practice here (click one of the
book icons on one of the links on the left):

<http://haltalk.herokuapp.com>

~~~
mjs
Do you mean like <http://haltalk.herokuapp.com/rels/signup>? But every site
will have different requirements or mandatory fields on signup, so there's
still no "discoverable" way to sign up to Amazon and eBay and Google without a
human looking at <http://amazon.com/rels/signup> and
<http://ebay.com/rels/signup>, etc., in which case you're back to still
needing to read the documentation.

I guess it's a slight improvement to know exactly where the documentation for
a particular link relation will live instead of having to search for it, but
it doesn't seem to solve an especially difficult problem.

~~~
AffableSpatula
it is an example of a discoverable API with discoverable documentation though,
which is what you were asking for I think.. maybe not?

~~~
mjs
Ah, maybe I misunderstand what you're trying to do. I thought Hal was trying
to make it possible to write clients that can buy from Amazon or eBay without
knowing anything about the specifics of either. But actually what it's
attempting is a sort of machine-assisted documentation system? (Since you
still need a human to read the docs to find out how to signup, place orders,
and so on.) I can see how that might work, but it doesn't seem like an
important problem to solve.

~~~
AffableSpatula
That is one of the benefits that relates to documentation, which is the part
of your comment I was responding too.

The larger goal of hal+json is to establish some conventions for linking that
allow the development of generic tools for doing hypermedia. Not to create
magical machine clients that can interact with any random API you point them
at. Nobody made that argument so I don't know why DHH addressed it in his
post. I'm guessing he ran out of things to be an angry-pragmatist about.

~~~
mjs
Well ... the spec does say "HAL is a bit like HTML for machines, in that it is
generic and designed to drive many different types of application." To me (and
perhaps DHH), that suggests that it's designed to be used to create clever
clients. Perhaps you can clarify this point in a future edition of the draft?

<http://stateless.co/hal_specification.html>

~~~
AffableSpatula
I have no idea why you would conclude that from that sentence. HTML allows
browsers, the same way that HAL allows generic libraries. I'm struggling to
understand why this is controversial, there are already 16+ examples of
generic HAL libraries across a bunch of different languages.

------
gamache
DHH's API philosophy (send simple JSON serializations over the wire, mostly
from the server to the browser) got him through the 2000's OK, but the longer
he argues against hypermedia, the less relevant Rails becomes for API design.

These are not particularly good arguments he puts forth here. I believe they
are in response to Mike Kelly (designer of the HAL+JSON hypermedia format) and
his recent post: [http://blog.stateless.co/post/38378679843/hypermedia-apis-
on...](http://blog.stateless.co/post/38378679843/hypermedia-apis-on-rails-why-
dhh-should-give-a-fk)

DHH's comment that URLs instead of IDs are a good idea is true (but he even
gets that part wrong in how he implements it, leaving a '.json' extension on
the URL). And the rest of the article is trying to hand-wave away the value of
hypermedia (links -- all hypermedia means is links, at its core).

And comparing HAL+JSON, a blessedly lightweight standard, to WS-* is just
dirty.

This post didn't convince me of much other than that DHH may run out of steam
on this issue before too long.

~~~
fbuilesv
_These are not particularly good arguments he puts forth here_

Would you care to explain why you think they're not particularly good? You're
blaming DHH of being "hand wavy" but I don't see any more depth from your
comment.

~~~
gamache
I spell out some concrete advantages to hypermedia APIs elsewhere in these
comments. <http://news.ycombinator.com/item?id=4948652>

But to specifically address DHH's arguments from the article:

* Enabling Discovery is a strawman argument; no one expects an API to use / as its only documentation (at least no one expects it of a good API).

* Standardizing API Clients is also a strawman; no one expects to have one generic client magically make sense of any API.

* Comparing HAL to WS-* in order to paint it as committee-driven standards bloat is not fair to HAL, which is an admirably tight and cogent specification.

------
steveklabnik
Bias alert: I'm the newest Rails committer and one of the bigger proponents of
Hypermedia APIs in the Ruby world.

Anyway, many, many other companies _do_ find hypermedia principles to be
useful. See Balanced Payments, for example:

> Fun fact: our internal statistics show that client libraries that construct
> the uri receive roughly 2 orders of magnitude more 404 status codes from
> Balanced than clients which use the uri directly. > >
> [http://www.theatlantic.com/magazine/archive/1999/03/the-
> mark...](http://www.theatlantic.com/magazine/archive/1999/03/the-market-as-
> god/306397/)

GitHub is starting to add hypermedia stuff to their responses:

<http://developer.github.com/>

Here's their main API guy talking about the advantages even this partial
implementation has achieved:
<https://twitter.com/pengwynn/status/281849041707474944>
<https://twitter.com/pengwynn/status/281849329243787265>

Twilio has always had elements of hypermedia in their API, and are considering
moving further in that direction in the future. They had me speak at their
conference for the last two years in a row about the topic specifically.

That said, if not doing hypermedia doesn't hurt you, don't change what you're
doing! If you're interested in evolving your API over time while supporting
old clients and behaviors in a simpler way, then consider checking it out.
REST/hypermedia APIs are focused on long-term stability, evolvability, and
massive scalability. If you don't need those things, you don't need
hypermedia.

That said, I'd be happy to answer any questions on the topic, though I'm
really busy today, so it might take a while to get back to you.

~~~
necrodome
What are the best options to implement that for an API backend with Rails <->
a javascript library/framework for the interface?

~~~
steveklabnik
The current Rails Party Line is "Jbuilder + you don't need integration"
<https://twitter.com/dhh/status/281802247480958976>
<https://twitter.com/dhh/status/281802391316201472>

Some members of the core team and I have started a "Rails API" project
specifically to explore these possibilities, and to extract common patterns
from the apps we build: <http://github.com/rails-api>

This will encompass hypermedia and non-hypermedia APIs, as well as SPAs with a
Rails backend. Most of us are focusing on Rails 4 at the moment, and will
start working hard on this post-Rails 4 release, but about 23,000 people have
installed the main gem, and I know several running it in production already.

I personally feel the best option right now is "Rails API +
ActiveModel::Serializers + Ember.js". But we want to encourage a multiplicity
of options. Not all apps are identical.

~~~
mahmoudimus
We also have developed custom Python tools and frameworks that help us with
our API at <https://balancedpayments.com/>

If there's enough interest, I'd love to share some of our ideas with the Rails
community and discuss some potential downfalls and successes we've had and why
we needed to build these tools internally.

I'm sure Rails would benefit a lot from these.

Love love love the hypermedia work -- keep it up!

~~~
steveklabnik
That would be fantastic. If you find the time, we have a discussion list over
at [https://groups.google.com/forum/?fromgroups#!forum/rails-
api...](https://groups.google.com/forum/?fromgroups#!forum/rails-api-core) ,
please feel free to share thoughts there!

------
bct
His questions about discoverability seem to assume that people are suggesting
not writing any documentation at all.

"The idea that you can write one client to access multiple different APIs" is
a straw man.

The connection he's making between HAL and WS-* is ridiculous.

------
programminggeek
hypermedia APIs mean that our API's basically become web pages and every
client should act like a browser. That's great if you want... a browser app.

If you want to just expose data to let people build interesting things with
it, hyperlinks in your api are a bit silly. They might be nice if you want
someone to write a browser for your api, I don't see where else it would be
awesome. Maybe if you want a web crawler to crawl your api.

Also, they add a layer of chattiness to your app. If you have well documented,
unchanging url's people can write apps against those. If people have to go to
a resource to find the url to another resource, you're going to have a lot of
API requests just to look up URL's. People will realize that is a waste of
bandwidth, and will hard code URL's anyway.

I rarely agree with DHH, but he's right.

~~~
icebraining
So you could write an app against the API of some service, then a competitor
of that service comes and implements the same API, and so you want to switch.

What should you do? Replace every URL? What if you want to support _both_
(say, Twitter and Identi.ca)? Should you implement a map from codename ⇒ url?

Now what if you want to support _every possible implementation_ , even if you
- the developer - don't know about them? Why shouldn't the user be able to
plug-in the entry URL and use your app?

People snickering about hypermedia APIs seem to me like people ridiculing the
idea of having standard ports and protocols for devices, because they can't
imagine a world where you don't have to install yet another crappy 200MB
driver that is only available for Windows 95 to use a damned mouse.

~~~
programminggeek
I think that assumes some serious hand waving on the part of both your app and
the api's. Assuming that just pointing at the root of another API is going to
mean your app will magically behave properly using both API's is making a lot
of assumptions either in your app about how the API's work. For example, say
one API calls a delete call "delete", another calls it "remove", another calls
it "obliterate", and at the same time one API takes an id on that method as
input, another takes an email address, another takes a username. In that
scenario your app might be written to remove something based off an id and
when you switch API's the new one expects you to obliterate based on username.
Hypermedia doesn't save you there.

I'm all for pluggable protocols and standards, but Hypermedia links in your
API don't magically get you there. They might help, but no more than a
standard spec with standard URL patterns would.

------
icebraining
_According to hypermedia lore, you will be able to willy nilly change your
URLs without needing to update any clients. But that’s based on the huge
assumption that every API call is going to go through the front door every
time and navigate to the page they need. That’s just not how things work._

This is missing forest for the trees. The point is not that you, the API
implementor, will be able to change URLs willy nilly. It's that I, the client,
can support a _different_ API implementation by just changing the entry point
URL, without changing the application.

So, if my application supports the API that Flickr implements, and tomorrow
someone creates Blinkr, which implements the same API, the user could just
copy-paste the entry point URL and use it, just like I use my RSS reader for
all the blogs and news sites out there.

Of course, this depends on using standard document formats and a restricted,
standard set of methods. Rings any bells?

 _Thinking that we can meaningfully derive all that by just telling people to
GET / and then fumble around to discover all the options on their own just
doesn’t gel with me._

Well, am I glad that straw-man was burned to the ground!

~~~
MatthewPhillips
> and tomorrow someone creates Blinkr, which implements the same API,

So the benefit of hypermedia apis is predicated on something happens that
almost never ever happens?

~~~
icebraining
On the contrary; REST, of which hypermedia is a constraint, is an
architectural style derived from the observation of a very successful
implementation of just that.

It's called HTTP + HTML, and there are millions of services implementing it.

------
RyanMcGreal
This post seems hyper-defensive and hyper-reactionary. URLs are _fundamental_
to HTTP web services - they're how the web works. Imagine if you visited a
website in your browser and instead of link to the various pages, the page
just printed IDs and you had to copy/paste them into the address bar after the
domain name.

------
kodablah
Sounds like a redefinition of HATEOAS. I don't put URLs as my ID's, but every
resource has an "href" value which is the URL which makes it somewhat
discoverable by tools such as <https://github.com/jed/hyperspider>.

~~~
steveklabnik
"Hypermedia APIs" are just "Real REST." Arguing over terms isn't productive,
so many RESTafarians just stopped:
<http://blog.steveklabnik.com/posts/2012-02-23-rest-is-over>

HATEOAS sounds big and scary. "include some links" is much more
understandable.

------
mamcx
I use <http://www.remobjects.com/ro/>, and have something similar, called RODL
Files (<http://wiki.remobjects.com/wiki/RODL_Files>), this is how look
like:[http://wiki.remobjects.com/wiki/RODL_Library_(Service_Builde...](http://wiki.remobjects.com/wiki/RODL_Library_\(Service_Builder\)).
This are like SOAP, but better. With them, I can parse the RODL file, and
build a python client automatically in seconds, with a python script that
output a python client for the remobjects server, with all the class, method
calls and that stuff. RemObjects also generate automatically the clients for
.net, delphi, js, obj-c, php
([http://wiki.remobjects.com/wiki/Generating_Interface_Code_fr...](http://wiki.remobjects.com/wiki/Generating_Interface_Code_from_Your_RODL_\(Xcode\))).
Is something I miss very much when try to do a REST server without RO.

And the documentation is embebed in the RODL file, so is possible to output
the client with the docs inline, and get it to show in the IDE when a call is
made to the python client...

So, I think a meta-data about the service is VERY usefull. But the hypermedia
is a poor attempt at that (IMHO).

------
jasonkostempski
I think these are good points for general public facing web APIs that are
specifically designed for being mashed up or will be consumed by third-parties
that will never talk directly to your company. I feel hypermedia lore has some
really worthy ideas for APIs designed for business use where workflow and data
integrity are under strict control but will change frequently and where
several client apps will be made by the same company or companies working
closely together.

------
pkeane
Wow. Very surprising (in a bad way).

Basic Web architecture: 1. expose resources, 2. resources have names (URL), 3.
allow basic actions on these (GET, DELETE , PUT, POST ) as needed. 4. Include
URLs (as links/forms) in representations.

OK, _now_ build your API . Please do NOT start w/the API and work back to
basic web architecture. Servers should _always_ provide URLs (url templates
are fine), NOT the client (by way of snowflakey construction algorithm).

------
MattRogish
Hypermedia hype has always struck me as parallel to the "semantic web"
nonsense from the late 90s.

"If we just use RDF triples to encode everything then machines can learn that
apples are fruits and fruits are good for you, thus apples are good for you!
Huzzah!"

Then, Microsoft invented SOAP.

"If we just have a WSDL that explains all of our API, then we can have
automated methods to communicate between services! Programmers can just auto-
generate code and life is great!"

Except it doesn't work that way. Most WSDL parsers auto-generate code that you
then hand-edit and maintain over time. And, you, as the programmer have to
_know_ what methods to call. SOAP is just excessive ceremony transmitted over
XML - another excessively rigid structure.

So then we got JSON REST APIs. Simple text structures. Reasonable defaults.
Basic vernacular. Easily understood.

What concerns me about Hypermedia APIs is that folks are using the same sorts
of grandiose, architecture astronaut-y stuff that we got out of the last two
failed revolutions.

Hypermedia API proponents say that REST APIs are "highly coupled" (to the data
model and versioning) and don't expose workflows.

Heck, that's why REST is so pervasive - they're super easy to write and
consume/interact with. Enterprisey folks are so focused on long-term
extensibility and maintainability that they overlook the cognitive overhead
and inability to work with it on a daily basis. And that you in fact, tend to
move _slower_ in development because you can't comprehend or follow what is
going on. And that all abstractions leak, leading to libraries and tools
(SavonRB) that don't quite work if an API doesn't follow the spec exactly (and
they never, EVER follow the spec 100%).

My attitude is the opposite: _Don't_ design your APIs as if you expect them to
be the next 1,000 year reich. If your API stays small and nimble enough, your
consumers will also be able to be flexible to accomodate it. Yes, if you have
a User REST API and you decide that your app no longer has users, well, then
you have to get rid of it. But adding another layer or two of hierarchy and
ceremony on top via Hypermedia wouldn't solve that either! Fundamental
universe changes ought to break shit!

From what I've seen (I have a subscription to designinghypermediaapis.com
which is very well written) Hypermedia APIs are an over-complicated solution
to a problem that has a reasonable solution. Yes, links are nice. Want to
propose a "Standard" to handle links? Okay, although that's REST! If you have
an Object and you want to DELETE it, I don't need to know the URL. I have a
convention via REST that allows me to derive it from my data. If that's
different, OK, use this thing. But Hypermedia APIs are a _lot_ more than just
links (state machines, workflows, media types, etc.)

~~~
gamache
_What concerns me about Hypermedia APIs is that folks are using the same sorts
of grandiose, architecture astronaut-y stuff that we got out of the last two
failed revolutions._

You owe it to yourself to read the HAL+JSON specification:
<http://stateless.co/hal_specification.html>

Anyone who considers a three-page spec like that astronautics probably has a
low capacity for complex thought.

~~~
nkohari
A hint: when trying to convince, it's not a great idea to passive-aggressively
insult the other party. Since they're human too, they will probably just get
pissed off and write you off as an asshole. Like what I'm doing right now.

~~~
gamache
Given the level of respect I have for the OP's exhaustively researched points,
I should not have said a thing. Someone on the Internet is wrong. My mistake.

------
thibauts
Aren't links also proper global IDs, allowing us to address objects cross-
service ?

~~~
steveklabnik
Yep. URIs are 'universal.'

~~~
thibauts
I feel there is a lot of noise around IDs as URIs yet I've seen nobody (I may
not have looked deep enought) talk about their main benefit as I perceive it,
which is putting resources into the perspective of a global namespace. Thus
allowing a service to reference resources provided by another service. _This_
to me, looks like a practical revolution. We may at last have our contacts
served by one entity, our statuses by another, our shared files by still
another, and yet all of them could be handled as one distributed API.

The code glueing the whole to produce, say, a distributed facebook-like app,
would still have to know the specifics of each provider (before standards
emerge), but it would effectively break the isolation and allow us to
composite services much more easily.

Where am I missing the point ?

~~~
steveklabnik
> their main benefit as I perceive it, which is putting resources into the
> perspective of a global namespace.

Sometimes people do. I mentioned this problem in my talk at Øredev this year:
<http://oredev.org/2012/sessions/designing-hypermedia-apis>

> Where am I missing the point ?

Well, you're on HN: everyone wants to control everything. Startups don't 'win'
by playing nicely with each other, they drive their competitors out of
business.

------
jashkenas
One neat trick that I think is worth adding to the discussion: URL templates.

Returning URL templates as part of your API response can give you the benefits
of having a clean way to access sub-resources, without the headaches and bloat
of having to enumerate every possible desired sub-URL.

For example, in DocumentCloud, a document's canonical representation has a
unique URL for the content of every page as plain text, and as an image, in
several different rendered sizes. Instead of doing something silly like this:

    
    
        resources: {
          text: [
            "http://www.documentcloud.org/documents/1/pages/page-1.txt",
            "http://www.documentcloud.org/documents/1/pages/page-2.txt",
            "http://www.documentcloud.org/documents/1/pages/page-3.txt",
            ...
          ],
          largeImages: [
            "http://www.documentcloud.org/documents/1/images/page-1-large.png",
            ...
          ],
          thumbnailImages: [
            "http://www.documentcloud.org/documents/1/images/page-1-thumb.jpg",
            ...
          ]
        },
        ...
    

... where you might have 5,000 pages in a document, you can imagine how
unacceptably large that response might become. Instead, a single URL template
can do the work. (<http://tools.ietf.org/html/rfc6570>) The spec has a whole
bunch of goodies in it, but we just need the most basic interpolation feature
for this case (real example, you may have to scroll sideways to see the
complete URL):

    
    
        "pages": 5058,
        "resources": {
          "page": {
            "image": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}-{size}.gif",
            "text": "http://www.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}.txt"
          },
          "pdf": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.pdf",
          "published_url": "http://documents.nytimes.com/sotomayor-s-senate-questionnaire",
          "related_article": "http://www.nytimes.com/2009/06/05/us/politics/05court.html",
          "search": "http://www.documentcloud.org/documents/21939/search.json?q={query}",
          "text": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.txt",
          "thumbnail": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p1-thumbnail.gif"
        },
    

Basically, all of the simple URLs a Viewer might need to use in order to
browse the document, search the text, and view related resources. In the past,
when we've needed to change or expand the number of resources (adding HTTPs-
only support, changing the URLs at which the page images are stored, or adding
larger sizes of page images), it's been relatively easy to do, without
breaking the viewers, or invaliding previously-valid JSON representations of
the document. Here's the complete link to the above:

[http://www.documentcloud.org/documents/21939-sotomayor-s-
sen...](http://www.documentcloud.org/documents/21939-sotomayor-s-senate-
questionnaire.json)

~~~
dribnet
> One neat trick that I think is worth adding to the discussion: URL
> templates.

100% agree - URL templates enable succinct discoverability. Moving link
generation to the client reduces computation time on the server which no
longer has to generate inline links as well as the payload size of the
response.

The RFC6570 spec is great, has numerous implementations[1] and a through test
suite[2]. I am also surprised that this is not a standard part of every
hypermedia implementation.

[1] <http://code.google.com/p/uri-templates/wiki/Implementations>

[2] <https://github.com/uri-templates/uritemplate-test>

