
Everything Should Have An API - scottjackson
http://words.scottjackson.org/post/232078864/everything-should-have-an-api
======
mrshoe
I agree with 95% of what he says, but I'd make one change to the Unix
philosophy: text files aren't the best primitive.

Text files have a super low-level API (stat, seek, read). I think the system
he describes would be much more usable if he picked slightly higher-level data
abstractions. I would recommend Python's built-in types: list, tuple, dict,
set. Most data can be easily represented by these types and they provide a
higher level API than streams (files), which means the barriers to using them
are much lower.

~~~
scottjackson
That's probably fair. I honestly don't think we're at the point where we can
abandon text files altogether, but you're right -- a lot of data could
definitely be stored in slightly higher-level data structures.

I still _really_ like text files though :)

~~~
emmett
Text files are just a wire protocol. It would be nice to have a standard
(similar to http) for what you send over the wire in general.

Great post by the way, very thought provoking.

------
dwiel
The printing press enabled more people to read, though still few could write.
The net has enabled more people to write, though still few can program. A net
where everything has an API is an extremely helpful step in creating a system
which enables more people to program.

------
hxa7241
The scenario descriptions are distinctly reminiscent of Berners-Lee's scenario
descriptions for the 'semantic web' several years ago. Of course, that is now
rather out of fashion, but not entirely justifiably. The 'semantic' angle was
somewhat fanciful, but the core idea was _the data web_ : It is one thing to
put APIs on everything, but another for all the data types and formats to work
together. That was more the aspect that RDF was addressing. It is neither
thoroughly convincing nor completely apposite, but worth factoring in to the
consideration anyway. (The essential RDF concept was elegant, whatever anyone
says.)

------
clawrencewenham
Having a good API for everything would be almost as good as The Singularity
(tm), but making a good and secure API is very hard and comes with a massive
hidden cost.

1) An API can require even more planning to design than the application or
service itself.

2) Mistakes made in the design or implementation of the API turn into
liabilities you can't solve without breaking users of the API.

3) The cost of maintaining an API so that it and its documentation doesn't
fall out of date with the app/service can be as high (and sometimes even
higher) as the cost of maintaining the app itself.

4) It widens the attack surface.

And those are just the ones off the top of my head.

~~~
grncdr
I agree with all of your points, but I think many of these costs are minimized
by the other point mentioned in the OP: following the UNIX philosophy. The
smaller and simpler your service is, the better. If you have a service that
can't be accessed with a simple API, you probably have more than one service,
and would benefit from splitting it into multiple APIs.

Another thing worth mentioning is the possibility of building your API first,
and then building your service on top of it. If you are providing a web
service, then it [usually] boils down to data access and management. Figure
out what data you are managing for your users and what you need to do with it,
then build your API around that. If you plan to have a public facing API from
the start, this approach makes sense (in the same way that using javascript
only for progressive enhancement of a working site makes sense).

Of course, because you are an "internal user" of the API, you can bend the
rules a little more than the average client, but the less you do this, the
more useful your API is.

Regarding points 2 and 4: if public APIs for data access become the norm, this
will simply become the cost of doing business. Secure API's aren't _that_
difficult, and updating an API while maintaining backwards compatibility can
be done, especially if you transition these changes and notify all broken
clients.

~~~
Periodic
For the UNIX philosophy I like the idea of small, useful tools loosely
coupled. A big part of UNIX is that there are a lot of little tools that do
_one_ thing well. The smaller the tool is, less attack surface there will be,
and each tool can interact with a common authentication/authorization
interface to further simplify things.

------
tocomment
Should creating email accounts have an API? How would the captcha be
presented? How about adding reviews of products?

~~~
silentbicycle
Don't captchas and open APIs fundamentally oppose each other?

The point of an API is to have a clear interface so that programs can be built
on a service, and captchas _exist_ to try to block bots from using services.

~~~
tocomment
That's my point. Not everything <should> have an API since there are so many
things we put captchas around.

------
bkudria
I want, I want, I want. How about you propose how this should be done? It's
harder than it looks, and no, Unix is not the one-and-all answer to
everything.

Take one of his example scenarios:

"When my alarm clock goes off in the morning, get the latest tech news podcast
(downloaded overnight) from my computer via WiFi or a wired network
connection. Play the podcast through the alarm clock’s speaker/s. If I hit
“snooze” on the alarm clock, pause the podcast. Once I stop the alarm,
remember where I was up to in the podcast and give that information back to my
computer. Now when I go to listen to the podcast while I’m eating breakfast,
it will continue playing where it left off."

This has a number of hidden assumptions:

* What is "the latest news podcast"? One user-selected source, or a blended filtered combination, or a crawled set selected based on previous behavior?

* How does a (not _the_!) computer know how to talk to the alarm clock? I hope I don't have to create a home network myself, cause I know my Mom certainly can't. How is this connection authorized? What happens if the power goes out on the computer - what will the clock do? What if the clock batteries fail - will the computer play the podcast instead?

* Is the saved position synced only the one source computer (I hope not). What if the network to the computer is down - can the alarm clock sync somewhere else?

* Can I resume listening to the podcast on a different computer at breakfast? My partner/sibling/friend's computer? How do I log in? How is the position data saved along with my identity? How does the breakfast computer address the first computer, or the alarm clock?

The problem here is not access to the external APIs (not that hard) but
federation, identity management, authorization, privacy considerations, and
addressing between these disparate systems. Proposal adopting existing
standards for these requirements, or new proposals for standards are, of
course, appreciated.

~~~
noste
On the existing standard front, there is Universal Plug and Play
(<http://www.upnp.org/>) that aims to "to enable simple and robust
connectivity among consumer electronics, intelligent appliances and mobile
devices from many different vendors." I haven't played with this stuff at all,
but there appears to be fair amount activity in this space:
<http://en.wikipedia.org/wiki/UPnP_AV_MediaServers>

I don't know how much of the example scenario you could implement within the
confines of UPnP, but perhaps it could be part of the solution.

------
grncdr
As previously said, the financial incentive is simply not there. As much sense
as it makes for you to have easy access to your information, there are too
many businesses that rely on controlling that access.

On the other hand, tools to do this with almost any site already exist. RSS is
common enough, and where it doesn't exist, screen scraping isn't that hard
(and getting easier all the time). If I'm not mistaken, this is the exact
problem that Yahoo pipes are trying to address.

Of course, sites that require a complicated login process (banking sites) are
still out.

~~~
stuartjmoore
I think Twitter is an example that owes its success to its API; it depends
what your business plan is.

As more companies become Google-esque data hogs, APIs will probably be
encouraged.

------
DanielStraight
As right as you are, making an API for your service so people can replace your
crappy interface with their own good interface probably isn't the best
business decision. Online banking sites can display ads. The bank can't inject
ads into another client though.

Also consider that the bank would get blamed in popular opinion if someone
wrote an insecure client that was determined to be leaking data.

~~~
scottjackson
I think it could be done -- I mean, Pine's an example of a secure client
application that uses a kinda-API (SMTP) to deliver data.

If we don't make APIs because someone _might_ write an insecure client, would
anyone make any APIs ever?

I do agree with you though :) It's probably not going to happen in the real
world any time soon. C'est la vie.

~~~
DanielStraight
I'm sure it could be done.

But consider for example if Bank of America did this and someone created an
awesome client app that basically became the main way anyone used their Bank
of America account. If a bug is discovered in this application that means
everyone's data is no longer secure, are users going to blame the app creator
or Bank of America? Can Bank of America afford to take that chance?

~~~
scottjackson
You're right -- that's almost certainly the reason you don't see stuff like
this happen.

Though, didn't Bank of America make that iPhone app a little while ago that
allowed you to photograph a cheque in order to deposit it? They're obviously
OK with sending financial data around. I guess maybe it was OK in that case
because they were the ones that wrote the app?

In any case, you're right. Security is definitely the reason this kind of
stuff hasn't been implemented. I still think that it's possible to write an
open, secure interface to things. Email is the example I fall back on here.
Like I mentioned above - sensitive information gets sent around in emails all
the time, yet there are countless email clients and they're almost all secure
(I say "almost all" to cover my ass if there's some that aren't). I think that
if it can work for email, then maybe it can work in other domains.

~~~
grncdr
Unfortunately, email is a terrible example. While it's possible to have a
modicum of security in regards to receiving your e-mail, spoofing emails is
trivially easy (unless you use cryptographic signing, which isn't well
supported by the majority of clients afaik)

------
d4rt
For my own personal use I think text based interfaces (with pipes) are
superior, the higher level pluggable components in AppleScript/Automator seem
more usable in general.

That said, there was an excellent article recently with the experiences of a
trainer in getting people to use *nix, who found the minimalistic "command
mode" more usable due to it's text oriented simplicity.

------
ryah
I want a build system with an external API.

------
stuartjmoore
API is kind-of a misnomer in this case. API implies each service has its own
set of rules, the data is open, but the access is different.

Everything should talk to one another, but a service's name or url should
never have to appear in any code. That's the user's job to input.

------
SlyShy
Yes. I was just doing my college applications and thinking this exact thing.
The Common Application would look like a piddly idea if the college
application process just had an API.

------
mattmanser
Personally it sounds like a security and privacy nightmare to me.

~~~
scottjackson
Maybe. I think that if we get enough smart, awesome people working on it, it
could be secure, safe and open all at the same time.

------
Raphael
Couldn't agree more. This is what my next startup is based on.

------
rythie
They shouldn't just have an API, they should use the same APIs where possible.
Each site having it's own API is a major barrier.

------
milkshakes
for anyone who hasn't seen it,
[http://www.ted.com/talks/tim_berners_lee_on_the_next_web.htm...](http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html)
provides a nice context

------
dustingetz
well, most data is exposed via a website, so if the api doesn't exist, just
mechanize it. its not really feasible to rely on goodwill for these grand
visions.

------
sophacles
This is ridiculous, we can't just let people have access to their data!... i
mean OUR data. Just imagine, they may learn things we could have forced them
to pay absurd fees to get! :P

------
mrtron
But at what cost?

