
Barack Obama Directs All Federal Agencies to Have an API - mcrider
http://blog.apievangelist.com/2012/06/01/barak-obama-directs-all-federal-agencies-to-have-an-api/
======
polemic
I'm reminded of this recent post by James Fee, talking about geodata, but I
think it applies to the general case:

    
    
        "... The question was APIs or downloads... 
         Personally, I believe [data] is one of the best 
         ways for citizens to keep track of their government 
         (local to federal) ... APIs tend to deliver what 
         their “owners” want them to do. Raw data means 
         everyone has an opportunity to check each other’s 
         work. Of course, raw data can be manipulated as 
         well, but it is harder to obscure."
    

\- [http://spatiallyadjusted.com/2012/04/03/sharing-data-
downloa...](http://spatiallyadjusted.com/2012/04/03/sharing-data-downloads-
are-the-key/)

I couldn't agree more. APIs are great, but are not the key to open government,
for two reasons:

 _1\. They don't provide simple and easy access for non technical individuals
into raw information._

APIs shouldn't exist for querying historical datasets if the dataset is not
already available in a static format. Release the data, then build an API if
there is demand (or the private sector doesn't do it, better, for you).

 _2\. Historical data access is poorly served by APIs._

There is no such thing as a good 'general use' API[1]. API's are appropriate
for specific service based transactions that involve some level of processing.
Examples:

    
    
        * VAT/GST number validation
        * Road closure notifications
        * Identity services
    

_3\. Bonus reason: government agencies suck at building APIs._

They're not good at determining what is genuinely high value to end users,
they tend to prefer visible projects that can justify budget increases, over
genuinely useful, but less easily communicated ones (cf. the US national
highway system and pork barrel politics), and there is an entire industry of
enterprise companies heavily invested in keeping it this way.

 _TL;DR Release the data, let users build the APIs. Everyone wins._

Bootnotes:

 _[1] I lie. That's exactly what publishing raw data at stable URLS on a
website achieves._

~~~
kjhughes
Hmm, not sure API vs data is productive to see as a dichotomy.

Consider that an API might provide access to "raw" data directly.

I'd ask not for data over APIs but for more universally useful properties such
as stability, currency, consistency, etc.

~~~
polemic
> _Hmm, not sure API vs data is productive to see as a dichotomy._

It's not, necessarily. An API provides a [very] limited view into the data. A
view selected by the publisher. That's part of the problem.

> _I'd ask not for data over APIs but for more universally useful properties
> such as stability, currency, consistency, etc._

Amen. I'd add: clear license and usage conditions, simple and concise
metadata, and frequent updates.

~~~
dmoy
I think kjhughes was going for the possibility that one of the things exposed
in the API would be downloadEverything()

Not sure whether that's likely, he was just pointing out that API vs data are
not mutually exclusive.

------
grandalf
As is obvious to most on HN, requiring an API (as opposed to a CSV file
release schedule, etc.) is fairly meaningless, and most definitely not a
presidential-caliber dictate. Some agencies' data might be far better suited
to publication in a CSV and posted on a web page, for example.

If a president could have a meaningful impact on this sort of thing, it would
be in setting a high bar for the quality of information released by agencies.
Any sort of requirement of this kind is completely absent from the
announcement.

So rather than being about transparency as it's being touted, the announcement
is a celebration of high tech obfuscation. Soon the same sort of insulting,
opaque, useless information spouted by officials in press conferences will be
available via HTTP. This is at best a neutral day for democracy.

~~~
mattdeboard
Seems like the executive mandate is an "80% now" solution instead of waiting
for perfection. At least they're thinking about the dissemination of
information.

~~~
chernevik
Seems like, if the President were interested in "now", they would have been
working on this for going on 3.5 years, and would be actually publishing some
APIs.

~~~
chc
I'm going to my apartment now. 3.5 years ago I would not have been going
there. Now and 3.5 years ago are not the same thing. He's interested in doing
things now, and since neither he nor I can turn back the clock, that's as good
as either of us can do.

~~~
chernevik
Just like he's just getting around to planning his re-election campaign "now".

If the President of the United States wants to be impressive -- and I don't
know that APIs are where one would start -- he can do better than make
announcements.

~~~
jpendry
They were putting out ads against Romney at the beginning of the Republican
nominee campaign.

------
kjhughes
I think this should be judged against the status quo as a positive development
rather than against an abstract ideal as a flawed concept. Having seen too
many clients stuck in analysis paralysis or blocked by political/turf issues
while trying to develop corporate-wide standards (protocols, object models,
etc), I'm just happy to see online access to public/government data advance in
any way.

If we had to wait for higher-level, coordinating standards first, progress
might never come.

------
waffle_ss
Meanwhile, just yesterday the House Committee on Appropriations voted to
[indefinitely delay][1] making legislative data available in machine-readable
(XML) format. It's a repeat of a move taken in 2008 to "make a plan to make a
plan" that never really goes anywhere. In other words, it's not gonna happen
for a long time yet.

[1]: [http://sunlightfoundation.com/blog/2012/06/01/bulk-access-
de...](http://sunlightfoundation.com/blog/2012/06/01/bulk-access-developments-
after-the-h-approps-hearing/)

~~~
lnguyen
Don't expect someone to do something that goes against their self-interest.
Especially if they can do a song and dance and make people forget that nothing
is getting done.

------
jroseattle
"...Within 90 days of the date of this memorandum, create a page on its
website, located at www.[agency].gov/digitalstrategy, to publicly report
progress in meeting the requirements of the Strategy in a machine-readable
format.... ...implement the requirements of the Strategy within 12 months of
the date of this memorandum and comply with the timeframes for specific
actions specified therein"

3 months to get a "machine-readable" status report on implementing an API?

Then, complete the implementation in 12 months?

If it takes 3 months for an agency to get a status report up, how long will it
take them to implement said API? Government work, sheesh....

~~~
sp332
I'd like to see you hire an entire department's worth of people, wait for them
to devise an API for your agency, code it, provision server space and servers,
and deploy in 12 months!

~~~
jroseattle
I get that this is a rhetorical question, stated in such a manner to imply I
have insufficient context of the real requirements of this project and cannot
gauge the size and complexity of it. But, I've got a full cup of coffee, so I
will take a shot to see if I can address this impossible situation.

    
    
      - hire an entire department's worth of people

The worst approach I can imagine is to start this project by hiring new,
dedicated people. This project only succeeds by incorporating it into the very
fabric of an agency's operation. Remember, new work must go on even after this
API is in place. Hiring a dedicated group that somehow has to reverse-out
everything the agency does going forward for purposes of external API access
is a recipe for failure. Separate teams will already be at odds with each
other; better to leverage the existing teams, as their the ones in best
position to understand the context of how external access to their operations
should function.

    
    
      - wait for them to devise an API for your agency

I'm not sure if this refers to a lack of understanding the requirements, or to
a lack of competency on the part of the team itself, but the presumed outcome
is late delivery. If it's complex requirements, that simply suggests a basic,
iterative project where scope is managed tightly and the duration is rather
short (allows the team to learn as the project moves along.) If the suggestion
is competency on the part of the team, hiring/contracting a few competent
individuals to align with team leaders on the project has worked well for past
projects. In either case, proper management of the project approach can
address issues of timeline.

    
    
      - code it

This goes to team capability, but also to an understanding about integration.
Again, solvable problem based on the team capacity. If this suggests a unique
code stack that's not already available elsewhere, I'd need to understand the
justification. Project management 101.

    
    
      - provision server space and servers

This implies the technical operations of our agency are fully-loaded, or can't
address this in a reasonable timeframe, or purchasing requires some inordinate
amount of lead time, or some other unknown reason. If the suggestion is that
dependencies exist that threaten the timeline, those dependencies should be
mitigated. Again, project management 101.

    
    
      - deploy

This is a physical step of pushing bits live, so automation tends to make this
a quick step. In my project, we thought about deployment actions when we
determined how to devise our API. At this point, deployment is an operational
aspect -- not a how-do-we-do-this function. This project doesn't proceed
without understanding this step.

Disclaimer: I've actually done this type of project work for significant
operations of size and complexity. I've also worked with a few federal
agencies, so I have some familiarity with the lay of the land.

It's really not that daunting of a project. I would respectfully suggest that
you re-consider your own presumptions, as there are a lot of ways to address
this type of work.

------
pwg
This seems quite relevant now: <http://xkcd.com/927/>

What will this bring? Well, the US govt has X agencies. The result of this
decree will be that, within 12 months, all of the public will get to enjoy the
thrills of having X incompatible web API's, one unique one per agency.

~~~
pmb
Better than it is now, where none of that data is available except possibly
through FOIA lawsuits.

~~~
jlarocco
That hasn't been true in my experience.

What data are you looking for that isn't available?

The only thing I can think of that's difficult to get is some law enforcement,
military and "national security" related information, but APIs aren't going to
change that.

I don't have a ton of experience with it (mostly USGS and NOAA), but the
Department of Commerce and the Department of Interior both make a _ton_ of
data available.

------
EricR23
This reminds me of the push here in NYC for all of the city agencies to open
their data via an API. It's gotten better over time, but when the initiative
first took flight, it was terrible. Some of the APIs flat out did not work,
and the ones that did often returned all sorts of malformed, non-normalized
data. It was a nightmare to work with. I'm curious if the government can do
better.

------
codeonfire
This is kind of interesting because maybe it shows how far the thinking is
from technology right now. I can't wait to integrate FBI files into my web
app, and maybe I can bypass 'authorized e-file providers' to file my taxes.
Maybe I can download daily spy satellite imagery. My point is that what is
already meant to be available is probably already available.

Decision makers are often excited about technology but don't really get the
ground level experience. They want to do all the things...on a roadmap...with
milestones. Mobile has to be involved in some way.

------
bmelton
This will likely go down the same way the original IPV6 mandate went down,
before it was postponed, and before it likely will be postponed again when
nobody's met the mandate.

The issue is far more complicated than the comments I see in here are giving
credit for. Don't get me wrong, there's going to be delay as the PHBs get
themselves wrapped around what an API even _is_ , but they'll have the
directive routed to their CIOs before that, and they will understand the
requirement, and how impossible it is.

The biggest issue is that the data isn't really owned by the government
entities. I mean, the data is theirs, but it's locked up in their vendor
provided tools, and/or their custom, built-by-vendor products. If they're
using Oracle AquaLogic (or whatever it is now) to host the majority of their
portal content, they're dependent on Oracle to either come in and show them
how to implement the feature (which is a significant service dollar cost) or
they're going to have to wait until Oracle builds the ability for API exposure
into the product if it doesn't exist yet.

If they've got custom-built portals, they'll need to consult with the vendors
who wrote them or maintain them now and get them to add that in. That means
that they'll have to modify the contract originally bid for the project, which
is going to eat up a couple months of the timeline alone. Then they'll have to
figure out what sort of things actually make it into the API, how to segment
sensitive data reliably, get it through ISSO testing, etc. It's almost
impossible for a project of any significance.

On top of that, they'll have to do it with a budget they don't have, and with
resources allocated elsewhere. The only way the government really gets
_anything_ done is by committing large amounts of resources to it in an
uninterrupted fashion. They don't have the capacity to be agile, and to some
extent, that's by design.

~~~
intended
Its going to be a pretty huge undertaking.

I assume that this will lead to some discussion on API standards, as multiple
agencies simultaneously realize the implications.

------
DanielBMarkham
15 years ago I contracted with several large federal agencies. Back then, we
were pushing for the same thing. It never flew.

I imagine after 15 years they may have a chance at this, but I would caution
those of you who have never worked in huge government IT shops to take this
with a grain of salt. The situation is so bad in many places that Congress has
been passing laws making it illegal for the federal systems not to behave in a
certain way. And still things are broken. We passed the point of desperation
many years ago.

Big IT in general is broken, and government IT is the most dysfunctional of
any IT on the planet. I remain hopeful that this executive order can
accomplish something, but I'm not holding my breath on it. Hopeful is one
thing. Excited like this guy is? Not at all. Maybe in another 15 years. Maybe.

------
gresrun
I'd be interested in the NSA's API...

~~~
ConstantineXVI
Just a new HTTP code for the other services to implement:

    
    
        312 Pay No Attention To The Proxy

------
stcredzero
In enterprises where departmental data has been opened up through APIs, like
Wells Fargo and Amazon, there have been tremendous benefits.

In the case of Amazon, this was achieved by CEO fiat, and strongly tied to
employee evaluation. (To the point where employees in groups that failed to do
so would have been evaluated right out of the company.) I wonder if POTUS has
this kind of power over the federal bureaucracy.

Also, I would wonder if this is to be done securely.

------
djKianoosh
As a programmer/contractor for DHS, I'd love to hear what you all think would
be a useful set of APIs for DHS to make public. It's all fine and dandy to say
'oh yeah we have an API' but it needs to be something useful. So what would
you want to see? Financial/Budget type data? Performance metrics across the
different Components within the Department? What?

~~~
malyk
How about TSA budget, # of TSA employees, # of terror plots stopped, some
ratios between those things, # of nail clippers confiscated per employee, etc.

;)

------
ColinWright
<http://news.ycombinator.com/item?id=4055622>

~~~
mcrider
Weird, I searched for the URL and also thought HN would automatically detect a
dupe. Didn't mean to hijack a post!

~~~
ColinWright
The URLs aren't identical, one has "index.php" at the end, the other doesn't:

yada yada ... barak-obama-directs-all-federal-agencies-to-have-an-api/

yada yada ... barak-obama-directs-all-federal-agencies-to-have-an-
api/index.php

With regards searches, it depends on what you search for. HN_Search is not
always obvious, and it doesn't index minute by minute. You may have done the
search before the engine caught up.

------
domwood
In rudimentary terms, I suppose at least it's a step in the right direction.
It doesn't necessarily suggest the US government is going to immediately
embrace openly disseminating its data, but it's still a step in the right
direction. Third party services will most likely proliferate quite rapidly.

------
gregors
It'll be interesting if they decide to use NIEM (National Information Exchange
Model) as a way to transfer information to the public as well.
<https://www.niem.gov/Pages/default.aspx>

------
sidwyn
The first thing that caught my eye was the misspelling of Barack in the
address :)

------
jakejake
It's hard to imagine some federal agencies being able to do much of anything
within 90 days. But, I look forward to poking around with some of these APIs.

------
cjoh
Wish they'd done some reasonable amount of procurement reform before this. All
this means is another big payday for gov contractors

~~~
isamuel
The two issues are completely orthogonal.

~~~
cjoh
Actually, no, they're not. They're tied at the waist. Governments antiquated
technology acquisition schedules mean that govt cannot tie itself to Moore's
law. Because currently government buys from an exclusive system rather than an
inclusive one, it means the tech they do end up getting, often 18 months late,
isn't very good, and is very hard to upgrade.

Thus a mandate that "all agencies should do x" with tech, before reforms in
the acquisition arena create severe problems down the road, and only really
benefit the contractors who make them.

By the time these Apis really see the light of day, we will be complaining
about them.

------
wickedchicken
from dod import air_force

air_force.launch({"f22": 3, "b2": 4})

~~~
mrpollo
hope we can apply for H-1B similarly real soon

------
imrehg
Get ready for some hackathon! I'm sure there are a lot of useful and
interesting ideas to build with whatever comes up.

------
moomin
Has he been reading Steve Yegge's redacted post?

------
CUR10US
API's? C'mon.

Bulk data.

Agree with polemic.

------
Treisfeo
It's called a Web Service noobs!

~~~
solnyshok
API is much better than nothing. I say, begin with API, then require any API
to have a basic "raw_data_dump (start date, end date)". And yes, this isn't
about transparency, but it is more about efficiency, and still can lead to
transparency.

------
anaheim
Bloomberg "learning to code", Obama directing agencies to have an API.

All part of politicians (particularly Democrats) trying to look like they have
a clue. Give up already for heavens sake and get back to managing the deficit.

