

A Personal API - dannyr
http://x.naveen.com/post/51808692792/a-personal-api

======
psycr
I have been working on an open source version of this idea:
<http://www.quantify.io>

Right now, things are fairly sparse. You can check the repo out here:
<https://github.com/jdjkelly/bodyimage>

_An open source Rails API to track "quantified self" data.

Right now, the hardware companies in this space are storing your data in
closed systems. Let's change that.

Take the Withings WiFi scale or the FitBit, for example. Both are essential
pieces of the quantified self hacker, yet both store your data in their
respective, siloed databases. What happens when one of them goes bust? Or
decides to pivot towards an ad-based business model?

This project includes a Rails + MySQL RESTful API to store your data.`_

Contact in profile.

~~~
naveenium
this goes very deep; i like it. i'll drop you a note soon.

~~~
dawson
Here's another one <http://api.howareyou.com> (I'm the founder)

------
danielsiders
This is one of the use cases we had in mind when in developing Tent
(<https://tent.io>).

Tent is an evented data store for arbitrary JSON blobs which can be typed b
the developer. It has the added benefit of operating as a decentralized system
that lets you push and receive posts with other users via webhooks.

Tent can be used to create applications and systems from the quantified self
to microblogging to cloud-backed file sync to collaborative real-time document
editing.

------
spdy
One day we will own our own node. Where we have control over everything, what
we want to share with whom and why.

All we need is a place for storage and we add services like apps. With ipv6
around the corner everyone will get his personal ip attached anyway so we can
build upon that.

Many of us have this idea but no one made a breakthrough so far but it will
emerge. We have not figured it out quite yet we have the technologie but we
need to get the idea of how we glue everything together.

I will look forward to the day when i pay a certain amount of money per year
to have my own node without being the product of some megacorp thats only
interested in quarter performance.

~~~
amirmc
I'm working on exactly the kind of thing you're describing. Except ipv6 isn't
really going to solve the connectivity problem as all the NATs and middleboxes
on the network aren't going anywhere. We'll be using DNS instead as the
'identity' piece.

Since you said you would pay for such a thing, can you elaborate how much and
what you'd expect (in terms of functionality)?

~~~
spdy
I would pay for storage videos/pictures/documents etc. This could be one
monetization stone. My gut feeling maybe around $9.99 / month or $50 / year.

The basis everyone would except are normal social network features sharing of
"items" galleries friend system feeds and groups. Maybe this part should be an
open source solution like for example diaspora.

On top of that you can build an app/service network. Maybe something like
app.net.

Important on all parts is that you own the data and you should be able to pull
it at will from any part of the network.

------
aridiculous
The prisoner forging his own chains.

~~~
quantumpotato_
Amusing ourselves to death in the 21st century.. this technology does raise,
once again, the question of why people are saying so much personal information
online. As if they had a giant ampitheatre.

------
ngoel36
This is so cool. Somebody should turn this into a platform - a mobile app that
at least helps you record the stats you want, paired with an API platform that
lets you grant and request permission for other "personal APIs".

Then other API calls can be built on top of individual people - College
students in the US, citizens of India, etc. The possibilities are pretty
endless.

~~~
jpreiland
The Biggest Loser - play along at home! Be the "biggest loser" in [your
region] to [acquire swag], etc.

Sorry, maybe I'm being a bit facetious. There's definitely a lot of cool stuff
that could be done with a personal API though. But how do you get people on-
board? You have to have equipment to track/update stats right? Are people
willing to buy into this potential network?

~~~
ianstormtaylor
If Fitbit and all of those types of companies start having Export APIs a
service could have wrapper around all of them. Then anyone with one of those
devices could join easily. Could be the way to start, since you definitely
don't want to be building the hardware and API altogether for an MVP.

------
checkoutmygenes
Nice! A few similar services come to mind that might be of interest:
<http://lockerproject.org/> <http://humanapi.co/>
<http://developer.runkeeper.com/healthgraph/>

Get your DNA in there too! <https://api.23andme.com/>

------
realrocker
Interesting. Not the api part but the idea itself. What we need is bitcoin
like encrypted decentralized personal file with an open spec. This file can be
edited only by the owner while can be read by anyone. Then we need exporter
applications which can read this data and provide as an api. Now there are two
levels of abstraction:data owners and data providers. There can be "data
unions", groups which can negotiate with data providers and data consumers for
a better deal(money,privacy). Identity verification can now be handled by data
unions instead of data providers(as is the case now). "Take control" anyone?

------
jtchang
I like the idea of this. I know there was a post recently about a guy selling
24 hour chunks of data about himself.

Is there an opportunity here to create a secure data store for your own
structured data hosted securely somewhere else and that providers would pay
YOU to access?

Of course you'd give free access to it for your doctor who needed to pull some
info. But what if you had total control over your own API. This is a kind of
like Facebook but I would never let facebook hold some of my most personal
data. I would like to control authorization to my own data though to certain
people.

------
nemesis1637
My first thought was "this is about as narcissist as you can get". But, it's
still pretty damn cool. Nice job.

~~~
stinkytaco
Interesting take. I looked at it almost the opposite way: "You are turning
yourself into a set of numbers that has no identity outside of those numbers."
Certainly if this were to be a thing (a la online charts or Microsoft Health)
it would have that end result.

------
hawkharris
I enjoyed the post very much. Just out of curiosity, though, why did you
choose not to capitalize any words?

~~~
brvs
Because he's tuned his body and mind to machine-like efficiency and calculated
that hitting the shift key is an unnecessary ancient relic of the past.

Basically he's disrupting capitalization, which I've been thinking about a lot
lately.

------
parm289
I actually started hacking on this idea a few weeks ago. A personal API could
have tons of uses. Many others have commented on the benefits of aggregation,
but this could allow users to control and selectively sell/provide information
to companies, apps, and advertisers. Applications could be built which utilize
the personal APIs of friends. In Naveen's example, my app could compare my
fitness level with my friends, recommend new restaurants based on ones my
friends actually went to, track progress on goals, etc.

What I'm trying to say is that with a personal API platform, apps simply
become a front end to standardized personal data. Switching costs would be
reduced significantly, which would encourage a lot more innovation.

Here's a spec I've written out for my work on my own personal API. It's not
done yet, and the code isn't open sourced yet, but hopefully it will inspire
others to hack on the idea.

<https://dl.dropboxusercontent.com/u/570656/spec_public.md>

------
groby_b
I've been spending a bit of time thinking about this lately, too. Here are the
things that I think need to be answered for this to be truly useful.

* Secure access. It's great you decided to share everything publicly. I'm looking at fairly confidential data like blood tests, though.

* How do you express permissions? It's likely you'd give out sets of data, not per-url permissions. (I.e. my photos from date A to date B, in this general vicinity - your vacation pics)

* How can others discover what data sets & types you've shared?

* Related: A type description. Some events cover time ranges. Some are momentary samples. They all use different units. Some have location info attached, some don't. (I.e. my heart rate doesn't. Pictures do. Steps/day - geolocation is meaningless)

Either way, it's exciting to see people push ahead on this - many of these
questions will probably be answered as we go.

~~~
naveenium
you're totally right on all these points: for the time being, i've only put
out information that's okay being 'public', but i imagine when you do want to
share a set with a small group (let's say your physicians), you'll want an
additional layer of access control.

~~~
groby_b
Oh, I didn't mean it as criticism. I admire that you went ahead and just
created a v0 - that's further than I've gotten on my project, that's for sure
:)

------
joshomh
I work for a non-profit called Open mHealth. We are working to build an open
architecture around health data from mobile apps and devices. The basic idea
is that existing applications can have their own proprietary way of getting
data into their systems, and we want to evolve a standard way of getting data
out of these systems. We are developing an API specification around this.
<https://github.com/openmhealth/developer/wiki>

Once app developers adopt the Open mHealth approach, much of the work of
getting data out of these silo'ed apps will be done. This is important because
the real value of this data comes about when the streams are combined to be
analyzed and visualized.

I am posting here because many of the issues outlined in this thread and in
the original post are things we are actively trying to solve.

------
revorad
This is awesome. I've also been working on a personal data dashboard app,
which is similar. It pulls in data from various social and fitness app APIs.
You can also enter in data manually. It's mainly for viewing the data and
trends, but I'll add an API too if there's demand.

Is anyone interested in using my dashboard app?

~~~
taylorwc
yes - want to explore

~~~
revorad
Great, can you please drop me an email (see profile) with some details on what
data you're interested in tracking?

------
kmax12
Such an api sort of already exists in the form of Fitbit's api which can be
found at <http://dev.fitbit.com/>.

Fitbit holds a lot of the data hostage, but it's at least something that
anyone wearing a Fitbit can easily grant access to.

~~~
whazzmaster
What data do they hold hostage?

~~~
randomchars
It was on HN[0] a few months ago that they don't give you access to minute by
minute data, only daily totals.

[0]: <https://news.ycombinator.com/item?id=5001301>

~~~
whazzmaster
Ah; FYI the API now supports Intraday Time Series fetching.

See [https://wiki.fitbit.com/display/API/API-Get-Intraday-Time-
Se...](https://wiki.fitbit.com/display/API/API-Get-Intraday-Time-Series) for
more information. You can retrieve data in 1 min or 15 min increments.

~~~
kmax12
you still cannot get access to the raw data or data in real time.

------
tectonic
This is also something that I hope Huginn can tackle. Huginn makes it easy to
retrieve and react to your data. Adding an API beyond that should be
straightforward.

<https://github.com/cantino/huginn>

------
dreamfactory
Kind of interesting to think about slowly building up and aggregating more
layers - e.g. activity, location, shopping, internet, phone, financial,
educational, medical, legal, work, relationships etc - and interesting how
much of this exists already.

So if this comprehensive personal dataset becomes the norm at some point in
the future - what happens when this is under somebody else's control against
your interests (whether an illegal attacker or a legal authority)? And what
about more gray areas, such as marketing, insurance etc (which would likely
encourage the creation of that kind of API in the first place)?

------
ace_of_spades
Had a similar idea before but looked at it as more of a decentralized social
platform where everyone hosts his own node with all personal information
(maybe even locally on a personal device). Kind of like a virtual avatar that
could connect and create a real social layer for the web, one that we could
control and trust. Technically this should be pretty easy to create, nothing
to fancy here. However standardizing and major real world adoption seems to be
pipe dream (look at all the failed attempts like diaspora, etc.).

------
notlisted
Love the idea. If you wish to incorporate social stream data, check out
<http://socialsafe.net>

I've used it to back up and then delete my personal FB account, and continue
to archive instagram, linkedin, twitter, blogs, etc. etc.

What's more, even though their interface is kinda weird, they store all kinds
of granular data in a very nice SQL database which contains more then they
represent in their reports, and can be be repurposed in many different ways...

------
clockwerx
The FOAF crowd already did a lot of this style of work - see also
<http://wiki.foaf-project.org/w/DataSources> ; you probably have a few data
feeds already, particularly focused on social networking site activity. Other
vocabs work - ie, crschmidt's menow vocab.

Express it as json ld to make it more modern, and you have a distributed
ecology of services that all talk the same sort of language, about you.

------
jckt
Interesting. I think the real challenge behind this though, is not the API
itself, but that for it to be really useful there has to be some sort of
_semi-automated_ form of data collection, that can be generally applied.

Relevant: [http://blog.stephenwolfram.com/2012/03/the-personal-
analytic...](http://blog.stephenwolfram.com/2012/03/the-personal-analytics-of-
my-life/)

~~~
naveenium
you're totally right

this is why i believe a smart scale is one of the best examples of good data
tracking: you step on it whenever you think of it (not on some weird schedule;
nor do you have to wear it or take it with you). it does the mundane stuff of
capturing the data and sending it to the cloud, which you would otherwise have
to write into some notebook.

if it's a system that's fully automated though, one might lose interest in
collecting it in the first place: the QSers always like to say – when you add
a little friction to a process, you end up paying more attention to why it is.
so instead of just blinding capturing your weight, setting up alerts for goals
and seeing trends fall and rise will get you to make the most of the data –
instead of just collecting data for data's sake.

~~~
dreeves
Ooh, we have a solution to that problem (losing interest in collecting it if
it's fully automated). Assuming you're collecting data that you want to
change, like, for many people, their weight, or, say, how much time you spend
on Hacker News: <http://beeminder.com/d/hn>

Namely, set up a commitment device that forces you to keep your datapoints on
a path to some goal. That's what Beeminder does.

------
stefanobernardi
Very cool.

I had actually mocked up something similar with openapi.me, but never got to
release it.

It included different level of permissions and data including addresses and
payment information. The goal was to have a centralized repo of data to update
only once and have it propagate to all the other services asking for the same
data over and over (email, address, links, etc).

------
CodeCube
This may be a bit of a privacy issue to publish, but an interesting dataset is
that of <https://openpaths.cc/> ... keep the app running on your phone and it
will keep track of your location. This data is then available to you over an
API, or to researchers that you approve to get access to your anonymized data.

------
cocoapriest
As an engineer, I can understand the excitement. But I wonder why and who
should use that data? I can't see any great use cases.

~~~
tannerc
Doctors, for checking activity/sleep/etc.

Friends/relationships, for checking general activity and check-ins.

Potential businesses, for seeing interests and the like. Or, if currently
employed, the employer could randomly check to see how the employee has been
sleeping (are they going to be tired/cranky today?) or to see how often they
check-in at the office.

Aggregated with more data from people doing similar things and you can start
to get a look at populations (what if everyone in SF was logging this data?
What if a good chunk of those in the US were?).

------
tejaswiy
I've actually been toying with the idea of having a private facebook account
to hold all this data and more. Facebook's pretty much become the default
social layer these days and almost every app can publish data to your news
feed.

Facebook also has a pretty cool Timeline UI to boot which will probably only
get better as time passes.

~~~
doc-m
I'm starting to think this is a really great idea - to have somewhere to keep
a private personal history - particularly for checking in to keep a track of
where I've been without appearing to be a self-important moron. You know who
you are FB friends!

------
crapshoot101
I like the approach you're taking Naveen. I do wonder whether individual,
disparate data steams have any value, or whether its a case of the sum of the
parts is less than the whole, so to speak (along with concerns like security,
etc etc that others have raised), but the concept is intriguing. Will be
watching / following.

------
Yhippa
If there was an easy way to collect and publish this data for the average
person would you be willing to sell this to companies directly? We're
basically paying companies like Facebook and Twitter with our information in
order to have, well, Facebook and Twitter. Maybe we could do something else
with our "money".

------
kcthota
Nice! I like the idea of keeping all the data about _me_ at one place. Given
that so many companies are entering the wearable gadgets and personal fitness
space, we will surely see some "Personal Portal" and the company making tons
of money selling your personal data or showing "relevant" ads for you.

------
dtwwtd
I've had a similar idea for a while but never executed it well. I wanted to
have my personal website merge all of my various feeds from other services
into posts on my site to be the "one feed" of things happening in my life.

I suppose API access to that would be a logical extension - this is very cool.

------
lnanek2
Reminds me of 2net by Qualcomm, except that has a lot of features and uptake
already: <http://www.qualcommlife.com/>

At CES, Qualcomm had a bunch of devices working with it through a plugin wall
unit. They offer free data APIs for storage, etc..

------
nkozyra
I've thought about this many times before. But, this is the easy part.

Storing, doing simple metrics, making assumptions and returning the data is
not a problem. Standardizing input methods is.

~~~
reeses
Federating data is a fairly 'solved' issue. Ironically (or not) enterprise IT
is ahead of the average organism in this space.

Faced with legacy systems communicating (or not communicating) via various
channels, Middleware, ESBs, SOA, message queues, content-based routers,
business process management, event processing, and data virtualization have
all been created so that a development group can factor these systems into
canonical representations of business events.

At that point, the input method ranges from the simple (existing web app to
API) to complex (treating a batch-oriented system in a semi-synchronous manner
or screen-scraping an ancient smart terminal as a queriable data source).

At that point, the real challenge isn't the input method but master data
management and security/authority/privilege.

------
thorntonbf
That's pretty cool!

As an engineer, I crave data. As much as anything, this process would force me
to organize that data into some archivable, more useful form.

------
EternalFury
Cool idea, Naveen. I would like to contribute and expand it.

~~~
iusable
Would love to show you a working example of a similar project with a public
API. We have a number of users giving it a spin and would really appreciate
the technical feedback. Email?

------
misiti3780
Similar to this post: <https://news.ycombinator.com/item?id=5021783> (minus
the act API)

------
bbenko
I'm also working on something similar as my weekend project. It's called
Pokelog. <https://pokelog.com/>

------
neeharc
As you suggested, I think this is a great way to expose your data to apps that
may be built on top of you.

Also, a stalker would love this :P

------
banders
This is great! I would love to hear some device recommendations. Fitbit,
Fuelband, or Jawbone? Withings scale or fitbit scale?

~~~
naveenium
i own them all and have tried each one on and off for the last couple of
years. i like each one in its own way (and, also, am friends with guys at each
place)

if you're just getting started and you aim to keep it light, then i'd start
with the withings scale. you leave it in your bathroom, set your goals, step
on it when you feel like it and watch trends and alerts from time to time. i
find my weight tells me more about me over time than many other metrics: it
shows my bad weeks from my good and, in indirect ways, shows when i stress out
and get bad sleep (and therefore, end up eating poorly and exercising
inconsistently)

------
question-all
Is it that all your personal information is in one place that makes this so
unique or is it the enormity of the data?

------
platz
Please include the units in the results!

~~~
naveenium
a good idea. for now, i left everything metric, but have thought about
dropping in the units.

------
mengg
<http://www.personal.com/>

------
dale386
This website was blocked by my company's firewall as "suspicious."

~~~
crisnoble
maybe because its a tumblr site? You must work at a strict company, there is
nothing NSFW in the article or the site.

~~~
dale386
They're pretty lenient -- there are all sorts of awful websites that I'm able
to visit without issue. They only tend to block sites related to hacking and
piracy.

------
tn13
That is a brilliant idea.

------
abraininavat
I was about to dismiss this but have thought about it a bit more, and there's
something interesting here.

There are already APIs built on top of you. There are the ones you know about,
like Facebook and Twitter. There are the ones we know exist but have no direct
access to, such as Amazon's backend or your police record. And there are the
ones we may never know about, such as the NSA bot scanning this post right now
and correlating it to my recent overseas cell phone calls.

I'm intrigued by the concept of aggregating this data. Imagine if the wealth
of data collected about you all went to one place, and, most importantly, was
_controlled by you_. Imagine if Company X had to pay _you_ to know about your
Amazon spending habits instead of paying Amazon. Imagine if scientists could
perform long-term health studies in an instant because the data they need is
already collected and just needs to be crunched.

I think the concept that's missing from your blog post is the idea of
_discoverability_.

~~~
rasur
Have you read Jaron Laniers new book "Who Owns the Future"? You have similar
ideas ("Imagine if the wealth of data collected about you all went to one
place, and, most importantly, was controlled by you. Imagine if Company X had
to pay you to know about your Amazon spending habits instead of paying
Amazon.")

It's a thought provoking book.. and slightly worrisome to boot (about the
state of things at the moment, with regard to the economics of "sharing
data").

~~~
naveenium
yes, i do think of it that way: as we leave all this data exhaust behind us, i
keep wanting a system to have it all in one place – or at least, an
"interface" that shows it to me in one place, even if that's not how the
backend really is.

thanks for the book recommendation - i will check it out this weekend!

~~~
amirmc
Academia has been thinking about this for a while. I'm involved with several
projects that aim to put users in control of their 'digital footprint' and
what you've described are two aspects of the work. First is how to collate and
access all the data and second is how to allow third parties to query/process
that data in a way that respects a users' privacy. [1]

I'm keen to commercialize the work when it's ready but it's a research project
precisely because it's a tough problem. Of course, one approach is to just
make that personal info public but I don't think that will really benefit
users in the long term (companies would certainly benefit, though)

[1] <http://perscon.net/overview/dataware.html>

------
Dirlewanger
Narcissism reaches a new level (or low, depending on how you see it).

~~~
randomchars
Narcissism:

1\. Excessive or erotic interest in oneself and one's physical appearance

2\. Extreme selfishness, with a grandiose view of one's own talents and a
craving for admiration

So explain to me; what does quantified self have to do with narcissism?

~~~
pknight
Well in defense of this snarky comment, I have to say that the social sharing
part of personal data like this can come across in this way (doesn't make it
narcissism but still...).

The thing I like about personal data collection and cool new apis is what it
could do for the individual, i.e. offer new insight, movitivation, encourage
better behaviours.

But I hate the sharing part because it opens the door to over-sharing. Not
just in terms of corporations having all kinds of data and using it in their
interests, but also just simply the noise that gets added by people sharing
information that frankly, has very little relevance.

I don't really care if someone just completed a 2 mile run and I certainly
don't go to twitter to read automatically generated tweets about it.

