

Show HN: Twitter graph monitoring over time, SocialGrapple - shazow

Ever wanted to track who unfollowed you last month? Or who followed your competitor last month? Or how your social graph is growing over time? SocialGrapple does this.<p>I've been working on SocialGrapple for a few months (since stepping down from imoveyou.com, YC2010S), this week I got the core product to the point where I'm proud of it so I want to show it off: http://socialgrapple.com/<p>I wrote up a blog post with some screenshots that will get you an idea of what SocialGrapple looks like:<p>"SocialGrapple snapshot: @limedaring's social graph growth after her HackerNews publicity"
http://shazow.posterous.com/socialgrapple-snapshot-limedarings-social-gra<p>Screenshots:
http://i.imgur.com/OzTIb.png
http://i.imgur.com/tghcT.png<p>The gist of it is: I pull the account's social graph every night (soon to be every hour) and do a diff from the last known state, then create an event log based from that. Then I do a bunch of graph operations like intersection between following/followers to determine stalkers/stalking/mutual, annotate the feed with metadata useful for skimming/ordering/filtering, intersecting two accounts to see how growth relates to somebody else, etc.<p>Backstory: I built Tweepsect some years ago (http://tweepsect.com/) which lets you intersect your followers/following as it currently is. It's still one of my most-used projects, and a lot of people were asking for more like support for larger accounts, data over time, charts (a lot of people use Webnumbr to track their follower count over time), track who unfollowed them (qwitter and similar stopped working months ago). This is where SocialGrapple comes in, with an angle on competitive intelligence too.<p>I would love some feedback. What would it take to get you as a paying customer?
======
shazow
Clickable links –

Blog post: [http://shazow.posterous.com/socialgrapple-snapshot-
limedarin...](http://shazow.posterous.com/socialgrapple-snapshot-limedarings-
social-gra)

SocialGrapple: <http://socialgrapple.com/> (Make sure you have a look at the
blog post first)

Screenshots: <http://i.imgur.com/OzTIb.png> <http://i.imgur.com/tghcT.png>

------
ninjastar99
This is great and something I see as an immensely valuable tool for social
measurement. It'd be nice to integrate Facebook into the mix as well, showing
fan counts and competitive analysis.

As a potential customer, I see this as valuable as a free resource for basic
insight, but a paid monthly service for more extensive analysis.

~~~
shazow
I've been looking at Facebook and originally all my language was service-
agnostic in the event that I toss Facebook in there, but now I'm kind of
backing away from it.

1\. Facebook doesn't give good access to the more granular stuff, mostly just
superficial counts. Especially if you don't own the page/account. 2\. Facebook
already does pretty decent analytics. Is there a big thing they're missing?

------
limedaring
<http://i.imgur.com/OzTIb.png>

This screenshot is why I really love using SocialGrapple for my account – in
the quick overview of what happened yesterday to my account, you can see that
I followed one person who followed me that day ("mutual" tag on it), and who
the most important accounts are by follower count (x2 means twice as many
followers as following, and a better ratio usually means a more important
person).

I love Twitter and using it to meet new people, but when 100+ people follow me
in one day, it was really awesome to have SG to help me parse the data.

</unsolicited review>

------
nym
Got this error when trying to sign up for tweepsect:

Traceback (most recent call last): File
"/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py",
line 515, in __call__ handler.get(*groups) File
"/base/data/home/apps/jsoauth/2.346920482575371317/main.py", line 127, in get
token, secret = client.get_credentials(oauth_token, oauth_secret,
oauth_verifier) File
"/base/data/home/apps/jsoauth/2.346920482575371317/oauth.py", line 150, in
get_credentials return self._extract_credentials(response) File
"/base/data/home/apps/jsoauth/2.346920482575371317/oauth.py", line 165, in
_extract_credentials raise OAuthException("Problem talking to the service")
OAuthException: Problem talking to the service

~~~
shazow
Hrm this happens sometimes when the Twitter API fails in a weird way. Try
again?

One of the cool things of Tweepsect is it runs 99% in-browser JS (including
all the API calls). There's only a small piece of the OAuth that had to be
done server-side which lives on AppEngine. I need to add better error handling
to that someday. :)

------
Tichy
Just curious: how do you deal with the increasing limits of the Twitter API? I
noticed you don't require me to log in with my Twitter account (which is
great), so it seems you are not using user's request quotas, but your own.

~~~
shazow
Indeed, I tried to make the experience as non-intrusive as possible. One of
the biggest complaints from Tweepsect was that people didn't want to OAuth to
try it out. Especially if I play the angle of "use SocialGrapple for corporate
intelligence" I can imagine users may be weary to give away their true
credentials, which is fine.

I applied to have my @SocialGrapple account whitelisted 6+ months ago (a bit
pre-emptive, but not enough), but they never got back to me and then last
month they announced there will be no more whitelisting. This sucks.

Right now I just try to use the API as efficiently as possible and spread jobs
out to avoid hitting the limits. In the future, I might need to distribute the
load across multiple auths or request the user to sign in.

I tried to make the pricing in-line with the parts that are "expensive" to me,
both in terms of limited API calls and data storage/performance. I hope this
is transparent enough to the users and doesn't get in the way.

I have looked at using gnip (Twitter's official bulk data reseller) but the
pricing structure is way out of my league (starting at $500/mo). Can't blame
them though, since they dedicate an EC2 instance to each customer. Pretty cool
scaling model.

------
JED3
(From the superficial complaint dept.):

I assume that your using <http://html5boilerplate.com/> to some extent, along
with the default favicon and icon.png that it ships with. I'd recommend
replacing that with an original icon or at least one that's less identifiable.
Great project nonetheless.

~~~
shazow
Ooof, you're right. Good catch! Keep em' coming, making a list.

------
mnickel
Would you consider sharing the technology stack that you used to create this
application?

I love the visuals. How perfect for a brand manager to show that doing X can
result in Y.

Thank you!

~~~
shazow
The stack is Pylons (behind uwsgi+nginx), PostgreSQL, and my own task
scheduler[1]. Right now it's hosted on Linode. Will be moving to DotCloud
eventually. :)

[1] <http://github.com/shazow/turnip>

~~~
mnickel
Thank you for the details!!!

------
tsmith
I really like the followers-over-time graph.

Question: is there a way to embed the graph into another "dashboarding" tool?
I.e. through an API or even a simple URL?

~~~
shazow
Data export via API is on the list. It'll be done as soon as someone actually
needs it. :D

------
johns
I'm hesitant to pay since I can't see any examples of what I'd get. A short
trial would be great.

~~~
shazow
Yea, you're right. Check out the link to the blog post? That has a pretty good
insight on what the premium meat looks like. I'm working on better non-user
facing stuff as I write this. :) Takatakataka

~~~
johns
I got distracted after the first link to the product :) This is right up my
alley, so I'm definitely going to try things out and hopefully have good
reasons to pay.

~~~
shazow
Sweet! Send me a note with what I need to do to get you as a customer. :) My
email is in the footer when you're logged in.

------
thomasdavis
Wow this tool is just great and something I've always wanted.

Will be using it extensively for poaching clients from competitors. Also will
love to see the effectiveness of competitors and my PR campaigns. Can't wait!

I have already signed up for a premium account!

