

Ask HN: Review My Doohicky - ajkirwin

I got bored this weekend and decided I wanted to data mine twitter and over the course of a day and a half, I made this:<p>http://malicelabs.com/linkalyzer/<p>and I would like suggestions from people as to how to improve it and what they think of it. :)<p>Edit: Intermittently broke things and blew away some data intentionally. Add(ing|ed) in tinyurl/is.gd/etc through-finding of real URLs and page title grabbing.
======
nirmal
Would be good to expand the tinyurls or at least provide the domain they are
pointing too. This may even help you consolidate tinyurls if multiple services
are used for the same original url. In Python I sometimes do the following to
get the finalurl (not fool-proof):

    
    
      finalUrl = urllib.urlopen("http://www.tinyurl.com/5w5wod").geturl()

~~~
ajkirwin
I added this. A bunch of people demanded it. :)

As it stands, it has a few things that twitturly doesn't have, but also they
have things I don't.

But, it's a beta product and hopefully soon, I can get a slicehost/linode
server and get it off of dreamhost and I can spruce it up and add more
features.

------
riklomas
Maybe try and reverse engineer the TinyURLs so that users can see where they
link to?

~~~
iamdave
This. Or maybe find a way to grab the page <title> from the target URL and
create a separate column with the appropriate information.

Otherwise, nifty little app.

------
physcab
Nice project. It would be interesting to see maybe a "popularity tree". Like
for example, it would be fun to see if Scoble or Kawasaki put out the link
first and see how it was distributed. Or if someone unknown sent out the link
and someone popular picked it up. Not sure if its possible to trace though.

Also, I just started a topic on a data mining related note. If you have any
thoughts, I'd love to read them. <http://news.ycombinator.com/item?id=406705>

------
wesley
Nice little project, good for learning the API.

You may want to check out <http://twitturly.com/> if you haven't yet.

------
braindead_in
view more information gives an warning.

Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result
resource in
/home/.dodi/jamesgarrett/malicelabs.com/linkalyzer/get_urlstats3.php on line 7

more warnings follow

~~~
wesley
Always always always turn off error display_errors on a production server :)

------
nirmal
Also, every image in your TwitPic section appears once as a png and again as a
jpg. Some of these are broken but they still link back to the same TwitPic
page.

------
jackowayed
Twitpic Hot 30 gives is missing about half of the images. There's spaces for
them, but they just show the alt text.

------
gsmaverick
Neat analyzer! I think there is so much data that can be gathered from Twitter
that we have no idea exists.

