

A warning to hackers: be careful building on Twitter's API - marketer
http://hoisie.com/post/a_warning_to_hackers_be_careful_building_on_twitters_api

======
davidu
My platform is called the Internet.

I'm still subject to some rules, but a heck of a lot less than the gardens you
guys develop in like Facebook, Twitter and AOL.

I included AOL, because while it seems ridiculous someone would do that today,
people used to until they got horrifically burned, and I think it'll be the
same with non-decentralized platforms like FB and Twitter in the future.

Once again, my platform is the Internet.

~~~
andrewljohnson
My platform is mobile devices. I am happy to develop Android and iOS apps, and
I feel like the companies who run these somewhat walled gardens play mostly
fair. As an added bonus, they have provided me a way to make money.

People who have found their success on the internet at large are my
colleagues, but occasionally they are also arrogant in their views that anyone
who doesn't make an HTML website is an idiot. These people's tunnel vision
will hold them back their whole lives, and this character flaw is likely
evidenced in other views they espouse.

~~~
pyre
You don't have to 'code' HTML to avoid walled gardens.

~~~
andrewljohnson
Usually when you put a word in quotes, you are quoting something. I didn't use
the word 'code' at all.

~~~
knome
<http://en.wikipedia.org/wiki/Scare_quotes>

------
martinkallstrom
I would see this as an opportunity. You have an app that has hit a hard limit
in terms of number of users. It is clearly a useful app, otherwise it wouldnt
be growing.

The natural thing to do is to make it a paid solution. You inform your users
of the situation and say that in order to keep the service up and running you
will start charging a signup fee of $1, the same as you would pay for a mobile
app. Existing users however, get 50% off and only pays $.50. Give them a month
to paypal it in and then another month of repeated notices to inactive users.
After two months there will probably 50k+ users you will have to close out
from the service. But give them an option to restore it easily.

This is not at all unreasonable, no one would ever be able to say so. Above
all it would be a tremendous learning experience for you. Perhaps you also
will make a few thousand dollars in the process, but that would not be the
point.

There are two alternatives to this as I see it. Either you do the above but
the only thing you require is for users to manually report in by clicking a
button in a form. You will still be able to weed out thousands of accounts.

The other is to set up Smart Tweets 2, hosted on another ip, and refer new
users there. Explain why and make it into something funny for the users.

Whatever you do, do something bold enough to make it to Techcrunch a second
time.

------
fingerprinter
They own the data. It is their ball and they can basically say "I'm taking my
ball and going home".

Facebook, twitter, linkedin, google. All of them are in the data business and
sell that data to make a profit. They don't really care about the devs as they
just see the devs as a way to bring people to their ecosystem; the more small
apps people write, the more ways they might be able to get data. But, if they
get big enough, and all of them have, they can cut you out.

I understand building a business around someone else's data (cottage
industry), and I would never say not to do it, but it isn't without peril.
Though, I would do it in a heartbeat for a lifestyle business that I knew I
could pivot on or build another if it failed.

~~~
philwelch
_They own the data. It is their ball and they can basically say "I'm taking my
ball and going home"._

Maybe, but that would make them assholes, especially from the perspective of
the user. _I'm_ trying to use an app to post _my_ tweets to Facebook, and
Twitter is shutting the app down because, as the reasoning goes, my tweets
really belong to them? You might say "that's what you get for using someone
else's service", but by that logic I should just have my own server in the
living room because otherwise Posterous would own all my blog posts and Hacker
News would own all my comments.

~~~
hibrian
Read the terms of service for the sites. Who owns the data and when are they
allowed to turn off? Answer: you "own" it but they can do whatever they want
with your content, and they can turn off the service any time. Boot up that
server in your livingroom if you want your content to live on.

Posterous : however, by submitting material to Posterous you grant Posterous
the irrevocable, fully transferable rights to use, reproduce, distribute,
modify, transmit, prepare derivative works of, display and produce the
material in connection with Posterous and Posterous's business, but solely in
accordance with these Terms of Use and our Privacy Policy.

Posterous is provided 'as-is' with no warranty of any kind. You use the
service at your own risk. Posterous expressly disclaims any warranty, express
or implied, regarding the site or its content, including any implied warranty
of merchantability, fitness for a particular purpose, or non-infringement.

<http://posterous.com/tos>

~~~
philwelch
I know what the terms of service say, and I certainly know the terms of
service allow Twitter to be assholes about this particular issue. They're
still assholes, though, and they're assholes because they're getting in the
way of their own users.

------
jgilliam
Twitter's new official app for the Mac violates several of these display
guidelines.

~~~
alnayyir
Yeah but dey da boss. They get to be as hypocritical and arbitrary as they
like.

If you go sharecroppin' on somebody else's plot of land, don't expect to be
afforded any rights or respect by the owner.

Eventually people will figure this out.

Eventually.

~~~
chime
> Yeah but dey da boss > go sharecroppin'

If this is a reference to something, I don't get it. If not, why are you
writing like this?

Edit: Thanks alnayyir. I knew there was something I was missing.

~~~
JonnieCache
It's a reference to civil war era america. Elaborate user-generated-content
metaphor?

~~~
philwelch
The post Civil War South, actually. Sharecropping wasn't really a thing in the
North, and before and during the Civil War, there wasn't very much
sharecropping because slavery was more cost-effective.

~~~
alnayyir
ding ding ding. This man gets ALL the dollars.

------
jrockway
Obviously Twitter does not want you to copy "their" data to Facebook. They
don't get any money when you do that, and they want money. Hence, a problem.

There are several ways to get around this. Be a middleman that publishes to
both Facebook and Twitter (a reverse FriendFeed). Or, sell a software product
for the user to use to move his own tweets to Facebook. A user obviously owns
his own content and can put it wherever he wants. A third party app, perhaps
not.

(Also, why not just get tweets via whatever method the native UI gets tweets
from? Do it from AWS if you are concerned about an IP ban.)

~~~
towelrod
Not only that, but he's stripping out the two things that would actually drive
traffic back to twitter.com: #hashtags and @names.

I can see why Twitter might want to slow this one down.

------
Aaronontheweb
This story rings true for any popular app that depends on a third-party
service, not just Twitter - often times you're at the mercy of the host once
you become big enough to be noticed.

------
rsarver
I've been following the thread and I commented on Michael's site, but thought
it would be worth sharing on here as well:
[http://hoisie.com/post/a_warning_to_hackers_be_careful_build...](http://hoisie.com/post/a_warning_to_hackers_be_careful_building_on_twitters_api#comment-125739196)

Since most of you probably don't know me, I'm director of the platform at
Twitter.

Let me know any questions/comments you might have. I'm interested in an open
discussion about it.

Ryan / @rsarver

------
dacort
As I realized after Twitter's "developer" conference last year, the era of the
Twitter hacker is coming to a close (see
<http://twitter.com/#!/dacort/status/12005978721> and
<http://twitter.com/#!/dacort/status/12032959629>). If you are not building a
business on their platform where money will pass into _their_ business, good
luck.

And who can blame them, providing 100,000 user tweets isn't free. Not sure
where he got the idea he would never have to start paying.

Turning off basic auth also hastened this demise - whereas you could once pull
whatever data you wanted from Twitter using a simple curl command, now you
have to figure out OAuth. It's not that much of a challenge, but it is enough
of a barrier to entry to dissuade somebody who's got a couple hours on the
weekend and wants to have some fun.

Finally, their partnership with Gnip is yet another indicator that this is
simply not the days of the wild west anymore. You want the data, you'll have
to pay.

------
lacker
Of course you're at their mercy. This is the nature of using a third-party
API. If you violate their rules, you have to accept the risk that they shut
you down in the future, even if their rules aren't optimal for your app's user
experience.

I recommend that you alter your product to conform to their rules, even if it
makes your product a bit worse.

------
A1kmm
The way around API limitations on how much data can be read is to resort to
scraping. However, Twitter probably blocks individual IPs that access too much
data. The solution to that is to convince enough users to install software
that lets you access their website - preferably via forwarded SSL so your
users can't compromise data integrity. Users get some reward, presumably quite
small, for relaying the requests for you.

The scheme could be opened to provide unofficial paid APIs for Twitter and
other 'walled gardens' that make data available to unauthenticated users on
the Internet.

------
wslh
With Microsoft monopoly you always had the opportunity to reverse engineering
the OS but when you can't see the binaries because they are in the cloud
you're in trouble, it's worse than closed source.

Building your business around web apis without an SLA is the most risky
business, you don't have control,

Enjoy your 15 minutes of your application placebo fame!

------
jv22222
I've had a whole litany of issues with Twitter doing stuff like this... but
ultimately I've found work-arounds. It just depends how committed you are and
how much time you have to make the fixes! More info here:
<http://pluggio.com/blog/>

------
rwhitman
Crap, I have an app that was build long, long before display guidelines
existed (heck even hash tags didn't exist yet!). No time or energy to fix it.

And honestly, that style guide is ridiculous. Way to alienate your earliest
champions, Twitter.

------
echaozh
Microblogging sites in China are thriving, much better than SNS sites (IMHO).
I often come to wonder why FB is worth so much more than twitter. Could this
actually be one of the cases when copycats win over the original?

------
ErrantX
I had an even worse issue (entirely my own fault) which highlights the dangers
of building on another platform. My app (tweetbars.com) had a tiny flaw in
that it didn't time out the Curl calls to the Twitter API, and my host didn't
kill hanging php executions.

So when Twitter started hanging (and eventually timing out after a minute or
so) the app basically ate one of their shared servers and the host took my
whole account down for 24 hours.

~~~
code_duck
I've found when working with Twitter, even more so than other services, one
must expect calls to fail frequently in a variety of novel ways. Code
defensively!

------
stuhood
John Kalucki from Twitter's platform team responded in the comments with what
sounds like a reasonable alternative:
[http://hoisie.com/post/a_warning_to_hackers_be_careful_build...](http://hoisie.com/post/a_warning_to_hackers_be_careful_building_on_twitters_api#comment-125554983)

------
dedward
One should normally be careful of any freely available service - they owe you
nothing. If you really want to partner with twitter for a cool app, approach
them about signing an official agreement or something, no?

~~~
riobard
They probably won't give a damn. The power is highly unbalanced. Try talk to
Google for a deal.

------
citricsquid
with regards to the lack of approval, could it be that Twitter want to avoid
app redundancy to save resources? 100,000 of these requests can't be cheap for
them, so when there are already multiple apps do what this one does does it
not make sense for them to say they're no longer supporting most to save
money?

I could be way off the mark, but that's how it appears to me. Multiple apps
with the same purpose that require lots of resources... makes sense to stick
with one or two high users and limit the rest.

------
fookyong
what is the 100k follow limit and why does it break your app?

~~~
mprovo1
From what I understand, he is using the 'statuses/filter' call of the stream
api, filtering with users that have installed his app. This allows him to get
the tweets of all his users in a single connection to twitter.
'statuses/filter' and 'statuses/sample' are the only two methods provided to
all accounts and they are very limited. I mean, 100K users is a big number for
sure, but not a surprising number given it's a twitter app on _facebook_. We
are far from the firehose level!

As a work around, he might be able to go back to the REST api, use
statuses/friends_timeline, and stay within the 300 per hour/per user limit.

Good luck Michael.

------
lhnn
the author has some points, but he does not mention something obvious to the
reader: He was always at the mercy of Twitter. It was his fortune that the cap
was increased at his whim, and only now have they stopped it, for whatever
reason.

The reason sounds fishy, and his retort is well-founded, but it's an important
thing to note.

~~~
coffeedrinker
I hate being at the mercy of another; that is why I have my own business.
However...

I just had my first major rejection from the app store on a unreleased app
(similar to an already approved app called Fresh Faces, which has been
approved four times).

I got a phone call from Apple today telling me it was rejected because it
violated section 18.2. I should have asked him to read it to me because I
think he was too embarrassed to let me know what it said. He mumbled something
about "social media" blah blah.

Well, I looked up 18.2 and was insulted. 'Apps that contain user generated
content that is frequently pornographic (ex "Chat Roulette" apps) will be
rejected.'

My apps allow users to upload photos of themselves and have others rate them.
All photos are examined before posting, and I would never allow anything bad
to be posted.

By this standard, any app that allows users to create and view photos should
be banned.

Today I downloaded the Android development kit.

~~~
jrockway
_Today I downloaded the Android development kit._

Good for you. Freedom is always prettier than a nice UI.

~~~
mst
And if you're happy with your UI -and- don't like animations, you too can be
mentioned on Daring Fireball!

