
New developer requirements to protect our platform - ColinWright
https://blog.twitter.com/developer/en_us/topics/tools/2018/new-developer-requirements-to-protect-our-platform.html
======
saurik
As someone who is apparently "old", I continue to not understand how Twitter
has managed to trick an entire generation of developers into believing they
need Twitter's permission to develop a third-party Twitter client. Has the law
changed in some noticeable way to make reverse engineering the official
Twitter client and extracting their first party keys for purposes of
interoperability suddenly illegal? We used to do this _all the time, at scale_
before Twitter's invention of the "API key", and the only recourse the service
has is to try to detect weird usages of their backend and then punish the
users... something that tends to be something of a PR nightmare. (FWIW, I am
totally willing to believe the issue is that Apple won't accept them, but then
that is Apple being the asshole, and we should be looking into ways of making
that market collusion illegal, assuming of course that it isn't already.)

~~~
elorant
I can see two counter arguments. One, you don't want to go with Twitter, or
any large corporation for that matter, to court. They have a legion of lawyers
and you don't. They have budgets for these kind of cases, you don't. Even if
any dispute you have with them won't make it to a court you'll lose precious
time and resources consulting with lawyers in order to answer their C&D
demands, which I guess at some point will escalate if you keep ignoring them.

Then you have the practical part. Would you really base your project on
reverse engineering and hacking a third party service? It will end up being an
arms race. Everytime they change something you'll race to hack it again. In
the meantime your service will be inoperable. If your user base is in the tens
of thousands that will kill you. I can't think of any reliable service that
every couple of months or so would go offline even for a few hours until the
developer finds a way to circumvent whatever countermeasure Twitter came up
with.

So you play nice and follow their rules. Even if they suck, which by the way
is the main reason why Twitter lost all the love of developers and has
stagnated for the last years.

~~~
ballenf
> It will end up being an arms race. Everytime they change something you'll
> race to hack it again. In the meantime your service will be inoperable.

They (any service with a large user base across many devices) are at a serious
disadvantage of not being 100% in control of app upgrades across the legion of
devices and app versions supported. They are just as likely to break a
legitimate user's app as yours (even higher, imo). I don't think the technical
war is nearly as difficult as you imagine. On the legal side, you might be
right depending on the geographical location of the dev.

In regard to the original question, I think we'll begin to see a shift in that
direction as these services tighten the screws. Like youtube-dl, Youtube++ app
is a relief to anyone who watches much YT on a phone/tablet. It's easily
sideloadable even on iOS and actually puts the user first with regard to
permanently settable playback speed and ability to hide algorithmic
suggestions among many other tweaks. It's based on a year+ old version of the
official app and is still working fine even after being DMCAd. So you know
they got Google's attention, but haven't had to update the app in almost a
year. I also see these gray area tools as an important check on the power of
the platforms to not abuse their dominance. Like jailbreaking was before Apple
started incorporating many of the more popular tweaks into iOS itself.

~~~
jhall1468
> They (any service with a large user base across many devices) are at a
> serious disadvantage of not being 100% in control of app upgrades across the
> legion of devices and app versions supported. They are just as likely to
> break a legitimate user's app as yours (even higher, imo).

Yes they are? Alternative apps use the official API, their internal services
use their internal API's, which are also likely versioned. So if the internal
DM API has an upgrade, the website can change to the new version while the
mobile app is using the previous version.

youtube-dl and Youtube++ are already solved problems. The fact that you have
to sideload it means the user market is tiny and at this point I highly doubt
YT even cares anymore.

> I also see these gray area tools as an important check on the power of the
> platforms to not abuse their dominance.

This is the weirdest thing you've said in a whole lot of weird things. YT
proved how powerful it was via demonetization of so many creators videos.
Nobody could do anything about it, primarily because the real power behind YT
is that _creators_ don't have an alternative, not users.

~~~
ballenf
> Yes they are? Alternative apps use the official API, their internal services
> use their internal API's, which are also likely versioned. So if the
> internal DM API has an upgrade, the website can change to the new version
> while the mobile app is using the previous version.

I think we might be speaking about different things. My original point was
that reverse-engineering Twitter's private APIs (used by the first-party app)
is not as difficult to do or maintain as the parent suggested. My reasoning
was that for the foreseeable future there will be legitimate users on non-
updated clients, forcing Twitter to choose between disabling those clients or
letting your 'cloning' of those apps' API calls continue to function.

> This is the weirdest thing you've said in a whole lot of weird things. YT
> proved how powerful it was via demonetization of so many creators videos.
> Nobody could do anything about it, primarily because the real power behind
> YT is that creators don't have an alternative, not users.

Maybe I wasn't clear. I'm not speaking about creators-vs-YT, but consumers-vs-
YT in terms of anti-user restrictions in the YT app and website. Issues that
grey area tools alleviate and therefore provide the only effective check on
YT's ability to take them further. Just as abusive ads and tracking has caused
a jump in adblock usage. The harder it gets to enjoy YT content through the
first-party app, the more people will take the time to discover alternatives.

The concept generalizes to Twitter or YT or any platform (or creative
industry) that achieves ubiquity or monopoly. Even if you disagree, maybe it
sounds less 'weird'.

------
kevin_b_er
The translation of this is: We're trying even harder to kill any and all 3rd
party applications. 300 tweets per 3 hours global rate limit is bonkers. My
favorite twitter app for android is probably dead after this.

~~~
toomuchtodo
On the plus side, signal to noise is about to skyrocket on Twitter.

~~~
slg
This will make maintaining an army of bots more expensive. It does little to
actually stop it.

~~~
toephu2
So you don't believe price effects the creation and maintenance of bots?

When maintaining an army of bots becomes more expensive than the amount of
money they are making from it, they will stop doing it.

~~~
existencebox
Having dabbled in the space a bit: (on the less-malicious side, scraping for
archiving, I imagine pro bot-writers are far beyond me.)

There's an upper bound to "bot complexity"; that being the complexity you want
to impose on a normal user. For many sites (e.g. instagram) it's often easier
to just "pretend to be a normal user" and hit it from a headless browser. You
lose some functionality, but at least for my needs, that's always been a
sufficient fallback.

As such, I've always been of the opinion that the primary function of API
limitations is to punish/limit/extract more money from the good participants.

------
strken
How long before we go back to dodgy apps that just ask for the user's email
and password, if regular API access is so heavily restricted? I know there's
an npm package out there for my bank that runs a headless browser and requires
full credentials, which is terrifying.

~~~
sgt101
I'm lost - how does what your bank does and what Twitter does connect?

~~~
strken
Twitter is currently locking down their API and making it harder for
developers to access. My bank doesn't even have an API, but developers still
gain programmatic access to it, only they ask users for their full credentials
instead of limited, monitored, and revocable API access. My concern is that
Twitter's new restrictions, and restrictions like them, will push apps towards
just collecting a user's login credentials.

~~~
xfer
How is running a chrome headless any different than running it normally,
that's how you login to your account, don't you?

~~~
strken
It's exactly the same thing, which is why it's bad. When I log into e.g.
CoolQuiz[0] on quizzes.cambridgeanalytica.com with Facebook using OAuth 2, it
requests a limited list of permissions from Facebook, which are shown to a
user, and which don't include things like like resetting the user's password
or changing the user's email address. It gets back an access token which can
_only_ be used for doing those things, and Facebook can track what's done with
that access token, in the case that e.g. CoolQuiz uses it to read personal
information from all a user's friends and microtarget advertising during a US
election. Headless chrome is indistinguishable from a normal user, which means
it can do absolutely anything a user can do and can't be tracked.

[0] probably doesn't exist, just a hypothetical example

~~~
xfer
Right, i missed the oauth2 scoping aspect.

------
ttepasse
_The new default limits for each endpoint are outlined below and will apply in
addition to existing user-level rate limits for these actions. By default, an
app (across all of its users) will be limited to: \- Tweets & Retweets
(combined): 300 per 3 hours \- Likes: 1000 per 24 hours \- Follows: 1000 per
24 hours \- Direct Messages: 15,000 per 24 hours_

They write that they can lift this requirements for certain apps. But any new
3rd party twitter client is pretty much toast.

~~~
mrbill
This is per-user, not per-entire-app-user-base, AFAIK

~~~
jasonlotito
No, this is per app.

------
detaro
That sounds like the end of a lot of API use for small players... Review is
noticeable hurdle for small/private projects, and at the same time services
serving many small users will easily have trouble with the rate limits, unless
they make it easy to get those raised (which is possible, but I'd be
skeptical).

------
css
Instagram did this a long time ago with their API - they require you pass a
Permissions Review [0] where you submit your featureset along with a website
landing page (ex. for a product offering) for review by a human who then can
accept or deny your application.

[0]
[https://www.instagram.com/developer/review/](https://www.instagram.com/developer/review/)

~~~
nkozyra
This is also the approach Facebook has taken in the wake of the Cambridge
Analytica debacle.

Everything has shifted back to the walled garden approach. Twitter had
destroyed a wide ecosystem of applications that promoted their brand several
years ago via a similar approach. FB has done (and may be again doing) the
same.

The message remains clear: don't build your business on another business
without contracts and guarantees. When the utility of having you as a
promotional instrument wears off, they will dump you.

~~~
css
> don't build your business on another business without contracts and
> guarantees

Or even with them: [https://techcrunch.com/2018/06/21/twitter-smytes-
customers/](https://techcrunch.com/2018/06/21/twitter-smytes-customers/)

~~~
meesterdude
That's... amazing.

These two tweets really show just how WTF this was

> A vendor notified us of their acquisition at 6am this morning and shut down
> their APIs 30 minutes later, creating a production outage for npm (package
> publishes and user registrations). The sheer unprofessionalism of this is
> blowing my mind.

> It takes weeks to negotiate and sign an acquisition. You didn't find out at
> 6am. You couldn't give us a week? Even a couple of hours to take your
> service out of our critical path and avoid an outage? Fucking shocking
> behavior.

------
Macha
A common workaround after Twitter attempted to kill third party client apps
was to have users register their own "app" and insert the API keys. So this
will put a stop to that.

~~~
notatoad
These new limits seem specifically designed to continue allowing that
workaround. 300 tweets per three hours is about the upper limit of what a
single user would do, so if you're registering a developer account just for
personal use you can do that without any manual review or having to
demonstrate having a real product.

You only require approval from twitter and a product website if you have
higher usage than that, which is what would prevent you from using that
workaround.

~~~
pnevares
I think I see what the OP meant, here are relevant excerpts from the article.
New API access:

> Beginning today, anyone who wants access to Twitter’s APIs should apply for
> a developer account using the new developer portal at developer.twitter.com.

Existing API keyholders:

> Eventually, all developers with existing access to our APIs will be required
> to complete a developer account application in order to maintain their apps.

~~~
notatoad
Maybe i'm misreading it, but it looks like you're essentially just filling out
a form with your contact info and agreeing to the T&C for the initial
"application".

------
taytus
Meantime:
[https://www.dropbox.com/s/hb7c0l54r8m0zoy/Screenshot%202018-...](https://www.dropbox.com/s/hb7c0l54r8m0zoy/Screenshot%202018-07-24%2015.34.08.png?dl=0)

I know that fixing/changing/upgrading things at scale is much harder that what
most people think, but man... they have done nothing to fix these type of
issues, and don't tell me is not something easy to fix.

The other day a journo complained on twitter about the problem of dealing with
crappy messages:
[https://twitter.com/sarahfrier/status/1020512187322789888](https://twitter.com/sarahfrier/status/1020512187322789888)

So in less than an hour, I had a prototype to help to filter those type of
messages:
[https://www.dropbox.com/s/z914olzgn68v660/Screenshot%202018-...](https://www.dropbox.com/s/z914olzgn68v660/Screenshot%202018-07-21%2008.57.01%20%281%29.png?dl=0)

If a random dev can do this, why they can't fix it?

I like twitter, and I'm genuinely curious why some of these seemingly easy to
fix problems are not taken care.

~~~
pnevares
What are we supposed to be seeing in your second screenshot? I just see an
array of DMs, what's being filtered?

~~~
taytus
Yeah, sorry. You are right is not showing anything else besides the API
result.

But is trivial to return a secondary, already parsed array. I haven't used the
API in a while and was just seeing if I could achieve that.

------
AznHisoka
How would this impact a service like Buffer? Limiting posting 300 tweets per 3
hours certainly wouldn't be feasible, given the number of customers they
have...

~~~
detaro
The existing large players probably already have their exceptions worked out,
or will very soon.

Want to join that space? Gotta be lucky with review/limit raise request I
guess. It's possible they just want services to start small and act legitimate
to raise the bar for "bad apples" and will raise the limits easily afterwards,
we'll have to see.

------
tonystubblebine
I know a lot of the conversation about the API separates out good guys and bad
guys, i.e. spammers.

But I'm more and more thinking that Twitter and other platforms should
consider marketing automation to be a form of spam. Once you start questioning
that there isn't a human behind the human account you give up on human
interaction.

So yeah, maybe there's some vanity metric that marketing automation props up
for companies like Twitter. But underneath that vanity metric, the real
quality of the site is being diminished.

------
skybrian
This is basically "know your customer" for an API. It's episode N in "why the
Internet can't have nice things." If you make a technology available at scale,
it will be abused.

Compare with email spam countermeasures making it more difficult to set up a
email server that other email providers will accept mail from. Or malware
resulting in the rise of app signing and app stores.

------
jhabdas
No more free $ETH giveaways. Just darker, more deceptive black hats.

------
hivacruz
Well, I guess my movie quiz bot (@whattheshot) which turns 8 in a few hours,
is almost coming to an end then. I had to adjust so many things the last few
years that the game lost completely interest to most of my players. I was
shadow-banned anyway.

My last hope is that the "review process" they talked about could really
differentiate bad apps/bots and interesting bots and let them work. But I
honestly doubt it will happen.

------
adamw2k
Thinking through this a bit... So the end result of this move is less SPAM and
higher quality content. Great! But if SPAM is a form of activity (albeit a
less desirable one) and activity represents feed liquidity - which directly
impacts revenue - how does this not significantly hurt the bottom line? Time
to short TWTR in Q4ish.

------
dmitrygr
Given how many times twitter has screwed developers attempting to build things
on top of their API, does anyone still care? Does anyone still think
developing on twitter's API is a good idea?

------
sctb
We've updated the submitted title from ‘Twitter introduces new developer
requirements “to protect our platform”’. Submitters, please leave out the
editorial.

~~~
ColinWright
I'm the submitter:

This wasn't intended to be an editorial, it was intended to provide context. I
believe that it's not clear from the title alone as you've included it here
that it was Twitter. Yes, it's on the twitter blog platform, but so are many
other things that aren't actually from twitter.

Personally, I spent some time trying to make sure the submitted title was more
accurate and informative than the one on the post. I freely acknowledge that
it's your call. I think you got this wrong.

~~~
sctb
According to
[https://news.ycombinator.com/from?site=blog.twitter.com](https://news.ycombinator.com/from?site=blog.twitter.com)
it's the Twitter blog site, is it not? The quotation marks seemed to convey
something else, whether you intended them to or not. But since the original
title was neither misleading nor click-bait it's besides the point.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

~~~
ColinWright
One can't know that without going to the site and working it out. Exactly that
title on Medium, for example, would leave it completely unclear without
visiting the article. So unless you _know_ that this is Twitter's official
blog - which some people (including me) wouldn't - it's useful to have the
modified title. In particular you say:

 _... since the original title was neither misleading ..._

To me the title as it stands _is_ misleading. That's why I wanted to make it
clearer. <fx: shrug />

But seriously, your call. You thought it was editorialisation, intentional or
not, and you changed it. I think the existing title is less useful or
informative, but it's not my final decision. I've long felt that the blind
reversion of submitted titles to the title of the article is misplaced and
over-zealous, but as a moderator on other platforms, I know that every
decision requires work, so not actually making a decision is a huge saving in
time, effort, energy, and will.

 _(Several edits have been made as I try to make my points clearer. Not sure I
've succeeded, but I have no more time.)_

~~~
sctb
Ah, I see. Appreciated! I did jump to conclusions here and I apologize.

~~~
ColinWright
Thank you. I'd suggest some middle ground, but it's probably too late.

