Warning: long somewhat related story that is basically humblebragging, but the summary is that bypassing Twitter ratelimits is not very hard.
I didn't feel like playing around with Twitter's annoying certificate pinning so I just uploaded the Twitter APK to Corellium, turned on what they call the "network monitor", opened the Twitter app since it lets you use Twitter without signing in. I clicked around, searched and viewed tweets. Then I looked at the requests in the log and saw it has a similar guest token process to the website but with a few differences. Anyways, if you recreate these requests, with one IP address you can generate a few OAuth tokens with no expiry per day. These tokens are for unauthenticated users so obviously they have no write privileges but that's not what was needed here. So if you have a proxy provider with a large pool of IPs where you can buy like 1GB of bandwidth you can use a very small percent of your bandwidth allowance and get thousands of tokens/secrets easily, all with their own separate rate limits. It doesn't even matter what IP you end up using the tokens on. Then I followed https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW... and the fact that /statuses/lookup.json still allows you to return 100 (!) tweets at once to reconstruct something close to what the 50% Twitter firehose would look like. And Twitter doesn't even block datacenter IP addresses! Was going to display the data at https://firehose.lol but the fact that it required a few hundred requests a second made me feel bad so I didn't end up running the program for more than a few minutes at a time and shut it down.
Looking at (a fraction of) the Firehose for a few minutes was interesting, originally I accidentally forgot to not display tweets labelled possibly_sensitive so I saw some pretty salacious material for a few seconds. Lots of Chinese gambling ads even though Twitter is blocked there, dubious investment promoters, accounts with usernames like FirstnameLastname3781264872 who would tweet three random words at each other every couple of seconds, and a handful of funny tweets.
Nice. This isn't nearly as efficient, but a simpler way to bypass the ratelimit is to use archive.md, which is immune to the ratelimit. It's useful if you don't have an account and just want to see a few tweets here and there.
I didn't feel like playing around with Twitter's annoying certificate pinning so I just uploaded the Twitter APK to Corellium, turned on what they call the "network monitor", opened the Twitter app since it lets you use Twitter without signing in. I clicked around, searched and viewed tweets. Then I looked at the requests in the log and saw it has a similar guest token process to the website but with a few differences. Anyways, if you recreate these requests, with one IP address you can generate a few OAuth tokens with no expiry per day. These tokens are for unauthenticated users so obviously they have no write privileges but that's not what was needed here. So if you have a proxy provider with a large pool of IPs where you can buy like 1GB of bandwidth you can use a very small percent of your bandwidth allowance and get thousands of tokens/secrets easily, all with their own separate rate limits. It doesn't even matter what IP you end up using the tokens on. Then I followed https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW... and the fact that /statuses/lookup.json still allows you to return 100 (!) tweets at once to reconstruct something close to what the 50% Twitter firehose would look like. And Twitter doesn't even block datacenter IP addresses! Was going to display the data at https://firehose.lol but the fact that it required a few hundred requests a second made me feel bad so I didn't end up running the program for more than a few minutes at a time and shut it down.
Looking at (a fraction of) the Firehose for a few minutes was interesting, originally I accidentally forgot to not display tweets labelled possibly_sensitive so I saw some pretty salacious material for a few seconds. Lots of Chinese gambling ads even though Twitter is blocked there, dubious investment promoters, accounts with usernames like FirstnameLastname3781264872 who would tweet three random words at each other every couple of seconds, and a handful of funny tweets.