If someone finds a way to monetize tweets that's more effective than Twitter's, then Twitter would have an incentive to copy it, therefore competing with its customers.
Long story short, that's the reason Google stopped selling white label search via its api to customers like Yahoo ten years ago.
There are many uses for twitter data outside of just repeating the tweets with a different design. There are quite a few Twitter apps that generate revenue and some have also been acquired by bigger companies. Twitter and Google main service offering are not comparable, since Twitter has the actual data while Google is a gateway.
"we will require you to work with us directly if you believe your application will need more than one million individual user tokens"
which sounds like "show us the money" to me. You can of course assert that "we want to talk to you" obviously means "we don't want to talk to you", but once you do so without evidence you're firmly in tinfoil hat territory.
There is a reason for a hard cap: the limit's goal is to ensure twitter-owned clients dominate the market for basic twitter use.
You may have a point by saying that twitter can simply make the tax per additional tweet equal to the approximate revenue they plan to lose as a result. But that figure may be hard to calculate, especially its long-term value. So even if a company is willing to pay 5x to twitter per additional tweet over the hard limit, twitter doesn't know the long-term impact of it
Have you thought about how Twitter would actually structure something like that? Twitter is monetizing FAR FAR better than any of these other small Twitter API clients. Something like Tweetdeck might have generated hundreds of thousands of dollars, but thats a joke compared to the kind of money Twitter is dealing with.
I hate ads as much as anyone, but they are working for Twitter. If you think Twitter's executives don't know what they are doing, you underestimate them.
Regarding ads, people consume twitter for opinions, networking and links to content. They are rarely with purchase intent, as opposed to when they are actively searching something on Google, for instance. Facebook are facing a similar problem with their ads, but Facebook still has a much larger userbase and the time spent on the site is much larger than with Twitter. Promoted tweets are too obvious and often have a reverse effect on people who receive it.
The point is, you have no data to back your assertions about what's good for Twitter as a business. They spent years refining their business model based on their data, and they picked the one they have now.