Infinite scroll annoys me, period. I spend a lot of my time on 3G mobile, and data costs me money. When I scroll down to the bottom of the page, it's because I want to read what's there, not because I want to give my mobile provider more money. When we're all on flat-rate, unrestricted hyperspeed connections, maybe. (I wait with bated breath.)
Infinite scroll annoys me, period, too, but for entirely different reasons (ones that aren't going to change with improving network infrastructure, more related to ComputerGuru's reasons).
When I start browsing the sort of page that web developers are prone to applying infinite scroll to, I usually begin with a sort of semi-subconscious assumption that I'll just read down to the end of the page, or just the first N pages (for some smallish N) or something. I start doing so, but for a while don't realize that I'm never going to reach that point because the page is continually growing as I read it, and thus lose my way of quantifying (admittedly very vaguely) how much time I've wasted^W spent on it and where I should stop. My usual solution is to hit the "end" key (or its moral equivalent) and scroll up from there, ignoring whatever loads beyond that point.
Now from the developers point of view this may be a good way to get me to spend more time on their page, but if they're actually "gaming" me like that, it really just makes it enormously more obnoxious than it already was to begin with. Gaming intended or not, it strongly disinclines me to continue visiting the site in the future.