I don't think they mean like that. If something is obviously slow its a problem regardless. However, especially when you have a certain number of users, even a fraction of a second faster load time can have a visible change in your analytics.
I've read before that Facebook has shown that users tend to spend a fixed amount of time on their site. Once users hit that time limit, they're done. If your site exhibits similar usage patterns, the faster your pages load (even if they're already fast), the more users can get done on your site, which, depending upon your revenue model, may result in more revenue.
That information about fixed time usage is interesting. But even with that aside, doesn't everyone know that fraction of a second decreases in load time is important for high volume sites?
For example, MySpace was getting at one point 24 billion page views per month. If you could reduce each page view by 1/100th of a second you save 40 weeks per month for your users (assuming the model where they look at a fixed number of pages).
In the Facebook model, if you assume a page comes up in half a second, this delay results in a 2% decrease in page views -- which is a pretty huge deal when your business model indirectly revolves around page views.
I guess my point is that even for people who come from a background where page load didn't matter. it would take 30s to point out it does, and I don't think you'd get any pushback.
I've read before that Facebook has shown that users tend to spend a fixed amount of time on their site. Once users hit that time limit, they're done. If your site exhibits similar usage patterns, the faster your pages load (even if they're already fast), the more users can get done on your site, which, depending upon your revenue model, may result in more revenue.