This is because when sending the response you need to know the HTTP status of that response. Is it a redirect? And if you're querying the database lazily, after your page started to render, what about DB errors that could happen? Then you'd need to send an HTTP 500.
This is the drawback of using this feature in Rails 3. You have to ensure that there's no way something unpredictable happens. And you have to activate the feature explicitly in your controllers. And after you did that, you're probably going to be aware that TTFB is not that relevant.