I don't think having a more engaged audience is necessarily that good of a metric to look at. The more popular a website is, the more casual viewers it will attract. The fact that justin.tv has such a high viewership time may have more to do with the fact that its audience is tiny, and that viewers kind of have to consciously seek it out. As opposed to youtube, which is one of those timewasters you go to without really thinking. If/when justin.tv grows, it may find its time per viewer ratio shrink proportionally.
Of course, there is absolutely nothing bad about having an engaged viewer base. Especially if that core groups' enthusiasm can be used to help recruit more eyeballs.