Well nothing is a full substitute for TV/movies, just like nothing is a substitute for novels. But the question is, is it an overall stronger form of entertainment? I'd say that video games definitely have the potential to be, the industry itself is already bigger than Hollywood and rapidly approaching the book industry . And the market for games just keeps getting bigger that there is no point in trying to dissect why games are not a fit for certain people. It's still unexplored space as Minecraft, GTA, WoW, farmville and the Wii have shown. Make the right game and you can potentially gain a new audience.
And I'm not dodging your assertion either, because games can also be passive. I've spent god knows how many hours watching Starcraft II games - well more than I'd like to admit. That's a game that was built with spectators in mind. I probably enjoy watching the game more than playing it, as wonderful as it is. Apparently so do other people, TwitchTV is evidence enough.