I was doing the "chase the beam" approach, but never got it off the ground (home project, mostly curiosity). But always on the back of my mind was: even if you're doing everything in low latency HDMI signals, there's no guarantee that the display isn't still buffering it, even if you have "game mode" on on your TV.
This is a thing in the last 5-10 years, but was absolutely not a concern to the majority of display manufacturers before gamers complained. 100ms is significant in twitch games where network latencies are 20ms or lower.