If you hang on to 10 frames of RGB plus motion vectors at 1080p, and have 4 4K buffers where you're messing with output, that's 6MB * 2 * 10 + 25MB * 4 = 220MB of video memory. That's barely anything. And that's an extra-high estimate of the resolutions that would be involved in a switch game.
But we're talking about a computer with several extra gigs lying around running the same game. And specifically that number is for 4K output; anything less and you're under 100MB.
If you hang on to 10 frames of RGB plus motion vectors at 1080p, and have 4 4K buffers where you're messing with output, that's 6MB * 2 * 10 + 25MB * 4 = 220MB of video memory. That's barely anything. And that's an extra-high estimate of the resolutions that would be involved in a switch game.