Thankfully, the GPU hasn't been afflicted with terrible inefficient slowness for 2D programs. The interesting questions lie in what the CPU is doing and wasting time on.
And, I mean, the C64 offloaded rasterization to a different chip too.
To scroll the screen on the C64 actually involved a non-trivial amount of CPU usage, because the hardware can only shift the screen by up to 7 pixels, so every 8 pixels, one has to manage all of that because the VIC chip can't — this involves offset copying all of the screen RAM, which can be done over previous frames, and same for the colour RAM, except this must be done in realtime (because the colour RAM has fixed location / cannot double buffer). Doing all of this actually takes a significant proportion of available CPU time per frame in C64 games. And that's just for the character-based graphics mode, not the bitmap graphics mode — which you simply have not got a chance in hell at scrolling on the C64, because the VIC offers no assistance here and the CPU is nowhere near fast enough to do it.
Don't even get me started on drawing more than 8 sprites!
And, I mean, the C64 offloaded rasterization to a different chip too.