I think you've misunderstood level 4; it seems to be defined as you suggest 4.5 ought to be: [http://www.sae.org/misc/pdfs/automated_driving.pdf ]; the car has to operate safely after autonomous mode is enabled, even if the human has no further interactions (even if the car requests human intervention).
It seems like the definition of level 4 is a bit ambiguous; your link defines it as operating without human intervention, but only in "some driving modes". I've seen this interpreted as meaning that the vehicle can operate without human intervention as long as the driving mode does not change, but may still require human intervention if the conditions change (e.g., it starts snowing, or you exit the highway).
The key distinction here is that a Level 4 is safe even when the driver does not intervene. If it starts snowing, the car will park by the side of the road if the driver isn't there to pick up the wheel.
Level 3 is when you have ample time before the switchover, and Level 2 is when the driver must be there in a matter of seconds.
