Well, I would guess that it leaves the vehicle stranded until someone comes along to remove the cone.
The vision system recognises the cone, gets confused because a cone on the hood is outside of the normal operating parameters, and refers the problem to a teleoperator. Pulling high-G manoeuvres in the hopes of dislodging the cone obviously isn't an option. The teleoperator can't do anything about the cone, so the car becomes stranded.
They don’t have a way to remotely drive the vehicle. They complete Mechanical Turk-like tasks, classifying images, approving the vehicle’s plan around an obstacle, etc.
That actually depends. You can either have teleoperators do waypoint control, or you can have them directly drive the vehicle remotely.
Waypoint controling has its upsides and downsides, and so does direct teleops.
The main upside of waypoint control is you have less strict latency requirements and its easier for operators to "drive," the obvious downside is that it requires the onboard planner to be working.
Direct teleop allows for more granular control, but has stricter latency requirements and is often annoying to actually drive with.
Interesting, I assumed that latency and dead zones made direct drive a no-go. Plus throughput could become an issue as AVs scale. Is there actually some truth to the 5g = AV hype?
I think direct drive has its use cases but big players (i.e waymo) use the former method of waypoint teleoperations, for the reasons you mentioned.
From a business perspective its also more scalable since you can just remotely control the bot with a mouse and a screen (and not need a full gaming setup). Also as you can imagine many companies probably also multiplex single teleoperators over multiple vehicles. Having indirect control facilitates this better since you dont need operators to effectively "drive" bots with their undivided attention one at a time.
5g does make teleops better in general, but its also kind of expensive and still incurs a latency penalty (i.e remote driver action --> bot action feels insanely sluggish). I think as you scale it becomes less feasible to do direct drive, as you said.
Disclaimer: I work at another AV adjacent company so this is just my take on what I think others are doing from hearsay.
The vision system recognises the cone, gets confused because a cone on the hood is outside of the normal operating parameters, and refers the problem to a teleoperator. Pulling high-G manoeuvres in the hopes of dislodging the cone obviously isn't an option. The teleoperator can't do anything about the cone, so the car becomes stranded.