The part of your post about following hand signals of construction workers is interesting. Could anyone jump out in front of a Cruise car and direct it? Would you have to be wearing something like an orange vest?
Do be fair, anyone could jump in front of any car with a regular driver and direct it with hand signals. If someone was standing on the road in front of me directing me into another lane I would do it, with or without an orange vest.
Most human drivers have an ACL to only accept these inputs from LEOs working in an official capacity. How would we add a visual ACL to a self-driving car?
Eh, I don't think so. When someone is standing in the street waving you into another lane (or not!), the alternative (other than just stopping there) is to ignore the person and drive into them. Most people seem to take the first option and trust the strange, non-uniformed person who risks their life to stand in traffic and direct it.
Yeah, the only reason to not do it is if I suspect some weird robbery heist is underway. In which case my only real option is to just stand there anyway...
In practice your assumption doesn't pan out. Humans follow the directions of almost anyone standing in the street waving their hands. For example, a pedestrian in the crosswalk may stop and wave a car on ahead of them. The car is supposed to stop and wait for them to exit the crosswalk.
By waving their hands, the pedestrian can actually be held accountable if this results in an accident.
Human drivers aren't really aware of the capacity or credentials of those directing traffic. Source: Sometimes I direct traffic. I'm part of a volunteer organization, taking orders from another volunteer organization, often eventually, yes, getting marching orders from police. People who are wearing vests may or may not be cops, construction workers, or volunteers like myself who have little to no oversight preventing them from putting the vest on and walking out into a busy street when they shouldn't.
At an event I was at recently, traffic control involved at least four different organizations, which had their own gear and their own attire, so you really can't count on everyone you see on the road looking the same either.
Additionally in terms of lack of credentials, there's a certification process for directing traffic (I've renewed mine twice), but there's a pretty non-zero chance that the person who has directed you around a construction site actually bothered to get that, and you'll probably never know that either.
Your other mistake is believing human drivers actually do what people directing them tell them to. Source: Sometimes I try to direct traffic.
