How well would you work if I stood quite close to you, behind you, looking over your shoulder. "Ignore me! I'm just observing. Pretend I'm not here." I suspect many people would find that disturbing in a deep and hard to define way.
He's not doing anything illegal, but that doesn't mean he's not doing anything wrong. He is being rude, and he is confrontational.