When my own kid was very young I was paranoid about touching him in public. It didn't stop me, but I was mindful of it, and I would occasionally feel I was getting extra scrutiny by some parent; a man not at work in the daytime, holding a young boy's hand.
This strikes me as odd. As a parent (dad) of two, I wouldn't ever dream of feeling odd about holding either of my kids' hands - or even for that matter chasing them down when they run away screaming (which happens often).
I didn't feel odd for the act, I felt uncomfortable with the assumptions or speculation implied by the scrutiny. No one likes to be distrusted, regardless of the context.
Feh.