NAP allows for "retaliatory violence" (had aggression but as pointed out below is not the correct usage of the word aggression).
One of the difficulties, as you point out, comes when people don't realize that they are aggressing others. An example being when they attempt to limit who can get married to who (whom?).
Again, there are good intentions behind such aggression (limit the right to use marijuana to protect children) but the results of aggression seem to always lead to more aggression (creation of an illegal drug black market leading to instability in Mexico).