You may be interested in Karl Popper's (famous mostly for the importance of falsifiability) 'The Open Society and Its Enemies'.
Choice quote from it:
"Unlimited tolerance must lead to the disappearance of tolerance. If we extend unlimited tolerance even to those who are intolerant, if we are not prepared to defend a tolerant society against the onslaught of the intolerant, then the tolerant will be destroyed, and tolerance with them."
It is not a perfect book by any stretch, but I think it covers certain topics particularly well.