We see it with encryption. Yes, encryption can be and is used by bad people for evil ends, but it is also used by good people for good ends; encryption is simply the means. It's a human right to speak privately. Law enforcement can still investigate, can still use different powers to eavesdrop and compel parties to speech to reveal that speech; but it has no right to be party to private speech. Yes, bad things will not be detected as a result, but it's worth it.
We see it with arms control. Yes, weapons can be and are used by bad people to kill innocents; but they are also used by good people to defend themselves, whether from crime or tyranny. It is a human right to be armed. Law enforcement and the military can still use their superior numbers and training to overwhelm a foe, but they have no right to claim a monopoly on arms. Yes, innocents will die, but innocents will also live.
We see it with the war on some drugs. Yes, some people will take certain drugs and commit crimes; but other people will take those same drugs and enjoy a pleasant mental state. Law enforcement can still arrest criminals, but it has no right to arrest folks who haven't harmed anyone. It's a human right to alter one's mental state. The tradeoff is worth it.
The thing is, you have to take this attitude toward all things: religion; drugs; weapons; encryption; speech in general. Once you start to carve out exceptions and exclusions to liberty, you'll end up losing ever-more liberty. It happens faster than one might imagine.