That sounds more like a post-hoc justification, and I'm sure you could find similar ones for other rights - I gave just one out of many possible examples. Private property is great, until you're faced with a monopoly or environmental damage, free speech is wonderful, until you run into libel and threats, being "secure in their persons, houses, papers, and effects" likewise only applies as long as is "reasonable".
