You could say the same about a hypothetical encryption backdoor. The question is whether consumers should be allowed to use encryption the FBI can’t crack even with a warrant. I think the answer is “yes”, but…
Right, which is why the locked door is the wrong analogy.
It's more like burying treasure. The location is a secret map (key) that only you know. The critical distinction here, I believe, is between "having" and "knowing".
And I like to believe we have even stronger rights to what we know than to what we have.
All that said, this is assuming use of working cryptography. With the exception of the technically savvy (those who know how to hid things properly) asking for backdoors into encryption is akin to registering your treasure with the feds, something I'd assume gold miners wouldn't have put up with, for example.