That is a big question and maybe someone like 'rayiner or 'marcoperaza would be a better person to ask, because I have no legal training.
Based on my interested but layperson understanding of the law, my understanding is that it's an open question about the extent to which the Constitution provides a right to privacy. But I do not understand it to be an open question, or at least "as open" a question, about whether the principle of access to evidence is ordered higher than that right to privacy, whatever it may be.
(As, just, like, a person shooting the shit on a message board, I should say that of course I value privacy highly, and can think of a number of situations in which I'd order privacy higher than access to evidence! But I can also think of situations where it'd be the other way around. So, for instance: I highly value my right to discuss sensitive political topics privately, without being susceptible to dragnet surveillance trying to find "terrorists" by combing through everyone's messages. But at the same time, I'd like to see Wall Street executives be held more culpable for the harmful decisions they make to abuse their power and access, and those Wall Street execs no doubt feel a strong privacy interests in protecting their discussions about how to, say, price synthetic securities.)
In Griswold v. Connecticut [1], the Court found the state's law to to be in violation of "right to marital privacy" and deemed right to privacy as right to protection from governmental intrusion.
One Justice cited the fifth amendment to get there, another the ninth -- and plenty of legal scholars have argued the ninth amendment basically acts as justification for finding privacy provisions in the Bill of Rights that aren't specifically mentioned in the other eight amendments.
You're right that right to privacy is an open question of sorts (especially under the Bill of Rights), but there is nearly a century of case law protecting it in various forms.
There are a number of ... not-very-good books on the subject (I picked up Carolyn Kennedy's 1990s effort, it's ... poor).
A challenge is that privacy itself is an emergent concept (this is my view, not one widely shared), which is a response to ever-more invasive or capable technologies. As such, much of the relevant caselaw and legal thinking is fairly new, and reflects advances in publishing (Brandeis and Warren), recording, transmission, and detection.
There are some historical antecedants.
Some of the more interesting non-legal-scholar work has come from Jill Lepore in recent years. I'd suggest her work as at the least a good entry point. She's a historian, and should be copiously documenting sources which may be of interest. Unseen: A History of Privacy (2013) https://scholar.harvard.edu/jlepore/presentations/unseen-his...
