I have a tendency to believe that any research, in any field, veing used for any purpose, should be discounted if it doesn't present its source data. And in computer science, the source code. The fact that this doesn't happen 100% of the time already is baffling to me, it feels like science is broken.
This is the kind of black-and-white dogmatic thinking that is the tell-tale sign of software engineer applying their software engineering context to a context where it doesn't fit. I have no doubt many people on this board (not necessarily you) share a similar view, yet are similarly disdainful of privacy protections in the US around one's use of technology (e.g., cell phone tracking, DNS monitoring, cookies, ad-tech).
I suspect they're all for privacy. They don't want people's data made public, either. Rather, they're OK with the fact that policy can't be made, because they're not especially interested in the policy implications.
They likely assume that any regulation will be bad by default unless it's proven to them -- and won't put any effort into reading that proof when offered.
Software engineers get used to the notion that things either work or they don't. Policy is a much messier process, where you do the best that you can with incomplete information. And software engineers are generally comfortable enough under the status quo that they will set high standards for change -- and aren't interested in how that affects anybody not in their immediate circle.
That's definitely not me. The point is that an effort should be made to make available the data to the fullest extent possible. I don't think it's unreasonable to expect anonymised data to be made available. If the data connot meaningfully be anonymised that might be a special case, but I find it hard to imagine a scenario like that, unless the study only involved a very small number of participants (which would also make it less valuable). Maybe there are such scenarios though, I'm not stating categorically that there aren't, just that it seems unlikely, and if there is I'd expect them to be a small minority.
When your subjects are human beings, the rules are different - both for ethical reasons, and because if you violate your subjects' privacy you won't have any more subjects for next time.
Part of the EPA's mission is to protect human health. It is often illegal to release medical data underlying studies that link environmental factors to health problems.
So, we discount the research, and just go by what private industry says? Because that's the alternative, as industry will massage and cherry pick the data until it fits whatever narrative it wants. It has a lot of money at stake.
Does public research do it even remotely to the same extent as what gets us advertisements like "Four out of five dentists recommend Marlboro cigarettes"?
Policy has to be based on something. Public research is being attacked because it's not perfect. Okay. If you're going to do that, you should apply the same expectations to private research, and also require that firms prove that their products are safe, to the same rigorous standards that you expect out of public researchers.
>"Four out of five dentists recommend Marlboro cigarettes"?
I don't see how that's any different than when congressmen get interviewed by the talking heads and say "the research is clear, there's been study X by Y and P by Q that show that what we need to do is enact <insert some asinine extremist legislation that that congressmen supports>". They're both lying in a plausibly deniable manner in order to market something to the public. Anyone who bothers to verify either claim will find it's all crap.
The problem isn't the research. It's that we blindly trust "the research" and people know this so the research gets manipulated and cherry picked and whatnot. Two hundred years ago these sorts of people and entities all invoked gods name in order to peddle crap. I have no idea what the solution is but I think whatever it is will involve the public being more disapproving of parties that behave in a slimy manner like this.
> I have no idea what the solution is but I think whatever it is will involve the public being more disapproving of parties that behave in a slimy manner like this.
One of the two parties is currently seriously considering not accepting the results of an election they lost. The leader of that party is busy calling up election officials, and threatening them into magically finding thousands of votes that will give him the victory.
And the rank and file of the party isn't distancing themselves from this behavior. Instead, they are turning their ire at the person who leaked it.
On the order of slimy political behavior that voters don't punish politicians for, this issue we're discussing won't even register. We're currently so far down the rabbit hole, I don't think we'll ever find our way out.
> I have no idea what the solution is
That's because there is no perfect solution. We don't live in a perfect world. We just have to find the best solution out of a space of sub-optimal ones. And in this case, public research is a much better starting point for policy than private research, or no research (which is what would happen if we applied your standard.)
You have two sets of incomplete data. One from public research, where some of the data is missing, and one from private research, where nearly all of the data is missing.
Which do you base policy on? Doing nothing, by the way, is also making a policy decision.
I will also point out that changing the rules post-facto, like we're doing here, is like asking an open source project to verify that every single contributor to it has consented that the project migrate to a new, incompatible license.
