> a big weakness of Bayes rule is that you can look at any data and specify a prior that will make it look good
This isn't a weakness in Bayes' rule, it's a weakness in your experimental protocol. You're supposed to pick the prior before doing the experiment and seeing the data.
> In my opinion, in any switch to using Bayesian analysis in scientific work, pre-registering priors will be essential.
Pre-registering statistical criteria and assumptions should already be essential, whether you're a Bayesian or not. The fact that it isn't is a key factor behind the replication crisis.
> This isn't a weakness in Bayes' rule, it's a weakness in your experimental protocol. You're supposed to pick the prior before doing the experiment and seeing the data.
Sure, if the goal is to get to something true. If the goal is to publish or to maintain your position, though, you’ll work differently.
You don’t even need to have seen the data. If I set my priors for the earth being flat extreme enough, it’ll take a long time for even good faith updating to converge to reality.
As I said in another reply, I was simply pointing out that Bayesian analysis can also be abused, and that proper protocols still need to be followed. A point on which I believe we agree. :-)
This isn't a weakness in Bayes' rule, it's a weakness in your experimental protocol. You're supposed to pick the prior before doing the experiment and seeing the data.
> In my opinion, in any switch to using Bayesian analysis in scientific work, pre-registering priors will be essential.
Pre-registering statistical criteria and assumptions should already be essential, whether you're a Bayesian or not. The fact that it isn't is a key factor behind the replication crisis.