My problem with books like this is that they have almost no connection to why Bayesian statistics is successful: Bayesian statistics provides a unified recipe to tackle complex data analysis problems. Arguably the only known unified recipe.
The Bayesian book I want should emphasize how Bayes is a recipe for studying complex problems and teach a broad range of model ingredients. Learning Bayesian statistics is about becoming fluent in describing scientific problems in probabilistic language. This requires knowing how to express and compose traditional models and build new ones based on first principles.
An unfortunate reality is that you still need to know computational methods too, but that should change soon enough.
Yes, that's exactly what the objective of this book is! I am not using computation out of necessity, but rather because I think it provides leverage for understanding the concepts, and learning to (as you say) compose traditional models and build new ones.
As the book comes along, I am finding that many ideas that are hard to explain and understand mathematically can be very easy to express computationally, especially using discrete approximations to continuous distributions.
I'd recommend using as many real examples as possible. Things like forecasting, product recommendations, topic modeling, etc. While you can conceptually explain how Bayesian statistics is a unified recipe, it's incredibly hard to have this sink in with toy problems. This is especially true since many people using traditional tools are actually using advanced methods to solve real problems, so when they start reading about urns or doors it all comes across as rather academic. That's sad because the benefit of Bayesian coherency is mostly that it leads to a highly productive mode of practical data analysis.
Definitely shoot me an email at tristan@senseplatform.com if you're interested in the computational side of this area. At Sense (http://www.senseplatform.com), we're working on making applied Bayesian analysis as amazing as it should be.
E.T. Jaynes book, "Probability Theory: the Logic of Science" may come close to what you want. It emphasize that there are rules of thought, which lead to Bayesian statistics. As such, Bayesian statistics aren't just a recipe, but the law.
Now, I can only personally vouch for the first 2 chapters, as I haven't read the rest yet.
The Bayesian book I want should emphasize how Bayes is a recipe for studying complex problems and teach a broad range of model ingredients. Learning Bayesian statistics is about becoming fluent in describing scientific problems in probabilistic language. This requires knowing how to express and compose traditional models and build new ones based on first principles.
An unfortunate reality is that you still need to know computational methods too, but that should change soon enough.