What are the pros and cons of using pymc3 vs writing a model in Stan and using pystan/rstan? It looks like the minibatch support of variational inference is one feature that pymc3 has up on Stan.
Another question, what are the practical limits of MCMC with lot? I asked a question on statsexchange a year ago [1] that had ~2000 data points, but it is analogous to a research problem I have that would have more like ~100 million data points. Would a model like the one I suggested there ever finish running?
I am far from an expert on the pros/cons between the two, but with Stan I can do systems of differential equations a lot easier. In some cases, it is infinitely easier. There are some really good examples for Stan on this, and the language structure just handles this easier: https://github.com/gregbritten/BayesianEcosystems_IAP/blob/m...
Julia also does better with differential equations in MCMC in general, in my experience compared to pymc3, but it can still be problematic. In some cases, it is most ideal to use the wrapper DiffEqBayes, but it has been somewhat abandoned in support for a neural differential equations package, DiffEqFlux. In some cases if you have a lot of data input, it simply is not possible to use DiffEqBayes. Stan allows for a lot of data input too alongside the system of differential equations, without any issue at all.
Not only that, Statistical Rethinking is based on Stan, first and foremost. Everything else (and there are a lot of renditions in other MCMC packages) is reprogrammed and reformatted to perform the same tasks as Stan does, and in a lot of cases, it just is not as nice.
>In some cases, it is most ideal to use the wrapper DiffEqBayes, but it has been somewhat abandoned in support for a neural differential equations package, DiffEqFlux. In some cases if you have a lot of data input, it simply is not possible to use DiffEqBayes. Stan allows for a lot of data input too alongside the system of differential equations, without any issue at all.
As of 2019 you should probably always be using direct Turing.jl: https://turing.ml/dev/tutorials/10-bayesiandiffeq/ . If you slap the differential equation solver into a Turing model it just works, so the wrapper was basically obsoleted when that came into fruition which had nothing to do with DiffEqFlux.jl (on the surface at least: DiffEqFlux.jl led to internal changes that made it very easy). You're then free to write any Julia code you want for handling data, so you can minibatch or whatever you want.
DiffEqBayes.jl still exists because there's no reason to break old code (I am a fan of stability in package ecosystems), and it gives a way to automatically generate Stan differential equation code which is nice for benchmarking. But since the benchmarks show Turing.jl as faster and more robust than Stan (in this context), and the wrapper is less flexible than directly using Turing.jl, at this point the documentation is very clearly pointing people to just use Turing.jl directly, like: https://diffeq.sciml.ai/stable/analysis/parameter_estimation... or https://diffeqflux.sciml.ai/dev/examples/turing_bayesian/ . So if you follow what the current documentation says to do (as of the last few years), then what is quoted is a non-issue.
Perhaps an alternative to look into: Numpyro [1] has a JAX backend so can be really fast when compiled; and it can run on GPUs. So that might be helpful for your problem with loads of data.
Another question, what are the practical limits of MCMC with lot? I asked a question on statsexchange a year ago [1] that had ~2000 data points, but it is analogous to a research problem I have that would have more like ~100 million data points. Would a model like the one I suggested there ever finish running?
[1] https://stats.stackexchange.com/questions/441250/pope-effect...