> Source code for every dependency would be needed too.
Yes, reproducibility requires source code for the entire dependency graph, down to a set of bootstrap binaries. This is what tools such as GNU Guix and Nix allow you to do. They enable reproducible science.
I don't think the effort needed for this is worth it. Most scientists even ones who code, don't even know what bootstrap binaries are.
My original point is that the complexity of dealing with differing environments and code versions, combined with the sensitivity of floating point results on the exact source code and environment, practically kills the possibility for exact reproducibility.
That's why I'm claiming that a better goal is approximate reproducibility.
In a way we're arguing past each other: you're saying that approximate reproducibility is realistically the best scientists can do with today's toolchains. I'm saying that programmer's writing tomorrow's toolchains should do better. It seems at least possible that we could both be right :-)
Yes, I agree that the issue is in the toolchain. And more reproducibility in this area is a much more general and important problem. Even without aiming for bitwise reproducibility, it's still very hard to properly package your software and its dependencies. That is one reason why reproduction is so rare: simply installing the dependencies is too hard.
So a sufficient improvement in toolchains/package management etc. would probably enable bitwise reproducibility, but I think a lot of people in this thread, and the original article, were implying that current tools are sufficient and scientists just need to stop being lazy.
Yes, reproducibility requires source code for the entire dependency graph, down to a set of bootstrap binaries. This is what tools such as GNU Guix and Nix allow you to do. They enable reproducible science.