This may well be the future of scientific modeling for problems that are too hard to find complete analytic solutions to. It's often the case that most of the problem is tractable but there's a core part that's hard/impossible to analyze. Traditional reinforcement learning (RL) has allowed treating the entire process as a black box which can be "machine learned" by approximating it with a neural network. This works remarkably well, but it's inefficient because it discards everything we know about the parts of the problem that are easy to model, like basic physics, chemistry, electronics, optics—whatever domain the problem is in. Differentiable programming (∂P) now allows you to write a program that models what you do know about the problem, leaving a much smaller black box function to learn for the part that you cannot model. You can run this program with a random untrained neural net for the part of the model you don't know, look at the loss function of the result, and then differentiate (aka back-propagate) through the rest of the program (aka model) to improve the black box. This lets you efficiently approximate just the hard non-analytic core part of the model with a neural network instead of having to learn to approximate the entire problem as in traditional RL. Since you already have a good analytic model of the surrounding problem and don't have to learn those parts, this gives a huge—orders of magnitude—improvement in terms of learning speed, while also improving accuracy and reducing the risk of over-fitting.
Thanks for sharing that information about Differential Programming, I was not aware of its existence. I'm curious if this approach will help solve some problems I am investigating.
Do you have any suggestions for papers or libraries that implement Differential Programming?
If you need any guidance or help getting started, you may want to join the Julia Slack: https://slackinvite.julialang.org/. There's a very active and helpful #machine-learning channel on there.
