I found Gaussian Processes with the right kernel to be very powerful with even just a few data points and a very small set of parameters. I don't know if I was using it correctly tbh, but it worked out great in predicting values that I could not predict so accurately. I used it as a predictable yet non-linear process to tweak the input in a computer vision task. The proof was literally in the pudding.
