Mutual information is not trivial or even possible to estimate in many practical situations as far as i know. Example applications in robotics or computer vision, where mutual information would be useful are segmentation and denoising of unordered 3d point data, for example.
Yes, as someone mentioned above, the problem is getting the underlying distribution of the data, so you can measure SUM p_i log(p_i); this usually involves some binning, which can be tricky (and yes, I know the formula I gave is entropy not MI)
I try to remind myself that "it is just a model", as a corollary to "all models are wrong, some are useful." You are never dealing with the real world. And you are usually trying to estimate some future as of yet unobserved signal based on existing data. In other words, if your bins are reasonable and reasonably usefully accurate, you can build a working if not perfect system.
Don't try to optimize testing error performance to a value lower than the irreducible error in the system.
Even with binning, the problem is one of accurate sampling from an unknown probability distribution.
Biased samples produce biased results and this OP correlation coefficient might be sensitive to such an issue.
In one of the projects we were assuming gamma distribution (for speech processing) and sampling that is notoriously hard. Trying to use binned MI produced serious errors, as opposed to Minimim MSE one, even Maximum Likelihood did better (if noisy).
I'm not sure I understand how binning applies in e.g. segmentation of point clouds into distinct objects. The data would likely contain a mix of unknown distributions, partially observed (due to occlusions) and not easily parametrized (chair, table, toaster, etc.)... Locally you can find planar patches though, so correlation can still be useful.
