
Generalized Information: A Method for Judging Machine Learning Models - johnnyb_61820
https://journals.blythinstitute.org/ojs/index.php/cbi/article/view/52
======
johnnyb_61820
Solomonoff induction has long been known to be important in machine learning,
where the shorter of two equivalent models is considered "better". This paper
extends this in three ways. First, it shows that the data itself can be
considered a model of itself. Therefore, an absolute bound for being able to
decide that you have a good model from the data is having a model that is
smaller than the data. Second, the paper provides a method to quantitatively
trade-off accuracy and model size. Third, this methodology is implementation-
agnostic, and can in theory be applied to any type of machine learning model.

