# Model Selection

Suppose we have a generating process that generates some numbers based on a distribution. Based on a …

Does the data agree with the model? Calculate the distance between data and model predictions. …

To measure the generalization, we define a generalization error, \begin{align} \mathcal G = … Minimum Description Length ( [[MDL]] Minimum Description Length MDL is a measure of how well a … The loss calculated on all the data points The loss calculated on all the whole population Suppose we have a model that describes the data generation process behind a dataset. The … \frac{p(\mathscr M_1|y)}{ p(\mathscr M_2|y) } = \frac{p(\mathscr M_1)}{ p(\mathscr M_2) …

BIC considers the number of parameters and the total number of data records.

FIA is a method to describe the minimum description length ( [[MDL]] Minimum Description Length MDL …

Description of Data The measurement of complexity is based on the observation that the …

MDL is a measure of how well a model compresses data by minimizing the combined cost of the description of the model and the misfit.

$$\mathrm{NML} = \frac{ p(y| \hat \theta(y)) }{ \int_X p( x| \hat \theta (x) ) dx }$$