Model Selection

Suppose we have a generating process that generates some numbers based on a distribution. Based on a …

Does the data agree with the model? Calculate the distance between data and model predictions. …

To measure the generalization, we define a generalization error, $$ \begin{align} \mathcal G = …

Minimum Description Length ( MDL Minimum Description Length MDL is a measure of how well a model …

The loss calculated on all the data points

The loss calculated on all the whole population

Suppose we have a model that describes the data generation process behind a dataset. The …

$$ \frac{p(\mathscr M_1|y)}{ p(\mathscr M_2|y) } = \frac{p(\mathscr M_1)}{ p(\mathscr M_2) …

BIC considers the number of parameters and the total number of data records.

FIA is a method to describe the minimum description length ( MDL Minimum Description Length MDL is …

Description of Data The measurement of complexity is based on the observation that the …

MDL is a measure of how well a model compresses data by minimizing the combined cost of the description of the model and the misfit.

$$ \mathrm{NML} = \frac{ p(y| \hat \theta(y)) }{ \int_X p( x| \hat \theta (x) ) dx } $$