Model Comparison

#AIC #BIC

The parsimony model comes from the idea of Occam’s razor: We choose the simple model that has more explanatory power.

The instance theory is a good model to explain the lexical decision task but it is not the only one. However, it simply makes it popular.

What is a Good Model?

A good model should be presumably

  • plausibility
  • balance of parsimony and goodness-of-fit
  • coherence of the underlying assumptions
    • easy to understand when it breaks down
  • consistency with known results
    • especially with the simple and basic phenomena
  • ability to explain rather than describe data
  • extent to which model predictions can be falsified through experiments.

How to choose a model?

It takes some thinking and calculations to choose a model.

Compare Models

Many methods deals with the balance between parsimony and goodness-of-fit

  • Information criteria: AIC and BIC
  • Minumum description length
  • Bayes factors

Information Criteria: IC

We calculate the IC of all the models at hand, and specify the delta

$$ \Delta _i = \mathrm{IC}_i - \operatorname{min} \mathrm{IC} $$

calculate the weights of models

$$ w_i = \frac{ \exp{-\Delta_i/2} }{ \sum_{m=1}^M \exp{-\Delta_m/2} } $$

We prefer the model with larger weight $w_i$.

If we use AIC for IC in the formula, this weight $w_i$ is called Akaike weight; If we use BIC, the weight $w_i$ is called Schwarz weight.

MDL

Fisher Information Approximation is one of the methods to determine the minimum description length.

Bayes Factors

Bayes factors

Published: by ;

Current Ref:

  • wiki/model-selection/model-selection.md