Statistics

Knowledge snippets about statistics

Introduction: My Knowledge Cards

Bayesian Information Criterion

Published:
Summary: BIC considers the number of parameters and the total number of data records.
Pages: 31

Bayes Factors

Published:
Summary: $$ \frac{p(\mathscr M_1|y)}{ p(\mathscr M_2|y) } = \frac{p(\mathscr M_1)}{ p(\mathscr M_2) }\frac{p(y|\mathscr M_1)}{ p(y|\mathscr M_2) } $$ Bayes factor $$ \mathrm{BF_{12}} = \frac{m(y|\mathscr M_1)}{m(y|\mathscr M_2)} $$ $\mathrm{BF_{12}}$: how many time more likely is model $\mathscr M_1$ than $\mathscr M_2$.
Pages: 31

Akaike Information Criterion

Published:
References: - Akaike Information Criterion @ Wikipedia - Vandekerckhove, J., & Matzke, D. (2015). Model comparison and the principle of parsimony. Oxford Library of Psychology.
Summary: Suppose we have a model that describes the data generation process behind a dataset. The distribution by the model is denoted as $\hat f$. The actual data generation process is described by a distribution $f$. We ask the question: How good is the approximation using $\hat f$? To be more precise, how much information is lost if we use our model dist $\hat f$ to substitute the actual data generation distribution $f$? AIC defines this information loss as $$ \mathrm{AIC} = - 2 \ln p(y|\hat\theta) + 2k $$ $y$: data set $\hat\theta$: parameter of the model that is estimated by maximum-likelihood $\ln p(y|\hat\theta)$: log maximum likelihood (the goodness-of-fit) $k$: number of adjustable model params; $+2k$ is then a penalty.
Pages: 31

Reparametrization in Expectation Sampling

Published:
Category: { statistics }
Summary: Reparametrize the sampling distribution to simplify the sampling
Pages: 31

Explained Variation

Published:
Category: { statistics }
References: - Explained variation
Summary: Using [[Fraser information]] Fraser Information The Fraser information is $$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ When comparing two models, $\theta_0$ and $\theta_1$, the information gain is $$ \propto (F(\theta_1) - F(\theta_0)). $$ The Fraser information is closed related to [[Fisher information]] Fisher Information Fisher information measures the second moment of the model sensitivity with respect to the parameters. , Shannon information, and [[Kullback information]] KL Divergence Kullback–Leibler divergence indicates … , we can define a relative information gain by a model $$ \rho_C ^2 = 1 - \frac{ \exp( - 2 F(\theta_1) ) }{ \exp( - 2 F(\theta_0) ) }, $$
Pages: 31

Likelihood

Published:
Category: { Statistics }
Summary: Likelihood is not necessarily a pdf
Pages: 31

Box-Cox Transformation

Published:
Category: { statistics }
Summary: The Box-Cox transformation transforms data into Gaussian data, which is especially useful in feature engineering, e.g., fixing irregularities in variances of a time series.
Pages: 31

Multivariate Normal Distribution

Published:
Category: { Distributions }
Summary: Multivariate Gaussian distribution
Pages: 31

Copula

Published:
Category: { statistics }
Summary: Given two uniform marginals, we can apply the inverse cdf of a continuous distribution to form a new joint distribution. Some examples in this notebook. Uniform marginals [[Gaussian]] Multivariate Normal Distribution Multivariate Gaussian distribution copula: Normal, Normal Some other examples: [[Normal]] Normal Distribution Gaussian distribution and [[Beta]] Beta Distribution Beta Distribution Interact Alpha Beta mode ((beta_mode)) median ((beta_median)) mean ((beta_mean)) ((makeGraph)) : Normal, Beta Gumbel and [[Beta]] Beta Distribution Beta Distribution Interact Alpha Beta mode ((beta_mode)) median ((beta_median)) mean ((beta_mean)) ((makeGraph)) : Gumbel, Beta [[t distribution]] t Distribution t distribution : t, t
Pages: 31

2 t Distribution

Published:
Category: { Distributions }
References: -
Summary: t distribution
Pages: 31

1 Normal Distribution

Published:
Category: { Distributions }
Summary: Gaussian distribution
Pages: 31