Maximum Entropy models makes least assumption about the data
Naive Bayes
Does the data agree with the model? Calculate the distance between data and model predictions. Apply …
Bayesian Linear Regression
logistics regression is a simple model for classification
Boltzmann machine is much like a spin glass model in physics. In short words, Boltzmann machine is a machine that has nodes that can take values, and the nodes are connected through some weight. It is just like any other neual nets but with complications and theoretical implications.
Introducing latent variables to Boltzmann machine and restrict the connections within groups.
Fisher information measures the second moment of the model sensitivity with respect to the parameters.
In a [[learning problem]] The Learning Problem The learning problem proposed by Vapnik:1 Given a …
Latent variable models brings us new insights on identifying the patterns of some sample data.
Suppose we have a model that describes the data generation process behind a dataset. The …
MDL is a measure of how well a model compresses data by minimizing the combined cost of the description of the model and the misfit.
$$ \mathrm{NML} = \frac{ p(y| \hat \theta(y)) }{ \int_X p( x| \hat \theta (x) ) dx } $$
Bayes’ Theorem is stated as $$ P(A\mid B) = \frac{P(B \mid A) P(A)}{P(B)} $$ $P(A\mid B)$: …