cards/information/coding-theory-concepts.md

Maximum Entropy models makes least assumption about the data

Introducing latent variables to Boltzmann machine and restrict the connections within groups.

Minimum Description Length ( [[MDL]] Minimum Description Length MDL is a measure of how well a model …

The Fraser information is $$ I_F(\theta) = \int g(X) \ln f(X;\theta) , \mathrm d X. $$ When …

The information is a measurement of the entropy of the dataset.