Neural Network

Maximum Entropy models makes least assumption about the data

Introducing latent variables to Boltzmann machine and restrict the connections within groups.

Minimum Description Length ( [[MDL]] Minimum Description Length MDL is a measure of how well a model …

Initialize a neural network is important for the training and performance. Some initializations simply don't work, some will degrade the performance of the model. We should choose wisely.

For numerical stability we can use the log-sum-exp trick to calculate some loss such as cross entropy

Artificial neuron that separates the state space

Rectified Linear Unit, aka ReLu, and its properties

Using a newly defined loss function the authors could implement an idea that achieves the multi-task …