Maximum Entropy models makes least assumption about the data
Introducing latent variables to Boltzmann machine and restrict the connections within groups.
Minimum Description Length ( MDL Minimum Description Length MDL is a measure of how well a model …
Initialize a neural network is important for the training and performance. Some initializations simply don't work, some will degrade the performance of the model. We should choose wisely.
For numerical stability we can use the log-sum-exp trick to calculate some loss such as cross entropy