Maximum Entropy models makes least assumption about the data

Introducing latent variables to Boltzmann machine and restrict the connections within groups.

Minimum Description Length ( [[MDL]] Minimum Description Length MDL is a measure of how well a model …

Initialize a neural network is important for the training and performance. Some initializations simply don't work, some will degrade the performance of the model. We should choose wisely.

For numerical stability we can use the log-sum-exp trick to calculate some loss such as cross entropy

Artificial neuron that separates the state space

Connected perceptrons

BiPolar sigmoid function and its properties

Conic Section Function and its properties

ELU and its properties

Tanh function and its properties

Leaky ReLu and its properties

Radial Basis Function function and its properties

Rectified Linear Unit, aka ReLu, and its properties

Swish and its properties

Uni-polar sigmoid function and its properties

Using a newly defined loss function the authors could implement an idea that achieves the multi-task …