# Mutual Information

In contrastive methods, we can manipulate the data to create data entries and infer the changes …

Contrastive Predictive Coding, aka CPC, is an autoregressive model combined with InfoNCE loss1. …

Max Global Mutual Information Why not just use the global mutual information of the input and …

Mutual information is defined as $$I(X;Y) = \mathbb E_{p_{XY}} \ln \frac{P_{XY}}{P_X P_Y}.$$ In …