self-supervised learning

¹ An Introduction to Generative Models

#Self-supervised Learning #Generative Model #Basics

Discriminative model: The conditional probability of class label on data (posterior) $p(C_k\mid x)$ …

¹ Contrastive Model

#Self-supervised Learning #Contrastive Model #Basics

Contrastive models learn to compare1. Contrastive use special objective functions such as [[NCE]] …

¹ GAN

#Self-supervised Learning #Adversarial Model #GAN #Basics

The task of GAN is to generate features $X$ from some noise $\xi$ and class labels $Y$, $$\xi, Y \to …

² Contrastive Model: Context-Instance

#Self-supervised Learning #Contrastive Model #Mutual Information #Basics

In contrastive methods, we can manipulate the data to create data entries and infer the changes …

² f-GAN

#Self-supervised Learning #Adversarial Model #GAN #Basics

The essence of [[GAN]] GAN The task of GAN is to generate features $X$ from some noise $\xi$ and …

² Generative Model: Autoregressive Model

#Self-supervised Learning #Generative Model #Autoregressive Model #Basics

An autoregressive (AR) model is autoregressive, $$ \begin{equation} \log p_\theta (x) = \sum_{t=1}^T …

³ Contrastive Model: Instance-Instance

#Self-supervised Learning #Contrastive Model #Basics

It was discovered that the success of [[mutual information based contrastive learning]] Contrastive …

³ Generative Model: Normalizing Flow

#Self-supervised Learning #Generative Model #Normalizing Flow #Basics

Normalizing flow is a method to convert a complicated distribution $p(x)$ to a simpler distribution …

³ infoGAN

#Self-supervised Learning #Adversarial Model #GAN #Basics

In GAN, the latent space input is usually random noise, e.g., Gaussian noise. The objective of …

⁴ Contrastive Predictive Coding

#Self-supervised Learning #Contrastive Model #Mutual Information #Contrastive #Predictive Coding #Basics

Contrastive Predictive Coding, aka CPC, is an autoregressive model combined with InfoNCE loss1. …

⁴ Deep Infomax

#Self-supervised Learning #Contrastive Model #Mutual Information #Deep Infomax #Basics

Max Global Mutual Information Why not just use the global mutual information of the input and …

⁴ Generative Model: Auto-Encoder

#Self-supervised Learning #Generative Model #Auto-Encoder #Basics

Autoencoders (AE) are machines that encodes inputs into a compact latent space. The simplest …

⁵ Variational Auto-Encoder

#Self-supervised Learning #Generative Model #VAE #Basics

In an inference problem, $p(z\vert x)$, which is used to infer $z$ from $x$. $$ p(z\vert x) = …

¹⁴ Generative Models

#Self-supervised Learning #Generative Models

Generative self-supervised learning models can utilize more data

¹⁵ Contrastive Models

#Self-supervised Learning #Generative Models

Contrastive self-supervised learning models can utilize more data

¹⁶ Adversarial Models

#Self-supervised Learning #Adversarial Model #GAN

Adversarial models use a generator and discriminator