Contrastive Model: Instance-Instance

It was discovered that the success of [[mutual information based contrastive learning]] Contrastive Model: Context-Instance In contrastive methods, we can manipulate the data to create data entries and infer the changes using a model. These methods are models that “predict relative position”1. Common tricks are shuffling image sections like jigsaw, and rotate the image. We can also adjust the model to discriminate the similarities and differences. For example, to generate contrast, we can also use [[Mutual Information]] Mutual Information Mutual information is defined as $$ I(X;Y) = \mathbb E_{p_{XY}} … is more related to the encoder architecture and the negative sampling strategy1. Instance-instance method is more direct in solving the contrastive problem. It take the instance itself directly and make comparisons for discrimination.

Cluster Discrimination

Illustration of Deep Cluster based on Liu2020.

Illustration of Deep Cluster based on Liu2020.

Instance Discrimination

There are two interesting models under the umbrella of instance discrimination,

  • MoCo, and
  • SimCLR.
Illustration of MoCo based on Liu2020.

Illustration of MoCo based on Liu2020.

Illustration of SimCLR based on Liu2020.

Illustration of SimCLR based on Liu2020.


  1. Liu2020 Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218  ↩︎

Planted: by ;

L Ma (2021). 'Contrastive Model: Instance-Instance', Datumorphism, 08 April. Available at: https://datumorphism.leima.is/wiki/machine-learning/contrastive-models/instance-instance/.