Alignment and Uniformity

GROWING This note is being developed. It is readible but changes are being applied.
cards/machine-learning/embedding/alignment-and-uniformity.md Current Note ID: The unique ID of this note.
#Representation #Machine Learning #Contrastive Representation Learning

A good representation should be able to

  • separate different instances, and
  • cluster similar instances.

Wang et al proposed two concepts that matches the above two ideas, alignment and uniformity, on a hypersphere1.

From Wang et al

From Wang et al

  1. Wang T, Isola P. Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.10242 ↩︎

Planted: by ;

References:
  1. Wang2005 Wang T, Isola P. Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.10242
Similar Articles:
Batch Norm
Batch norm
Deep Autoregressive Network
DARN
Layer Norm
Layer norm
Machine as a Hologram
Tutorials on machine learning and data science productivity articles
cards/machine-learning/embedding/alignment-and-uniformity Links to:
Contrastive Model
Contrastive models learn to compare1. Contrastive use special objective functions such as [[NCE]] …

L Ma (2021). 'Alignment and Uniformity', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/machine-learning/embedding/alignment-and-uniformity/.