Word2vec

GROWING This note is being developed. It is readible but changes are being applied.
wiki/machine-learning/embedding/word2vec.md Current Note ID: The unique ID of this note.
#Machine Learning #Embedding #Word2vec

Word2vec is a word embedding model that learns the probability of some words being neighbours in a sentence $p_{neighbours}(w_i, w_o)$.

  1. Build a dataset of adjacent words. CBOW; skipgram; negative sampling;
  2. Encode the words using vectors.
  3. Build a model $f(\{\theta_i\})$ to calculate the probability of the words being neighours and improve the parameters $\{\theta_i\}$ using the dataset.

Planted: by ;

References:
  1. Word2Vec Tutorial - The Skip-Gram Model
  2. The Illustrated Word2vec
  3. On word embeddings - Part 1
Dynamic Backlinks to wiki/machine-learning/embedding/word2vec:
Improving Document Ranking with Dual Word Embeddings
Word2vec produces two embedding spaces, the in-embedding and out-embedding.
wiki/machine-learning/embedding/word2vec Links to:
CBOW: Continuous Bag of Words
Use the context to predict the center word
skipgram: Continuous skip-gram
Use the center word to predict the context
Negative Sampling
negative sampling makes the calculations faster

L Ma (2019). 'Word2vec', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/wiki/machine-learning/embedding/word2vec/.