Measurement

Measurements in machine learning

Introduction: My Knowledge Cards

Information Gain

Published:
Category: { Machine Learning } { Measurement }
Tags:
#Data
Summary: The information is a measurement of the entropy of the dataset.
Pages: 6

Gini Impurity

Published:
Category: { Machine Learning } { Measurement }
Tags:
#Data
Summary: The Gini impurity is a measurement of the impurity of a set.
Pages: 6

Population Loss

Published:
Category: { Machine Learning } { Measurement }
Summary: The loss calculated on all the whole population
Pages: 6

Empirical Loss

Published:
Category: { Machine Learning } { Measurement }
Summary: The loss calculated on all the data points
Pages: 6

Hilbert-Schmidt Independence Criterion (HSIC)

Published:
Category: { Machine Learning }
Summary: Given two kernels of the feature representations $K=k(x,x)$ and $L=l(y,y)$, HSIC is defined as12 $$ \operatorname{HSIC}(K, L) = \frac{1}{(n-1)^2} \operatorname{tr}( K H L H ), $$ where $x$, $y$ are the representations of features, $n$ is the dimension of the representation of the features, $H$ is the so-called [[centering matrix]] Centering Matrix Useful when centering a vector around its mean . We can choose different kernel functions $k$ and $l$. For example, if $k$ and $l$ are linear kernels, we have $k(x, y) = l(x, y) = x \cdot y$. In this linear case, HSIC is simply $\parallel\operatorname{cov}(x^T,y^T) \parallel^2_{\text{Frobenius}}$. Gretton A, Bousquet O, Smola A, Schölkopf B.
Pages: 6

Centered Kernel Alignment (CKA)

Published:
Category: { Machine Learning }
Summary: Centered Kernel Alignment (CKA) is a similarity metric designed to measure the similarity of between representations of features in neural networks.
Pages: 6