Cross entropy is1

$$H(p, q) = \mathbb E_{p} \left[ -\log q \right].$$

Cross entropy $H(p, q)$ can also be decomposed,

$$H(p, q) = H(p) + \operatorname{D}_{\mathrm{KL}} \left( p \parallel q \right),$$

where $H(p)$ is the and $\operatorname{D}_{\mathrm{KL}}$ is the .

Cross entropy is widely used in classification problems, e.g., 2.

Published: by ;

Lei Ma (2021). 'Cross Entropy', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/cross-entropy/.

Current Ref:

• cards/information/cross-entropy.md