Shannon Entropy

#Information Theory #Entropy

Shannon entropy $S$ is the expectation of information content $I(X)=-\log \left(p\right)$1,

\begin{equation} H(p) = \mathbb E_{p}\left[ -\log \left(p\right) \right]. \end{equation}


  1. shannon_entropy_wiki Contributors to Wikimedia projects. Entropy (information theory). In: Wikipedia [Internet]. 29 Aug 2021 [cited 4 Sep 2021]. Available: https://en.wikipedia.org/wiki/Entropy_(information_theory) ↩︎

Published: by ;

Lei Ma (2021). 'Shannon Entropy', Datumorphism, 09 April. Available at: https://datumorphism.leima.is/cards/information/shannon-entropy/.

Current Ref:

  • cards/information/shannon-entropy.md