Entropy

• Measure of information content
• $H=-{\Sigma }_{x}P(x)logP(x)={\Sigma }_{x}P(x)log\frac{1}{P(x)}$
• Units : bits of $lo{g}_2$
• Uniform Distribution maximizes entropy. Results harder to predict