KL Divergence

• Classification
• Entropy + Cross Entropy
• Distribution Based metric
• Measures difference between two PDF
• We first define xlogx for a weird edge case $x\cdot \log \left(x\right)$

Then Entropy $\mathrm{sum}\left(\mathrm{xlogx}\left(y\right)\right)\cdot //\left(1,\mathrm{size}\left(y,2\right)\right)$

Then cce as defined before $-\mathrm{sum}\left(y\cdot \log \left(\hat{y}\right)\right)$

Finally KLD $entropy+crossentropyloss$

• $KL(p,q)={\Sigma }_{x}p(x)log\frac{p(x)}{q(x)}$