Softmax

• Output : probabilities
• $p=\frac{1}{{\Sigma }_{i=1,..,n}{e}^{\frac{\alpha y_i}{T}}}(e^{\frac{\alpha y_1}{T}},e^{\frac{\alpha y_2}{T}},\dots ,e^{\frac{\alpha y_n}{T}}{)}^{\prime }$
• Softer argmax (0,1)
• Multinoulli
• 

Entropy

• $\alpha$ determines Entropy
• If it is 0, and Uniform Distribution and limit to infinity → binary vector which is 0 everywhere except at position i when y is maximal

Temperature

• Higher the T → Softer it the distribution. Aka less confident about distribution
• Lower → Harder. More confident
•