Chapter 5 - Loss Functions
- A large list of useful loss functions can be found loss
- A large list of useful distributions can be found here Distributions
- A loss function or cost function L[φ] returns a single number that describes the mismatch between the model predictions f[xi, φ] and their corresponding ground-truth outputs
Distributions
Maximum Likelihood
Log Likelihood Loss
Maximum Likelihood
Constructing New Loss Functions
Recipe for constructing loss functions
Loss for univariate regression
Loss for binary classification
Loss for multiclass classification
- Difference between the distributions : KL Divergence or Cross Entropy
For Multimodal
- Fusion loss
- Multiple attention heads : backprop from heads
Other Notes
- heteroscedastic nonlinear regression
- Robust regression
- Quantile Regression
- Focal Loss
- Van Mises distribution
- Non probabilistic approaches - Hinge Loss, AdaBoost