Recipe for constructing loss functions

• Using Maximum Likelihood
• For training data $x_i,y_i$
  • Choose a probability distribution $Pry|\theta$ defined over the domain of the predictions y with distribution parameters $\theta$
  • Choose an ML model $f|x,\varphi |$ where $\theta =f|x,\varphi |$ and $Pr(y|\theta )=Pr(y|f|x,\varphi |)$
  • Training → Find the parameters $\varphi$ that minimize the Negative Log Likelihood over the training data $x_i,y_i$
  • Inference → Either return $Pr(y|f[x,\hat{\varphi }])$ or the value where this distribution is minimized
• If data is differently distributed and there is no loss associated, just transform the distribution beforehand