Emperical Risk

• TRAINING ERROR. Mean loss computed over training examples

• $R(f)={\mathbb{E}}_{(X,Y)\sim P(X,Y)}[l(y,f(x))]$

• $R^{emp}(h)=\frac{1}{N}{\Sigma }_{i=1}^{N}L(h(x_i),y_i)$

• joint prob distribution $P(X\in A,Y=c)$ is unknown

  • Decision Boundaries

• Learning set $\mathcal{L}$ is finite

• Need an estimator to evaluate it

  • Supervised Learning
    • Compute ${\mathcal{L}}_{train}$
    • Risk train = (1/M)(sum of loss values for (y, f(x)))
    • This is an unbiased estimator, so we can use it to approximate the optimal function f* that minimizes $\mathbb{R}$
    • This means that we find $argmi{n}_{f\in F}\hat{R}(f,{\mathcal{L}}_{T}rain)$ (out of all the possible functions)
    • $li{m}_{M\to \infty }(f_{{\mathcal{L}}_{T}rain}^{\ast })=f^{\ast }$ : converges to the fn that minimizes emprical risk
  • Ordinary least squares regression