Freedom

• (N, D, P) N samples, D Degrees of Freedom
• If N<D , then ill posed
• Need N >> D
• If P learnable params , $P<N$ : underspecified
• If $P>>N$ : overparameterized
• No of params not a good indicator of overfitting
• Solution : Regularization