Lp Regularization

• Tikhonov
• Penalty considering weights
• 
• $L^{\ast }(\theta )=L(\theta )+\lambda {\Sigma }_i|{\theta }_i{|}^p$
• Frobenius norm
• Weight Decay
• p = 2
• Encourages optimization Trajectory perpendicular to isocurves
• 
• Tune $\lambda$
• Lasso
  • p = 1
  • Sparse
  • With linear model : feature selection
• Grid search : log scale
• Too large : underfit, too small : overfit
• Cross Validation required