AutoEncoder

• 
• Regression by predicting a reconstruction of the data
• Encoder $E:\mathcal{X}\to \mathcal{F}$
• Decoder $\mathcal{F}\to \mathcal{D}$
• $$E_\theta, D_\theta = argmin_{E_\theta, D\theta}||X-D(E(X))||^2$
  • Learn using Gradient Descent
• Compressed rep of data → Good for Classification or Regression
• MSE : Unsupervised

Difficulties

• dim $\mathcal{F}<\mathcal{X}$
  • Cannot learn the identity function
• usages
  • data compression / Dimensionality Reduction
  • encoder to obtain features (use the latent variable as feature)
  • denoising autoencoders
    • input noisy image and try to obtain image without noise
  • sparse auto-encoder
  • contractive autoencoder

Types

• Denoising Autoencoder
• VAE