Basic GAN

• Generative Adversarial Networks
• Learn a prob distribution directly from data generated by that distribution
• no need for any Markov Chain or unrolled approximate inference networks during either training or generation of samples
• Adversarial Learning
• 
• Min Max game $ma{x}_{D}mi{n}_{G}V(G,D)$ where $V(G,D)={\mathbb{E}}_{p_{data}(x)}logD(x)+{\mathbb{E}}_{p_{data}(x)}log(1-D(x))$
• G : Gradient Descent
• D : Gradient Ascent
• Discriminator Loss (Given Generator)
  • $L_{disc}(D_{\theta })=-\frac{1}{2}({\mathbb{E}}_{x\sim p_{real}(x)}[log(D_{theta}(x))]+{\mathbb{E}}_{x\sim p_{latent}(x)}[log(1-D_{\theta }(G_{\varphi }(z)))])$
• Generator Loss (Given Discriminator)
  • $L_{gen}(G_{\varphi })=-{\mathbb{E}}_{z\sim p_{latent}(z)}[log(D_{\theta }(G_{\varphi }(z)))]$
  • This is low if Discriminator is fooled by Gen, $D_{\theta }(x_{gen})\approx 1$

Training

• pick mini batch of samples 
• update discriminator with Gradient Descent based** on discriminator loss with generator obtained from previous update
• update the generator with Gradient Descent based on generator loss with the discriminator from the previous step

Issues

• Mode Collapse