Conditional GAN

• Image2Image, Face Aging, Text to Image
• Generate images with certain extra conditions or attributes
• The Generator and Discriminator both receive some additional conditioning input information. This could be the class of the current image or some other property.
• add an additional input layer with values of one-hot-encoded image labels
• Adding a vector of features controls the output and guide Generator figure out what to do.
• Such a vector of features should derive from a image which encode the class(like an image of a woman or a man if we are trying to create faces of imaginary actors) or a set of specific characteristics we expect from the image (in case of imaginary actors, it could be the type of hair, eyes or complexion).
• Whereas conditional generation uses labels during training, controllable generation focuses on controlling the features that you want in the output examples.
• We can incorporate the information into the images that will be learned and also into the Z input, which is not completely random anymore.
• This can be done by adjusting the input noise vector z that is fed into the generator after it has been trained.
• We can use the same DCGANs and imposed a condition on both Generator’s and Discriminator’s inputs. The condition should be in the form of a one-hot vector version of the digit. This is associated with the image to Generator or Discriminator as real or fake.
• Typically this is done with a one-hot vector, meaning there are zeros in every position except for the position of the class we want to generate

Architecture

• [GAN Z Space](GAN Z Space.md)

The Discriminator’s network

• Discriminator’s evaluation is done not only on the similarity between fake data and original data but also on the correspondence of the fake data image to its input label (or features)
• Same as [DCGAN] except [one hot](DCGAN] except [one hot.md) vector for conditioning

The Generator’s network

• To create an image that looks as “real” as possible to fool the Discriminator.
• Same as [DCGAN] except [one hot](DCGAN] except [one hot.md) vector.
• 

Loss functions

• We need to calculate two losses for the Discriminator. The sum of the “fake” image and “real” image loss is the overall Discriminator loss. So the loss function of the Discriminator is aiming at minimizing the error of predicting real images coming from the dataset and fake images coming from the Generator given their one-hot labels.

Gen

• The loss function of the Generator minimizes the correct prediction of the Discriminator on fake images conditioned on the specified one-hot labels.
• ${\mathcal{L}}^{(G)}({\theta }^{(G)},{\theta }^{(D)})=-{\mathbb{E}}_{z}log\mathcal{D}(\mathcal{G}(z|y’))$

Disc

• has to correctly label real images which are coming from training data set as real.
• has to correctly label generated images which are coming from Generator as fake.
• ${\mathcal{L}}^{(D)}({\theta }^{(G)},{\theta }^{(D)})=-{\mathbb{E}}_{x\sim p_{data}}log\mathcal{D}(x|y)-{\mathbb{E}}_{z}log(1-\mathcal{D}(\mathcal{G}(z|y^{\prime })))$

Training

• The Discriminator is trained using real and fake data and generated
• After the Discriminator has been trained, both models are trained together.
• First, the Generator creates some new examples.
• The Discriminator’s weights are frozen, but its gradients are used in the Generator model so that the Generator can update its weights.

Training Flow

• For the Disc
• For the Gen

Challenges with Conditional generation

• Not strictly unsupervised. Needs labels
• With a conditional GAN, you get a random example from the class you specify
• With conditional generation, you have to train the GAN with labeled datasets.
• [Feature Correlationa](Feature Correlationa.md)
• [Z-Space Entanglement](Z-Space Entanglement.md)
• [Classifier Gradients](Classifier Gradients.md)

DCGAN vs CGAN

DCGAN	CGAN
Output features are not controllable	Output features can be controlled
Unsupervised	Semi-Supervised
Discriminator does not receive labels	Discriminator requires labels
Discriminator evaluates similarity between input and target images	Discriminator considers input and target images and their respective labels