Understanding Generative Adversarial Networks

Background

MNIST hand-written digit images

The Generator

generator = Sequential([
Dense(128, input_shape=(100,)),
LeakyReLU(alpha=0.01),
Dense(784),
Activation('tanh')
], name='generator')
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 128) 12928
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 128) 0
_________________________________________________________________
dense_2 (Dense) (None, 784) 101136
_________________________________________________________________
activation_1 (Activation) (None, 784) 0
=================================================================
Total params: 114,064
Trainable params: 114,064
Non-trainable params: 0
_________________________________________________________________

How to Train the Generator?

The Discriminator

discriminator = Sequential([
Dense(128, input_shape=(784,)),
LeakyReLU(alpha=0.01),
Dense(1),
Activation('sigmoid')
], name='discriminator')
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_3 (Dense) (None, 128) 100480
_________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, 128) 0
_________________________________________________________________
dense_4 (Dense) (None, 1) 129
_________________________________________________________________
activation_2 (Activation) (None, 1) 0
=================================================================
Total params: 100,609
Trainable params: 100,609
Non-trainable params: 0
_________________________________________________________________

The GAN

gan = Sequential([
generator,
discriminator
])
> gan.summary()Layer (type)                 Output Shape              Param #   
=================================================================
generator (Sequential) (None, 784) 114064
_________________________________________________________________
discriminator (Sequential) (None, 1) 100609
=================================================================
Total params: 214,673
Trainable params: 214,673
Non-trainable params: 0
_________________________________________________________________

Training the GAN means Training the Generator

The Train Loop

The Discriminator Training
The Generator Training (via GAN)

But does it actually work?

References

[1] Generative Adversarial Networks

[2] GAN MNIST Example in TensorFlow

[3] MNIST dataset

[4] How to Train a GAN? Tips and tricks to make GANs work

[5] Generative Models

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store