Deep Learning with Tensorflow - Introduction to Autoencoders Video Lecture

Hello, and welcome!
In this video, we will cover the basic concepts
and the motivation behind autoencoders, a
type of neural network used in unsupervised
machine learning.
To quote Nikhil Bukuma: 'Autoencoders are
an extremely exciting new approach to unsupervised
learning and for many machine learning tasks
they have already surpassed the decades of
progress made by researchers handpicking features.'
An autoencoder is also known as an autoassociator
or a Diabolo network.
It's an artificial neural network that's designed
to recreate the given input.
They differ from Restricted Boltzmann Machines
because they use a deterministic approach,
rather than a stochastic approach.
An autoencoder is a type of unsupervised learning
algorithm, so it will find patterns in a dataset
by detecting the key features.
In this example here you can see how it could
detect the key features that make up a face.
Autoencoders excel in other tasks like emotion
detection, image reconstruction, and dimensionality
reduction.
High-dimensional data is a significant problem
for machine learning tasks.
In fact, data scientists commonly use the
phrase 'CURSE OF DIMENSIONALITY'.
Many common problems that we want to target
have a large number of dimensions.
Even a 256 by 256 image would have over 65
thousand dimensions, one for each pixel.
A high-resolution image from a smart phone
would be even larger.
According to a 1982 study by C.J. Stone, the
time to fit a model is, at best, the function
you see here.
So, as we increase the dimensionality, our
time to fit our model will increase exponentially.
We run into additional problems when trying
to reduce the dimensions.
If we have a small number of dimensions, our
data could overlap, resulting in a loss of
data characteristics.
If we have a huge number of dimensions, our
data will start to get sparse, which results
in an over-allocation of memory.
You can see how that looks in one dimension
and three dimensions.
Overlap and sparsity make it difficult to
determine the underlying patterns.
But with the proper number of dimensions,
the patterns are much clearer.
Take a look at the data separation outputs
for the MNIST dataset of handwritten digits.
The left output is from principal component
analysis, or PCA, a classic algorithm for
dimensionality reduction.
The output on the right is from an autoencoder.
We can see that it's easier to discern the
data with the autoencoder's output.
Here is another comparison between PCA and
an autoencoder, now applied to news stories.
The separability of the autoencoder is far
better than PCA in this case.
Since separability is important for applying
clustering algorithms, this difference in
quality is significant.
By now, you should understand that an autoencoder
can extract key image features, improve training
times of other networks, and improve the separability
of reduced datasets when compared to other
methods.
For these reasons, the autoencoder was a breakthrough
in the unsupervised learning research field.
Thank you for watching this video.