Deep Learning with Tensorflow - Deep Belief Networks Video Lecture | Tensorflow: Learning made Easy - AI & ML

Hello, and welcome!
In this video, we will provide an overview
of Deep Belief Networks, and show you how
to implement one using the TensorFlow library.
One problem with artificial neural networks
is that backpropagation can often lead to
'local minima'.
This occurs when your 'error surface' contains
multiple grooves.
During backpropagation, you may fall into
a groove that is the lowest locally, but not
overall.
Deep Belief Networks, or DBNs, solve this
problem by using an extra step called pre-training.
Pre-training is performed before backpropagation
and can lead to an error that's in the vicinity
of the final solution.
We can then use backpropagation to slowly
reduce the error rate from there.
Deep Belief Networks can be divided in two
major parts.
The first part contains multiple layers of
Restricted Boltzmann Machines in order to
pre-train our network.
The second part is a feed-forward backpropagation
network, which will further refine the results
from the RBM stack.
Let's first import the necessary Python libraries.
Before we continue, let's briefly review Restricted
Boltzmann Machines, or RBMs.
RBMs are shallow neural nets that learn to
reconstruct data in an unsupervised fashion.
An RBM takes a set of inputs and translates
them into a set of numbers.
These numbers can then be translated back
to reconstruct the inputs.
This process allows an RBM to automatically
extract meaningful features from a given input.
RBMs are composed of a visible input layer,
and a hidden layer where the features are
learned.
There are no connections between nodes in
the same layer.
Our Deep Belief Net will need to use RBMs,
so, let's declare one class for them.
For our example, we are going to use the MNIST
dataset, which is a commonly used benchmark
dataset of handwritten digits.
Now that the RBM class is created and the
MNIST Dataset's loaded in, we can start creating
the DBN.
For our example, we are going to use 3 RBMs,
one with 500 hidden units, one with 200, and
one with 50.
We are generating a deep hierarchical representation
of the training data.
Now we need to train our RBMs, which if you
recall, is the pre-training step for our network.
Now we can convert the learned representation
of the input data into a supervised prediction,
which will be a linear classifier.
Specifically, we'll use the output of the
DBN's last hidden layer to classify digits
using a shallow Neural Network.
Let's define this network now.
Now we can execute our code!
This will show the accuracy of our Deep Belief
Network for each epoch.
By now, you should understand the structure
of a Deep Belief Network, and how to implement
one using the TensorFlow library.
Thank you for watching this video.