Deep Learning with Tensorflow - Initializing a Restricted Boltzmann Machine

hello and welcome in this video we will
show you how to use tensorflow to build
and initialize a restricted Boltzmann
machine in preparation for the training
process restricted Boltzmann machines or
RBMS are shallow neural nets that learn
to reconstruct data through an
unsupervised learning process to do this
an RBM takes the inputs and translates
them to a set of numbers these numbers
can then be translated back to
reconstruct the inputs through several
forward and backward passes the RVM
learns which features are the most
important for detecting patterns this
feature identification is very helpful
for collaborative filtering
dimensionality reduction speech
recognition and several other
applications to begin let's first load a
file that contains different utility
functions the sole purpose of these
functions is to help process the outputs
in a way that's more understandable
after that we're going to run this cell
which loads in the necessary libraries
this includes tensorflow the main
library we'll use for the our BM we'll
also be using the written character
database m nist to train our r vm the
next cell loads that in let's now look
at the general structure of an RBM as
you can see from the image an RBM has
two layers the first layer is called the
visible layer or the input layer emne
stim ajiz have 784 pixels so the visible
layer must have 784 input nodes each
node in the first layer also has a bias
we will denote the bias as VB for the
visible units the VB value is shared
among all visible units the second layer
is the hidden layer which contains eye
neurons each hidden unit has a binary
state which we'll call s
I and turns either on or off with a
certain probability this probability is
derived from a logistic function of the
inputs it receives from the other
visible units J this is called the
probability of Si equal to 1 for our
case we use 500 nodes in the hidden
layer so I equals 500 we also have to
define weights among the input layer and
hidden layer nodes in the weight matrix
the rows are equal to the input nodes
and the columns are equal to the output
nodes so let W be the tensor that
represents the weights between neurons W
is 784 by 500 since there are 784
visible neurons and 500 hidden neurons
now we're going to see how an RBM works
in deep learning an RBM is a model that
can calculate the probability of
observing a case given some hidden or
latent values so it's a generative model
that assigns a probability to each
possible binary state vector over its
visible units the RBM has two phases the
forward pass and the backward pass which
is also known as reconstruction in the
first phase inputs from all visible
nodes are passed to all hidden nodes
this computation begins by making
stochastic decisions about whether to
transmit that input or not to determine
the state of each hidden layer the
process is very simple when using a
sigmoid activation function at the
hidden layers nodes v-0 is multiplied by
a weight and added to the H underscore
bias the result of those two operations
is fed back into the sigmoid function
which produces the nodes output or state
as a result one output is produced for
each hidden node so for each row in the
training set a tensor of probabilities
is generated which in our case is of
size 1 by 500 with 55,000 vectors 2
at all the cell below creates our input
layer and sets the sigmoid activation
function now let's look at an example of
sampling in the second phase the RBM
reconstructs data by making several
forward and backward passes between the
visible and hidden layers so this phase
has the samples from the hidden layer
play the role of the input this cell
implements the reconstruction phase by
now you should understand how to create
and initialize a restricted Boltzmann
machine using the tensor flow library
thank you for watching this video to
practice and learn more go to the lab
and run the code for yourself