RNN Example in Tensorflow - Deep Learning with Neural Networks 11 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody and welcome
to part 11 of our deep learning with
Python tensorflow and neural networks
tutorial series in this part we're going
to be doing is actually implementing a
recurrent neural network and to do that
we're just going to simply modify that
deep neural network that we started with
the just simple multi-layer perceptron
model so we're going to go ahead and I'm
just going to take that code from the
machine learning series scroll down
we're looking for how the network will
run am I blind no there it is it's
probably adjust to surge how the network
will run by the way anyway at the very
bottom we have the code I'm just going
to copy that
copy it and then come down here new
folder I'm gonna call it TF cuts open
that up and create a new document and
I'll just call this RN exampled up high
and open that up and paste that code in
so now what we're going to first do is
do some modifications the first thing
I'm going to do is from tensorflow dot
python dot ups we're going to import RN
n and the RN n cell now we're going to
change some of these parameters first of
all I'm going to get rid of this here
because we're actually not going to be
working with those and then I'm going to
move the how many epics from line 44 I'm
going to move that actually it's not
line 40 for you at least for me it is
anyway I move how many epics and paste
that up at the top where everything else
is also batch size will call 128 seems
like everybody kind of honors those
those numbers like 64 128 256 512 1024
and so on so anyways we'll go ahead and
follow that now we're going to now we
get basically to X but we're going to
add some new parameters but in the past
with a typical deep neural network you
just kind of you're just going to pass
everything all at once pretty much but
with a recurrent net the whole point is
that you kind of you go in some sort of
sequence right so where images are 28 by
28 so let's just go in chunks of 28
pixels at a time so 28 chunks
of or 28 chunks of 28 pixels I suppose
anyway so we're gonna say a chunk size
is equal to 28 and then we're going to
say n chunks here we go is 28 now the
final thing that we're going to add here
is we're going to say are n in size and
that's going to be 128 for now so rather
than having all those layers we just
have the single size of the RNN which
for now is 128 which will give us
hopefully better accuracy even than our
previous model gave us but you could do
something like 512 or something like
that if you really want it but 120 it's
good enough and it'll let us run this
really quickly on video and hopefully
not mess up my camera too much anyway
uh-huh so now in the actual neural
network model let's rename it to
recurrent neural network and since we're
actually passing X anyways let's call
this X and then we're going to get rid
of these here all the way up to here
so from output all the way up to that
first dictionary press delete then
rather than hidden one layer I'm just
going to call it layer and then the
inputs and stuff let's just this in this
case the the layer weights will be
actually our an in size by the number of
classes that we have and then the biases
is just the number of classes okay and
the output is still going to be like
some sort of data times the way it's
plus bias but it's not it'll be
basically layer biases layer weights and
then this will be we'll get to that let
me just put question marks here because
we're not ready for that yet
let's going to have that data so first
we are going to take X and we are going
to say x equals x dot TF Dutch are
actually TF dot transpose
cost like something is wrong TF dot
transpose X by and the transpose will be
one zero two and let me finish this
these these next couple lines and I will
explain transpose for those who are not
familiar so we're going to transpose
we're also going to do x equals TF trees
ape and we're going to reshape X to be
negative one by chunk size chunk size
there it is and then finally X will
equal TF dot split zero number of chunks
and chunks and then number of are
actually just X yeah zero number times X
so reshape simple enough hopefully you
understand what reshape does nowadays
split you should understand that but
transpose never really talked much about
transpose so let me go ahead and
illustrate so we're going to create a
new document here you can follow along
or just watch if you want I'm gonna call
it transpose example dot Phi open that
up and I'm going to import numpy as NP
if whatever you want to test any sort of
matrix operation you can always just do
it with numpy for the most part it's
going to act the same way in tensor flow
hopefully and so what we're going to do
is like let's say X x equals NP ones and
then one one
- 3 1 by 2 by 3 and then we're going to
say print X and then we'll also print NP
transpose X and then the transpose will
be 1 0 - save that come down here open
in terminal Python 3 transpose X and
then just to make this a little more
simple to understand I'm going to copy
that paste it down here and like if we
were to format this a little bit more it
would look like this
and then this one whoops this one would
look like this ok so you see that that
ever so slight difference right we're
doing this because this is what tensor
flows aren't in cell requests of us kind
of like with scikit-learn we needed to
reshape data for scikit-learn or just
that's all we're doing here we're just
formatting the data so so tensorflow is
content with us so that is our
modifications of basically the data and
then what we're going to do is actually
pass it through so we're gonna say LST m
underscore cell and that will be equal
to our nin underscore cell basic LST m
all caps there and then capital c' cell
and that will be a recurrent cell that
or an LST m cell that will recur for the
RN n size now what we're going to do is
we're going to have outputs and then
States so every cell will have outputs
and states at each recurrence I suppose
and the way we're going to do that is by
running with our n n dot RN n the cell
type is LST M cell the data will be X
and the D type will be T f dot floats
whoops gift up float32 die lose that
print my shirt it Oh sublime okay
so now for that output layer what would
we pass through as basically that output
that final output well that would be
outputs outputs negative one so it's an
a matrix
yeah matrix multiplication of the final
output times the weights times the
biases okay then we return output and
everybody's happy so now we're just
going to pass recurrent neural network
over recurrent or rather than just the
simple neural network our cost function
can actually stay the same optimizer can
stay the same again that's with a
default learning rate which I believe is
0.01
it might be zero zero one I can't
remember but anyway we're just going to
use a default learning rate then coming
down here the major thing that we're
going to have to change is basically
right here so recall that we're passing
it through we're running the session
we're passing the X's and Y's and right
now the X is what it's that basically
784 long data it's all of the pixel data
because we're just grabbing that from
the batches right here so what we
actually need to do is reshape epoch X
so epoch X actually becomes let's see
epoch X is now going to equal epoch
underscore X dot reshape and we reshape
to be the batch size by the number of
chunks by the chunk size
okay let's I'm trying to copy this so we
need that one to be reshaped there but
then because we're reshaping that in
this exact way here where else should we
actually where else should we be
reshaping the X or the input data well
down here when we go to measure accuracy
well not where we go to measure accuracy
but right down here where we actually
calculate accuracy that input right that
input dictionary we need to reshape the
images so again under images it is going
to have to be a reshape here dot reshape
and in this case it's not it's not you
know we're not we're not reshaping the
entire batch size so in this case each
individual image so is all that we're
passing through so you're reshaping by
negative one by number of chunks because
you want to you don't need the entire
batch but you need the entire image so
chunk size so it's just that flat kind
of number of chunks by number
yeah chunk size so just that one image
at a time is why we're doing it while
we're doing it this way here as a put
okay let's just do Matt oh I can do this
why we're doing it this way here as
opposed to what we had done I'm lost
here okay so I think at this point we
actually should be able to run this
we'll see out of my way open in terminal
and let's run Python 3rn exampled up hi
it is on oh it's very unhappy
it is unhappy mostly at x equals TF
transpose X did we forget possibly okay
so I'm pretty sure my errors up here of
course
so rather than like this X input is
obviously not anymore by 784 and I
started to edit that and then I added
these here and then I completely forgot
that we need to modify that so really it
needs to be none by the number of chunks
by the chunk size itself so hopefully
that was it
I meant to change number of epochs to it
looks like it's at least gonna run for
us but I'm going to go ahead and change
epochs to three just cuz it's gonna go
home quicker sees it might take a while
so while that's running let's go ahead
and view
wait for it massive improvement
initially and then even some good
improvement right 10 epochs is really
what you ought to be using but anyway
even with only three epochs we pass it
through and we have 97% accuracy if we
just got a little bit more at we could
round up to 98 so that's actually better
than our deep neural network that we did
before but like I said if we use more
than three epochs we'll definitely be
better it should be like 98 to 99
percent if you use let's say 10 epochs
also 1:28 are an N size kind of small so
probably make that bigger as well but
just so we can really quickly get
through it I wanted to use something
smaller so if anybody has questions
comments concerns whatever on any of the
code or how we're doing something or
whatever feel free to leave them below
otherwise as always thanks for watching
thanks for all the support subscriptions
and until next time