Training/Testing on our Data - Deep Learning with Neural Networks and TensorFlow part 7 Video Lecture | Machine Learning with Python - AI & ML

what's going on everybody and welcome to
part 7 of our deep learning with tensor
flow neural networks and Python tutorial
series in the last video what we did was
we actually created this sentiment set
dot pickle just in case we you actually
wanted to just load it from a pickle
probably this one will just straight up
use the function that created that
pickle but if your dataset was much
bigger it you would conceivably probably
just want to do this one time and never
run that process ever again but
obviously it was too big it will be too
big to store into that pickle anyway so
we created the sentiment feature sets
and now we actually want to run it
through a deep neural network so to do
this we're going to use the exact same
deep neural network we used with the M
Ness data set so I'm gonna pop over to
Python programming net and choose
machine learning this machine learning
series and for me it's the very bottom
one but when you're watching it won't be
the very bottom one so just look for the
part that is literally how the network
will run and we'll go there and then
scrolling down to the very bottom that's
where the full sample code is so for me
I just want to we're going to copy and
paste all of that over here I'm going to
say a new document and we're going to
just call this sentiment neural network
pi and then let's go ahead and open that
up and I'm going to just copy this into
here cool so there we have that now
obviously this code is kind of built for
for the eminence data not the data that
we're working with so now we kind of
need to change that in import or somehow
use the data that we want to use so
first of all we're going to comment that
out we don't need that and we're going
to comment that out we don't need that
but we are going to need a few things
first of all we need to from sentiment
I think Senate feature sets is that
right from create sentiment feature sets
create sentiment feature sets we want to
import create feature sets and labels
it's most likely it we can pull it up
though and make sure
this create feature sets and labels I'm
just going to copy and paste that just
to be absolutely certain cool now we
need all of the data and again we could
load from the pickle that we saved or we
could just do this so I'm just going to
do this because it's a relatively quick
process so whatever but if you want to
load from the pickle
by all means load from the pickle the
nodes with this kind of data you
probably want to McDouble this three
layers is probably good enough
the main constraint is almost certain to
be the data set size and we'll kind of
touch on that later on but for the most
part three layers 500 nodes that's
probably fine number of classes is not
10 anymore it's actually two batch size
can remain as 100 that's no problem but
when we get to X and wine the 784 that
was remember that was like 28 pixels x
28 pixels that's no good no more
instead we're going to say is this is
actually whatever the Len of Train X is
and it has to be like a specific example
train X because all these are identical
in size but it needs to be that one
exactly so then we'll take this copy and
that's the same we're just going to
replace that 784 as well down there okay
coming on down nothing else should need
to be changed there
we'll come on into the Train neural
network section and really nothing needs
to be changed until we get to basically
right here these two lines so these two
lines here are kind of M nough Stata
sets
where we're kind of iterating through
however many we should have and then we
kind of do this like nice next batch
function so we actually need to write
these two lines ourselves to apply to
our code so instead what we're going to
say is we're going to say we'll start
with I equals 0 this will just be a
really simple way of doing this but feel
free to do a different method if you
want none of this code that I'm going to
be doing is like optimized in any way so
feel free to make it better if you want
I won't be offended now what we're going
to say is well I is less than whatever
the length
of train X is so well I is less than the
training samples what do we want to do
well we want to take batches and what is
a batch it's like a chunk and then
another chunk or a slice and Python
lists terms we want to take a slice and
then another slice and that's how we do
a batch so what we're going to do is
while I is less than the land of Train X
we're going to say start equals whatever
that I is and we'll equal whatever I
plus the batch sizes and then what we're
going to say is rather than epoch X and
epic Y we could call them the same thing
that doesn't really make sense in this
case so we're going to say batch x
equals NP array as I'm writing this I'm
not positive we imported numpy we'll
have to check that in a second but
anyway we are working with the training
data so batch X is train X and then we
want the slice that is start to end and
then we want to do the same thing with
batch Y so paste
batch Y and then when this is against
train Y data and then we take this copy
paste over that batch X batch Y call
batch Y let's go ahead and finish this
in then I'll go up and check for the
numpy import I really don't think I have
it um so then coming down correct stays
absolutely the same nothing changes
their accuracy doesn't need to be
changed at least the definition but then
when we actually calculate it we do need
because we're using the test here so
here we just replace this with test X
and then this needs to be test Y so
let's go to the very top and grab a
numpy import numpy as NP cool all right
well let's go ahead and run this hit see
what kind of errors we gave ourselves so
let's ctrl alt T change directory to
desktop
are we in TF tuts I think
right so Python sentiment oops Python
three sentiment neural network pie I
want to truth everybody okay that's our
length of our lexicon there the 4:23
this is taking exceptionally long here
oh haha I already know what's happening
here we'll cancel that um so so here we
say I is equals zero and we're asking
the question of while I is less than the
line of tracks but then we're not
actually you know increasing I at all
ever so this is an infinite loop so
great so now what we need to do is after
all of this we're going to say I plus
equals whatever that batch size is so
batch size so here let's just run it
through a really quick quick example I
is zero so as we go through this
function we say start okay zero and is
100 then we're saying batch X is the
slice of train X data that's 0 to 100
right same for y then when we get done
we say I is now plus equals 100 so now I
is 100 start 100 and 200 the slice 100
to 200 and so on that's what's going on
we just forgot the I plus equals batch
size that's why that was hanging let's
do that one more time it's probably
getting really efficient I don't think
that one little batch no fantastic all
right so now we're running through epics
great we're improving we're only doing
ten epics we're also saying it because
you're out of town let's say epic and
then plus one here just so that looks a
little better the price you have more
than 10 epics but it's probably not
going to make a big difference okay so
the accuracy is 58.7% accurate which is
you know probably a little bit of a
letdown but also not surprising at all
right we're working with a really small
data set here and again
no networks were kind of ignored and
almost kind of looked down upon even
though nothing really I mean some things
have changed but not here like in this
traditional deep neural network
nothing's really changed in the world of
neural networks except for faster
computers or faster GPUs really and
bigger data sets to run through those
GPUs so we're going with the data set
that's like 10,000 in size is probably
not going to org I mean we realize it
worked with the MS data set and we got
pretty lucky there just because it's a
data set that really lends itself to be
trained against and optimized against
but most problems are going to require
huge data sets and pretty soon we're
going to actually have to leave the CPU
processing behind probably in this next
coming ish tutorials so in this data set
we got about 58 percent accuracy and
then what I want to have us do now is
okay so we got 10,000 features what
happens if we are samples what happens
if we increased those samples so there's
another decent sentiment data set that
has a little over I think it's one point
is actually exactly one point six
million positive and negative samples so
we're going to check out that data set
data set and feed it through the exact
same algorithm nothing is different and
see what happens to our number now again
even 1.6 million is not ideal but that's
better right we're probably you probably
would want closer to say 10 million or
something like that especially for
something like language data maybe even
more but anyway so let's go ahead and in
the next tutorial what we'll do is we'll
just kind of run through using a much
larger data set and then as we're using
that much larger data set we'll be able
to kind of cover some other pretty
important things like for example this
vector sentiment pickle that we were
able to get away with right consider
that the initial starting data size is
about one point we'll call it one point
three total megabytes if we put them all
together and what did it make well it
made a thing that was a hundred and
thirty megabytes
all right almost a hundred and forty but
basically times 100 in size to make that
that pickle there or to make to turn
into vectors so let's say if your data
set was ten gigabytes
well congratulations now you have like a
terabyte of of of data and you can't put
that into RAM so you're screwed all
right so you can't get away with this
method so instead much like batching you
have to use buffering or something like
that to get the data to come through so
we're going to cover that and then that
also brings up a couple other questions
like when you're using a bigger data set
you know training might take you a whole
day or 24 hours or it might take you a
week or something like that or even
months or more okay so what we're going
to talk about too is is saving the model
as you train it just have saving and
checkpoint files and then also finally
when you're done so far all we've done
with these models is actually test
accuracy right but at some point if you
had a higher accuracy you'd probably
actually want to use the model so we're
going to be covering that as well in the
next in the next data set and that will
probably be the end of us working with a
simple deep neural network just simply
because you're probably going to want to
employ something else other than a deep
neural network depending on what your
specific challenges at the time that's
it for now if you guys have any
questions comments concerns whatever
feel free to leave them below otherwise
till next time