Training - Using Convolutional Neural Network to Identify Dogs vs Cats

what's going on everybody welcome to
part three of the dogs vs. cats
classification competition what we're
going to be doing now is we've already
defined our network now we're going to
write some extra code do to be kind of
stuffing of values into the variables of
X and Y for feeder sets labels and then
run the network so let's get started so
the first thing we're going to do as I
kind of write this I'm also kind of
writing it in terms I still think in
terms of your writing everything in one
big program so so the next step in my
personal program because you know f5
you're going to rerun everything at this
step you would before you actually
bother training a network you would
probably want to ask if OS dot dot
exists and then if the meta file already
exists for format the model name this
means you saved a check point so you've
already trained for some number of
epochs so you've already got a network
you know the weights of that network
already trained to some degree so you
know that this way you can actually save
your progress as you go so you can model
dot load model name and then we'll just
go ahead and print model loaded if that
happens but let's see this one this one
might actually have let's do video run
that so down here cool so now because
most of you hopefully don't have a model
already that crazy Fritz you're from the
future
now it we don't have a model but that's
okay but at this point we do have our
training data which came from here right
and recall that's just a big basically
list I think there wasn't did we convert
so this one actually can be a list right
just this this training data is a list
one of the elements actually both the
elements inside that list is our an
umpire arrangement anyway continuing on
what we want to do is separate this out
into Train
testing data with the accuracy of our
models so we're going to say training
equals and we'll just say trained data
and you know you can do whatever you
want I'm going to say 500 so basically
what this means is the training data
will be all but the last 500 sampled
data and then we're going to say the
test data is the Train data minus 500
onward and this will be this will be
out-of-sample for our model in terms of
testing purposes for accuracy and all
that but this is labeled labeled data so
this isn't what we're necessarily
competing with but in theory we should
get the same accuracy on this data as we
should get on the actual train or the
actual testing data from kaggle cool now
we're going to do is actually separate
this stuff out reshape it get ready for
our for TF learn so we've got X needs to
be something and then we've also got one
X will be our feature sets of why are
our labels so what do we want X to be
it's going to be the numpy array I like
to just keep converting things to them
pi raises because it must be an empire
array so this might be a little
redundant but that's alright we're going
to say I the zeroth element for I in
train okay so basically train recall
consists of the data and the labels
itself so x and y i 0 the 0th element
was the image data so this is the image
data
pixel data and then outside of that
array we also want to dot reshape a
negative 1 image size by image size 5 1
all right now Y will just be this one
that is acceptable to
just be i like tight high one for i and
time okay now we're going to have test X
and test Y with this I'm going to copy
this and down here paste call this test
X call this test y all lowercase this
time rather than train it's test copy
did I really this needs to it Lissy what
have I done this needs to be trained not
test what was I thinking
all right let me make sure I didn't make
another mistake sir yeah that's the
training wide out of that this is what's
getting fit this is from testing
accuracy this must not be do not make
that mistake that you'll get in there if
you did but anyway that would be
annoying all the tensorflow errors are
like you know 22 pages of just
disgusting error so it just it stinks
every time you hit one I think that's
good that looks about right in my eyes
well know what we'll see the you know
the novel of error there it is I didn't
close whatever
oh yeah cool all right so now let's go
ahead and train the network so I dad it
copied but I don't anymore I'm going to
just recopy so I'm going to take this
line just that model dot fit just put on
to type all this stuff out paste input X
Y that's all fine let's just do three
epochs what I need to do that many tests
X test Y that's actually the same such
behind show metric yes please
snapshot step we pride go less than five
or greater than 500 but I'll just leave
that for now and this that's not what it
should be called we're going to call the
run ID whatever the model name so run ID
is what we're how we're going to find it
in tension board okay so let's go ahead
and in effect because we're going to use
tensor board let's go ahead and do five
ethics just so I can show you attention
before
let's go open where's looks good so far
and we're off okay so first epoch we can
see loss here lost didn't appear to
really do much we'll continue waiting
here also wins the accuracy over here
doesn't appear to be doing much in fact
losses losses going up there go maybe a
little better all right so now the next
thing that I want to show you fine
individuals is tensor board so while we
wait I think what I'll do because I want
to show you tensor board once this is
kind of done so so I'm gonna oh we like
if I'm it how I was going to have a
student more epic epics
oh well fine so the tensor board in fact
actually I'm still going to pause so I
can make sure I get the right tensor
boards link it's like tensor board - -
tense order and then you got to do like
foo you know give a name anyway
really pause for just a second okay so
so this will be like the command that
you'll run and make sure gets based on a
taste if you're on if you're on Linux
I'm pretty sure you just either you know
not even sure you need the lot I think
you do need to log dirt flag but
generally by default TF learn we'll just
do I think /tmp logs or logs I don't
know I can't remember which but anyway
if you specified log like I did you
should be able to locally if you run
tensor board locally I think you can get
away with a local but on Windows for
sure you'll want to give the full path
to it so it just depends on where you
are but you can never go wrong with the
full path so if you're having problems
on some other operating system give the
full path the other thing you need on
Windows is to give it this little name
right you can go with foo or whatever
that you want to go if I'm going to keep
it foo but it doesn't matter but on like
Linux you just just just give the full
path but for some reason Windows who
really wants this
so or the name but anyway copy that and
then let me just open up a just
hold there we go
shift command XE paste that in run it
and then it tells you tensor boards
running at your local address port 6006
so I'll just cut that or you can click
it now can't click it and then once you
get here basically we can for now I
forget what its default is I think it
defaults like zero you can record a lot
of variables and stuff as you go but for
now we're just we're just doing the
basic stuff so we can see here over our
short time of training we basically this
was our accuracy we pretty much did
nothing but maybe even go down slightly
like we can add a lot of smoothing here
and I mean we basically stay like 50% so
not very good actors to television yeah
ok so we can see that we can see our
atom compared to blossom raw no good no
good they're lost we want to really see
loss going down instead it kind of
started coming up almost which is kind
of weird
so nothing good nothing good happened
but so now I want to show you guys the
power of neural nets and why neural nets
are such a big deal lately within the
last few years because normal networks
have been around for a very long time a
lot of concepts for neural networks have
been around for a very long time but
what gives the neural networks power
well just the size so let's keep I'm
going to keep the image 50 the learning
rate arguably a larger learning rate
takes longer to train so I think of it
this way a really small learning rate or
a large learning rate let's say I think
if small in terms of you know the actual
number that we use here but actually
smaller this number the larger the
learning rate so a super small learning
rate is it's not going to like a plus
let's start with a large learning rate a
large learning rate is more likely to
miss
miss the best objective a super small
learning rate is either going to confine
itself in a local Maxima or minima or
whatever or it can just take a really
long time so so yeah 20 negative 3
that's that's fine ish probably some
networks could go all the way up top 5
but we should be fine I'm going to leave
the image size and learning rate the
same what I am going to change is the
number of layers let's do let's do 6
convolution layers now just for the
record one convolution layer could
should be able to classify a linear
problem to neural network layers and in
fact we've only really have three
Network layers we've got two convolution
one fully connected so there's actually
a three layer neural net one layer is
for linear problems two layer nonlinear
problems add more from there it's
they're still all nonlinear and this was
capable of correctly classified like 99%
of handwritten digits at a resolution of
28 by 28 so the question is what happens
when we just let us copy this and I'm
going to paste and paste what we didn't
leave it as is actually cool so all
we've done is now added we're six six
cons layers and just just kind of
arbitrary but I'm going to say that and
then actually let's see I think the
other thing I need to do is we need to
reset the graph so let me actually pull
up
it's like uh I think about should bring
in tensorflow
yeah so actually probably what I'm going
to do I think where I could put push
this in we could just push it in right
here so what I'm going to do is import
tensorflow as TF and then TF reset
underscore default underscore graph I
could also reset the kernel that would
work I think but we'll just reset the
default graph basically what happened
what happens is there's still grass kind
of because this is this notebook is
still operating so that in that graph is
still kind of happening so we need to
reset it so I'm going to go ahead and
reset that hopefully that will run looks
like it did great
hopefully we don't have that model still
because we shouldn't try and test full
fit hopefully I don't get in there if we
do I'll restart the Chromebook okay
so already we can see we are definitely
we at least made some gains through and
the next epoch made some more gains epic
5 is to sorry made it some slight gains
also we can come into tensor board
hopefully we can just refresh sense your
board I'll make you mad a little bit no
I don't see the other one just yet let's
see did we I don't think I did anything
wrong there hold on let me see here I
might need it is restarted I'm just
going to restart tangibles it should
have already been there excessive both
be in the logs it's not there Lucy if it
went to the logs
oh it's also not logged are we over
writing this one wow man
didn't I change the name did I not save
the changes name how my son is favored
to change no I did I don't know why
we're seeing this okay Oh Molly where
did we save what the wall name would be
right down here right no out there where
is it they'll fit right yeah what are
you doing me like this haha okay well
you know what I hate to do this well you
should get the idea this is this got
kind of messy and cut bummed I don't
want to show this because this isn't
quite right because it's like distorting
everything let me uh I'm just going to
restart everything and then I'll pause
restart and run all because it's gonna
have to try it's gonna process all oh we
should just mode it if I would have just
done Connor
anyway oh alright it's training again
I've got plenty of things to say you
know it'll be done before I'm done
yakking so yeah so what I what ahead and
did is just so it doesn't happen again
we get his load to the file that we
saved so now if we restart the kernel
should go much quicker anyway training
is going pretty well not bad not bad not
bad at all we ended up with apes and
angrily only five epics and again all we
did was add more layers we just copy
pasted snorkel here and that's it so now
coming over to tensor board still not to
itch it there is beautiful beautiful
stupid the other ones there extend we
could have compared them and been like
hey look you know here's an example but
as you can see it's got the other one
there but anyway this looks much better
accuracy is rising over time it does
seem to fall off there but I wager if we
kept going up right
so yeah we've got that Adam lost of
raw's going down as it should and then
finally lost hopefully going down
sure enough good so these are
good-looking metrics we could probably
keep going basically once lost kind of
flattens out or a philosophy it's 2-0
but it's lost gets to zero changes are
you over fit but eventually it'll kind
of level off
same thing with accuracy eventually to
level off if it gets to a perfect 1.0
yell over fit so okay so size matters
now the last thing I'm going to do
before I we leave and go to the next one
is you know you might be happy with this
so if like for example if you're happy
with this you could say model dot fav
model name don't worry about this this
is the model saved so then later what
you you can say instead is you can do a
model dot load which we've already got
up here so we could do model loaded okay
so now we could we could retrain this so
that models been loaded for example
let's run it for another five epochs so
actually we should be able just come
down here we might need to do that this
is probably gonna give us an error real
estate nope it's still going fascinating
is that gives you an error we're either
restart the kernel or do the potential
flow reset that we did somewhere around
yet here it is these lumps eventually
it's going to give you an air anyway so
as we can see accuracy is actually kind
of kind of improving we're receiving to
probably level out here around the low
80s but we'll see how it does as soon as
we continue here also notice how it
actually in this case it actually knows
what epoch we're actually adding epochs
that's pretty interesting I don't think
I've noticed that but it does that what
I did special this time for it to know
what epoch we were on usually it'll
restart the epochs I'm impressed
I'm very impressed
okay so we can see the new updated data
we still haven't really leveled out we
could probably keep going and keep
making improvements but instead what I
want to do now is in the next video show
you all what we can do how do we
actually okay we've got the model how do
we use it and then also submit data to
Chicago so anyways that's we're going to
doing in the next video if you have any
questions comments concerns or whatever
feel free to leave them below otherwise
I'll see you then I see you