Training Model - Training a neural network to play a game with TensorFlow and Open AI p.3 Video Lecture | Machine Learning with Python - AI & ML

what's going on everybody welcome to
part three of playing games with open a
Python tensorflow and all that where we
left off we created our training data
and now what we want to do is actually
create a neural network model and train
a model based on that training data to
do this we're going to be using
tensorflow if you're not sure yeah we're
going to be an intensive flow we're also
going to be using TF learn if you want
to learn more you can go to PI the per
minute to go to search TF learn and
it'll be the first and only option
anyway you can learn about that I
thought I was going to be able to just
copy and paste from this tutorial but I
forgot this tutorial is doing a
convolutional neural network and many
times when you're working with open AI
you're going to actually be getting all
the pixel data from the game not this
one doesn't return the pixel data from
the game there might be a way to acquire
it I don't know but what it returns for
you in the observation is actually just
like the pole position court position
and so on anyway so we're not going to
be using a convolutional neural network
so I can't copy and paste from that
bummer anyway
it won't take too long because there's a
lot of copy and pasting anyway when you
created a model let's get started so
we're gonna come down here and I'm going
to create a new function we're going to
call this define neural neural network
model and it's going to take an input
size why does it take input size you
might ask
well I'll answer that basically in this
case we can actually train and test this
in like I don't know well we can train
it in like five seconds or something and
then test it you know however long
however many tests won't run but we can
test it headless Li instantly too so but
in many many times when you're going to
really create models you might have to
train them for like days and then you
might acquire more data and you'll
retrain them so with tensorflow you can
save your model but to load a model
that's been saved you actually have to
have a model already defined and it
needs to be the identical shape to the
model you're loading so generally you're
going to want to separate out the model
and then the training of that model and
the you know using of that model and all
that let's define the model first like
that your input layer
so it's going to be know if it starts
out as just being the input data the
[Music]
shape of that data not going to be a
string it's going to be done by whatever
the input size is in this case that's
its for its it's coming from the
observation but we're going to try to
keep things as dynamic as possible so
later you could add you could try this
on different games so there you go and
then we're going to say name equals
input now we're going to do is create
our fully connected layers so we say
network equals the league and next if it
takes the input is network it's going to
be 128 nodes on that layer any
activation function will be rel you for
rectified linear okay then we're going
to do the drop out so we're gonna say
this process you might not be necessary
on this one but later on I think it'll
be proved to be useful anyway
drop out network 0.8 which is kind of
weird this is actually to keep rate so
yes we're going to do drop out on the
network but actually it's keep rate
rather than the dropout rate which is
kind of odd if you ask me anyway cool
copy maybe I'm wrong maybe it's actually
a percent drop out 2 3 4 5 we're going
to do 5 layers we probably could
actually get away with all 5 layers
named 128 but I'm going to make it a
pretty tree and I'm going to say a 256
512 256 128 beautiful don't you see it
beautiful so also if you're on this
should fit on I don't know what the
memory is of this network to be honest I
think anybody could run it on just about
any GPU if you're on your CPU either
typical Ram definitely but if you're
getting an error like failed to do
something with 0 0 m4 you know memory
basically do run out of memory so if
you're hitting errors change these make
them different sized smaller now once
we've done that now we need to finally
finish we're going to say network evils
this will be the output layer fully
connected network it takes two outputs
again like I said we actually probably
could get with this example one output
because in theory it's either on or off
and that could it be left or right it's
just a binary but let's go ahead and go
with two and later on like I said like
your big this is hard-coded so you're
going to have to change this
whereas if what we could do to make this
truly like you just change the
environment here and you're good to go I
guess you'd have to change the score
requirement too but you can make the
score requirement like the top 80% of
scores or something like that and that
would probably work but anyway you know
you could change when we do the random
take that line and replace this line
with that one and then later get all the
unique values for actions whatever that
number is that's how big your one-hot
array should be and then you could
actually down here make that dispatch
whatever number that is and then you
really could change the environments I'm
not going to try to do that here but I
think that's all you'd have to change at
this point anyway the activation in here
is actually going to be used softmax
I've got all this stuff you can you can
tweak if you weren't in this case it
should all work this is your typical
neural network but as you get into more
complex games and more dimensions and
stuff like that like right now we're
just moving left to right studi game so
super simple let things get more complex
you have to pry tweak things anyway that
was now we're going to do regression
against the network the optimizer is
Adam again you can do something else if
you wanted but Adams fine the learning
rate will be whatever we set one a
negative three is what we did loss will
be categorical broad and true P so put
it inside with that and then finally
name equals where did it the real
targets all right hopefully no typos in
that function now we're just going to
say model equals TF learned CNN deep
neural networks
on the network and then if you want you
can do tension border and you can save
blog if you're on Linux it'll just go to
slash temp on Windows this should I
think of creating directory for us if
you wanna learn more about actually
doing something with the log dirt on
tension board you can check out the
convolutional neural network with tangle
cats and dogs you should be able to
apply some programming at net let me
just search for it real quick make sure
this yes being Goa Python program to net
literally go to search tangle and then
do the class mine cats versus dogs maybe
there we go yeah so on this part we
talked a little bit about tension born
and all that not a lot but it shows how
you can play with centerboard
anyway I'll move this aside now okay and
then whenever we're done we returns the
model so this hasn't been trained it's
just just the model so now let's go
ahead and train the model so the define
train the model is going to take some
training data and it's going to take
load can take some treaty data and that
is going to take a model in theory it
doesn't have to but for now we're going
to say we're just going to say the
default is false so if it doesn't have a
model it's going to create the model but
if you do have a model already ie you
saved a model and stuff like that you
you could just well basically if you
saved a model never mind let me just
write the function shut up so x equals
numpy array this is a little unfortunate
if someone has a cleaner way of writing
this do let me know anyway we're going
to do i 0 if for I in training data
don't forget train data contains
observations and then comma the actual
output the the action that you took 0 1
1 0 so the X the feature sets that's all
your observations so boom observations
but we have to unfortunately reshape
them
negative one Len and then we're just
going to actually do Len training data
0:04 whatever that shape this keep it
nice and dynamic oops move a little bit
bad and then one yep
that let ran off the spring a little bit
hopefully that's okay again sample code
source code all that posted so if for
whatever reason you didn't get that
lemma in fact I can I make this a little
smaller
I think that'll probably fit nope yeah
there we go beautiful that's probably
still acceptable so that's your ex now
your Y data will just be basically the
same thing it's just going to be let me
just do this whew we didn't do em pi ray
um we need to do this contry action here
fixing errors left or right now we're
going to do the same thing though so
actually copy/paste rather than the
zeroeth it's the vs if you need to learn
more about list comprehension there are
tutorials on Python programming dotnet
so now what we're gonna say is if not
model so if we don't already have a
model we're going to say the model
equals this neural network model copy a
and rather than input size we're going
to say input size equals Len of well
since before we got X define we'll just
say 2 0 is X so whatever the length is
of that that item ok so now that we've
got that cool now we're going to do
models dot fit and if it takes a whole
bunch of fun things so first of all it's
going to take your input and the input
will be our X's it's going to take
output which is called forgets that's
going to be your Y that's going to take
epochs we're going to say number of
epochs
we're going to say four now five we
probably actually could have gotten away
with three but five will be enough if
you do too many epics you're going to
overfit it's just what's going to happen
so if your network is actually training
pretty quickly those are few things you
could do to lower the learning rate or
something like that but if it's a simple
problem like this is if you train too
much and you get too accurate you've
over fit the problem so actually as
you'll see when we go to fitness we're
not actually hoping for like 95%
accuracy if we get a 95 percent accuracy
we're going to be in trouble it's
probably not going to work as well so or
95 or greater snapshot step we'll just
set this to 500 a show metric we will
set that to true and then we'll give it
a run ID just in case you are using
attention toward okay let me make sure
that actually closes that off yes it
does and then we'll return the model
when we train the model so now to run
the stuff let's go ahead and do an
initial population again because I feel
like that other one's a little unfair
initial population model will equal
train model because we don't actually
have a model yet training data okay
let's run that see how we do hopefully
no errors we'll see we'll see yeah okay
in it just trust what oh my goodness and
I'm gonna just okay can I do this please
thank you
up a little dirty we could have run this
in a command prompt them to be a little
prettier but that's okay so actually I
didn't even think our loss is improving
that much all right it just five epochs
probably too much for this problem
that's okay
why did we just print that again would
that happen for why did I get printed a
kid that was weird why did it print that
out again average accepted score we
didn't ask it to do that we asked that
initially of it all right
weird okay no ignore that where did it
happen
didn't it happen up here yet have enough
people Wow my tripping well you know
what I bet we tuck it twice the bud
Center yeah that's FS air isn't always
messing with future hairs and oh my
goodness we're controlled okay so the
models trained you might be a little
scared of that accuracy so as you can
see here you know we start off let's see
what some of our initial loss was six
plus eight six nine two seven seven or
so let's scroll we actually we improved
loss a little bit not a lot but we you
know we did okay quite a bit of our
accuracy within the 60s but yeah we
ended on this and pretty pretty
disgusting fifty six point nine seven
almost fifty seven no problem well we're
going to continue long because I still
have awesome things to show you
never fear so now that we've got a model
train the next thing that we're going to
do is actually use the model in play a
game so that's we're going to do in the
next tutorial if you have any questions
comments concerns did you get an air or
something like that and the model
running it or whatever let me know below
otherwise I'll see you in the next video
and we'll actually see how this model
does