Testing Network - Training a neural network to play a game with TensorFlow and Open AI p.4 Video Lecture | Machine Learning with Python - AI & ML

what's up everybody welcome to part four
of our playing games with open at I
Python tensorflow neural networks and
everything else tutorial in the last
tutorial we basically covered creating
our neural network model and training
that model and we saw the results they
weren't that great as like 60% ish
accuracy and actually ended on like a 57
we did see loss come down though that's
arguably more important but now is the
moment of truth let's see how it did so
what I'm going to do now is whistle
we're just going to play some games
we've got a model once we trained it so
we could save like so for example just
in case you wanted to you could say
model that says blah blah blah blah blah
model and then later if you want to load
it since we already have a model defined
you actually could just do model dot
load this model but if you started the
script new ie you didn't have all this
code before you would still need the
model code and you would just use a
model equals this and you would need to
know the input size and then you could
load the model anyway that's okay this
trans fast enough that we don't have to
worry about any of that so now we're
going to do is let's just run through
this thing so we're going to say scores
equals empty list choices equals an
empty list for each each game that we
want to play in range let's do ten we're
going to actually visualize these games
so yeah anyway so we're going to export
equals zero we're going to say game
memory empty lists we're going to say
pre-op and collisions I said pretty
simple are similar to the thing we wrote
before we could probably make one
function that does all this and either
use this random or uses the agent but
that's okay first they end up reset and
now we're actually an iterated game so
four underscore which is basically your
frame in range of
the goal steps however many steps we
want to make which is 500 say and render
so this will slow him down that's okay
if Len Cree Bob
pre bonds is greater than zero or
actually what we're going to do is where
is it that is equal to zero so if there
is nothing then actually we're just
going to say the action equals random
dot R and range zero to two again so in
that first frame you know we're not
going to know what move to make that's
fine and then what we're going to say
once we've seen a frame whereas a else
action is going to equal again this
needs to be a zero or one or network
outputs one hot which is 1 0 or 0 1 so
we're going to say NP dot org max to get
the Arg max of that one hot and then
it's the Arg max of model River our
model is predict and we're going to
predict the previous owner score
observation and what we have to do is
reshape that observation just like we
did Claude is going to be able to show
it to you really quickly where do we
reach it where did we reshape that here
yeah basically here it's going to take
that basically the same reshape so
pretty Bob's
dot reshape League shape negative 1 Len
free Bob so it'll be 4 but just in case
one ooh I'm getting lost here model dot
predict where is this one for so that's
where the reshape this is for the
predict a predictable output a list it
takes a list and outputs a list we want
this 0 ik so that'll be the first sets
because we're only predicting based on
one frame right now so we just take 0
now if that's confusing to you by the
way anytime you don't know what why are
we using this index here why are we
doing that print it out he whipped it
just like print out basically that's the
0th index for that officer bait earth
for the model to predict you can just
print that like just print it out see
what we're dealing with get rid of the
index see what it is
so anyway that's the action that we're
going to take now outside of that line
there that's kind of one zoom it in now
I like to make the text as big as
possible I made it small so we could fit
like this line now so hopefully go back
a second if you're still tugging on line
but anyway I'm gonna make this big again
okay so outside of that else what we're
going to say choices got a pen whatever
the action is we want to just know what
all of the choices are the reason we
want to do this is I mean since we're
not headless I mean like if we were
headless a lot a lot of times what can
happen is your neural network will just
converging and only predict one thing or
it will predict like ninety percent one
thing which can be really frustrating so
we kind of want to know how what what is
the ratio our network is predicting here
so we're just going to append the choice
so we can do we can look at that now
we're gonna say new observation is going
to equal or actually rather observation
row Ward's done info equals ten step
then we're going to take whatever action
one create off is equal to the new
underscore observation and then I say
game new memory depend new observation
in action and then we're going to stay
score plus equals reward and then if
done break so this line here we actually
plot like this line this line we need
this line we don't need unless you're
lying to retrain so what you could do is
turn this into a much more reinforcement
learning operation and keep saving the
game because hopefully as you'll see
unless I'm wrong but I never won our
neural networks only going to get better
and it's going to get better than the
data it trained on therefore if you just
kept cycling through here eventually you
get a neural network that
like crazy anyway well we're not going
to work likely train based on this we're
just going to run one time through
that's game dot memory oh thanks so game
underscore memory okay if done we'll
break out of this for loop and then
we're just going to come down here we're
going to say scores dot a pen whatever
that score is okay and then finally
let's do will print what was the average
score and that'll be the sum of the
scores divided by the lens scores we
could also print we want to know like
choice one and then choice two dot make
this the same as the other one dot
format and then again let's do choices
counts one so how many times do we do
one divided by the length of the choices
and then I'll just come back I think I
can get away with this
and then choices dot counts and this
shouldn't be to the sheet zero count 0
divided by Len choice so it's really
going to be like what percentage of each
of these choices
just close off of this format right
actually we want to do that okay I think
that's going to work and then we'll just
print well we don't yeah let's just do
this okay
safe run let's see how we do this is
going to retrain redo everything unless
I forget it
forgot anybody who there's 128 in there
this one's training better I saw it's
like some 62 s in the accuracy we
probably should have done like five that
are three epochs rather it is a strange
flow you probably just do three epochs
it doesn't look like we're getting too
much better than that I bet if we kept
training it though because loss is
coming down so touch still ended on 58
okay so now it's playing as you can see
it's doing a little better still
actually come it falls a lot this guy's
got it
this guy's like this ain't nothing it
falls a lot to the left but he's weak to
the left look at them they're all
failing to the left come on you got a
man they all fail to the left that's
really interesting so the average score
here is a 144 it shows one 50% of time
shows zero also about 50% of the time so
let me close this out that's actually
kind of a bad one I expected that we
were going to beat this thing I demand a
195 of greater again again maybe just
maybe I'll just do three epochs maybe I
won't do five epochs I almost wonder if
it's just getting over fit or something
let's let me make it this big
I'm pretty surprised that like that I
would expect kind of like at least above
200 either way though a 144 on the data
that we trained against which was like
you know the average was like a 60 but
are we missing the game
we're missing them all those guys are
doing even worse disgusting what do we
get average score was at 74 oh my
goodness y'all I'm angry we're still
that's still better but here's how you
think you four come on come on
I'm demanding solved there we go some
weaker weather few large good games
anyway I changed the ethics to three I
think what will not a second three
epochs fifty-eight accuracy maybe I need
a bigger network menu smaller Network
Cummins yeah you get it cart get it it
is hold it
nice these are doing better
maybe we over fit get it catch it yes I
thought it was pretty good and lose
those guys killing it haha nice I think
that one actually made it through the
all 500 frames I'm pretty sure didn't
lose this was good too I think we're
going to get this one let's let's hope
for above 200 average a demand that we
solve this catch it this is taking a
while to get through ten games so I'm
pretty sure we've got this thing got
this on lock we get boom average score
of 304 now we solved okay save that
model what's that
let's see this is actually we can save
this model actually so I think we did
this in line right yeah this is oh okay
it's a model got save will call this a
three Oh 4.1 model thank you sir thanks
sir okay
so what you could do is instead it's not
going to the same model but I'll go
ahead and we can close out of this and
we could take away that and we could run
how many games didn't say we had to do
to solve this I think it said a hundred
games it needed to average 195 I think
that's right let me run this while I
look and confirmed
okay because hopefully we do just as
good as we did before let's see yeah
over a hundred consecutive trials we
have to score above 195 we'll see if
this one goes let's let it go ahead list
nice and quick Oh taking a while to play
a hundred games it's cuz dudes oh no I
refuse again maybe we need to load up
our model our other models killing it
the other thing we can do too is we can
play let's say rather than ten thousand
games we could play five hundred games
to train it not that I think that's
going to improve accuracy probably make
actors to decline quite a bit but anyway
we could do that too wait for it
okay nice if this was in parallel I
guess this is taking forever I guess
it's because we're hopefully this one
will return something better than what
the other one was then we train ten
thousand examples headless in like five
seconds or maybe it's just stuck there
we go average score of 386 next I'll
save that one too actually great stuff
beautiful beautiful okay I think that's
enough for now
questions comments concerns if you
didn't get it or whatever feel free to
leave them below if you can apply this
to another game I'd love to see it there
again just tons of these games at least
the one in basic open AI like Mountain
car is not going to work very well in
mountain car need to be something that
you're actually going to control so like
a lot of the Atari ones for example
again if you're on Windows this is just
not going to work but the Atari ones
would probably work pretty well board
games would be really good we could
definitely make a GoBot maybe maybe I'll
try that one next a GoBot would probably
do really well actually in this one if
you get trained it really quick
especially in like a small nine by nine
like this or if it has I've actually
never played this one or if it has a a I
that you can play against or if you have
to play against yourself which would be
fine you guys actually compete against
yourself no problem Oh anyway so yeah
try to apply it to something else than
post below if you do questions comments
concerns whatever even below otherwise
I'll see you guys in another tutorial