Multi Layer Perceptron Part 3 (Java by example) Video Lecture - Machine

java by example and we are looking still
looking at the multi-layer perceptron
and today we're going to continue with
the translation from C code last time we
translated two functions and we
translated the constructor and the
populate input which wasn't that much code
and then we translated the calculate
network that will prepare each of the
nodes in the network and today we are
going to look at training the network
which will change the different weights
inside of the network and also translate
the recall network which is the function
that takes input data and tries to get
the actual answer from that input data
and that function is a bit easier to
grasp so I think we're going to start
with that one so let's look in here and
copy the code from recall network pretty
much straight forward here this is not
used anymore now we have an amount of
data instead so we changed this to some
int values of data we have a calculate
network we have a winner and index we
have some output neurons and then I
think most of these should just be
system out print line could be is there
you can use print line here as well and
here I believe that we can't do multi-line
this way so I believe we just have to do
the actual multi-line by adding
the phrases I think that could be a
solution for this not sure the correct
solution should be see here add plus over there instead so here do some more
yes I see you want yeah you want the dashes and
so you frame each of the different
values as well ahh see so let's continue with
addition here Thank You IntelliJ please
do not make so many intelligent assumptions
you are kinda hindering my progress here
get into the flow will ya it seems to, so there,
okay and then we have a lot of numbers a lot of
numbers so this file number should
actually be inside of here i guess there is
a way to do this and we actually don't
have any file number so let's just see
here that we have
string value that takes a bit file name
so we can use that in the output guess
this function will actually need to be
different when we actually use it this
could work here so this is pretty much a
repetition of the previous values please
do not do that can we please continue just
create the rows and then it should be 10 values
right 0 1 2 3 4 5 6 7 8 9 and I believe these should be added there as well so
that's pretty much it i guess the output
mmm output neurons yeah and the
probability is in percent so instead of
actually printing 0 dot something we do
at times hundreds so we get a percent
instead of spelling error there fix that
so this is pretty much that function so
that actually took care of calling each
of these functions
that's good so now we only need to
define the training network do a bit of
separation there's those easier to read
why yeah we don't have the return
statement yea of course so let's look
at the training function that's a large
one here we all have a lot of things to
do so let's start up here we need to
float value here so that's these are easy
those could be just defined as is these
Delta values are actually float arrays I
believe and the float array should be
float and the output neurons size then
we have the float for h delta which is
the size of all the layers inside oh
sorry we need to create new deltas we need
these temp weights and previous weights
and I think we actually define those
above and then we read some image goals
for the training files
so this should be the actual the goals
for each image from the file since its
malloced by the reader we have the rep
sponsibility so responsibility of
freeing its memory of the training so I
believe these goals we have to get back
to those later but I think that we can
remove this for now in here we need to
check the is less than the least mean
square error so i think we do these
definitions and say here that we see i
think this is math abs so it should just
be above we can remove this function even though it's
lower than we take the absolute values
will always be positive so here we need
to push all of these down a bit and
stack don't like when things aren't
aligned correctly so let's see here we
populate the input from the counter
through all the training phase so here
we need to actually get the training
files if populate input is false than we
will system out print line and I believe that this never got
any fault so it's a void function now
then we don't have that problem actually
so we can remove this you still need to
actually run it but I believe we can skip
this and let's look into this file would
take a file name and run it through and
get data hmm so let's just comment out
this code and put here read data from
image file and populate input so I think we will
actually wait with this implementation
until later and because I might want to
change how this is actually done I think
that instead of in training files I want
to add a list of files to read and
change that up
but I think we should go through this
function and translate that first so
here we get the particular goal for each
file and I think that this should also
be populated up here let's comment that
out as well now we calculate a network
for each iteration so here we have
for loop of all the output and we will check
the target let's say target is zero just to
get less of an error so we go through
all the output nodes and check if the
current value that we want to hit is not
the value then we will do this
calculation here to correct or create a
delta that we can use for correction
later on in our network so output delta
at this function we need to change a bit
so this is Delta at out so I think we
will translate each of these first delta
output at let's see here we have those
so in our case Delta output at should be
just this array right so we have it's
quite easy for us
just need to use this array for the
actual values that we want to now we have
doubles up here that won't go let's translate those to floats no I see these value should be floats as
well simplifies things there we go and
the error is a float as well so this
translates quite nicely so that one was
easy so let's continue and translate all
of the different values that use this
output Delta at so here we have a j value
instead of i value and see if we
have any more output Delta at we don't so
then we have put that value in
propagating wonder why there is so many
spelling errors here I'm not a native
English speaker but I found that yeah could
be good to actually run things through
through a spell checker to help you out
there so let's see here we have hidden
Delta at we translate that one earlier
this is the hidden delta at perhaps we
haven't now that's quite easy as well
because it's just this value in the
hidden Delta array
so let's go back up here so this is a
hidden Delta array so hidden Delta at
should be in the hidden data array so
to take the wrong value there so then we
add i and this should be hidden layer
and that should be 0 then so that's the
first translation then we just continue
this hidden delta at same here and the
same here so that we have translated now
I think this is pretty much the same
function that we had earlier we just
change the actual or the same kind of
translations that we had earlier the
only difference is about the different
numbers so here let's see that we didn't
do anything stupid over here we didn't
and here it's actually plus 1 minus 1
which is just 1 and K value then we get
that one lastly we have the same values
above and it takes the hidden value at
mkay and does dersigmoid on that and
times that with the hidden delta
yeah and then we get this
here we have also the hidden delta at but in
this case we have one which means zero
so we can remove these all together so
it's just a j value the hidden delta array and input to hidden see here
what's the problem here yeah input to
hidden its input hidden why have
you changed the order is just to change
things up to make it more interesting stupid so
input to hidden is i and j for some reason
why they want to change that up yeah inputs is
the outer layer okay and then we have to
do that as many times as the layers yet
so that makes sense still a bit strange
to type it that way and we have hidden
delta at down here so let's look at that
again this should just be this function
with the twist of hidden delta
and then we have a k value here and a y value over here
that makes us gives us the
correct value hidden to hidden we can't use
that either this one is a bit more a bit
more to change up so from hidden to hidden
okay this is just direct translation and
see here hidden to hidden is the first
input layer times the hidden neurons so
that's the first row of hidden and then
we, so that's the translation
between hidden and the nodes so
that's the weights between there so we
have taken that layer off and then we
take the i value which start us at 2-2
so that's from zero and so on and does
the hidden times hidden so that will
give us the weights
row or actually it will get give us the
right matrix and then we take j to get
the actual row of weights the k> for the
actual value in the weights hmmm so this
is to calculate all the weights in the
hidden hidden layers it's a quite
involved method so let's tidy these up a bit
I like to have less space between things
just to make get a better overview of
what's happening and so you can have
more lines in each segment because often
each line does not have that much
functionality in it and with as much
comments as we have in this code its
verbose to have one extra row for each
for each or for loop so now we all have
one extra value that we haven't
translated here I believe output delta at
did we remove that one hidden delta
at think we did so let's look at the
neural no it was in this function output
delta at so that's just the simple output
Delta array and mapping
into that so let's see here output delta and the i value that one
and here we have hidden to outputs
this is the function that so i guess it's
like the one above but simpler because
we just i value is the maximum value
here so we want the last matrix in the
in the function so here we use i no
j because that's inner loop and then i
because that's the output value so this
should be the function for this then we
do a counter more counter values we we
reset to counter with a specific print_f
for the windows environment because I
don't know and then we have some break
logic for the epochs
some freeing information that we are not required to do
in Java and we increase the epochs so let's just say
that we don't have any printing here we
just say that if we end up if epochs
larger than thousand then we break
easily like that so we don't care to
do more than a thousand epochs
perhaps we will change this valid we
even smaller so one epoch is when you go
from the input layer all the way to the
output calculate that then you will
calculate backwards to see how much of
the change you need to do to improve the
network and then apply that change over
the network again so that's one round of
calculation and that is required to do
many times I think that statistically
you should do it at least 500 times but
the more time times you do it the more
exact the match from the inputs to the
output you get so if you train the
network too much that is actually bad
because then you will get a to close
match between the input values and
output values so you want a close match
but not a perfect match because then it
will not be as good with new inputs that
I haven't seen before so now we have
translated all the
functions in this file we haven't
taken care of the reading of files and
we haven't tested our code and I think
we have least less errors in this class
not sure if we can compile it take a
look into that let's just try and make
not cannot start compilation output path
has not been specified hmm okay think
these are quite cannot start a
compilation yeah I will look into these
the different problems I have in the
build environment until the next time we
see each other and then we will continue
to look into reading image files and
actually testing our training and see if
we can train a network and get a viable
result on a specific image see if it can
recognize a number so this should be
really interesting to look at so I
hope that you found this episode
interesting and that you learned
something from it some of the
mathematics and so on have just been
copied and we haven't really looked into
exactly what each of these
and translations does and i think that
further on i need to do a lot of
refactoring to make the code more
readable and now we have just done a
one-to-one match from the c code and if
you like to see more of these videos
please subscribe to this channel if you
have any comments please leave them
below and share this episode with your
friends and colleagues and i really hope
to see you in the next video