Weka Tutorial 16 Detail Cross Validation Results using API (Model Evaluation)

welcome to wake a tutorial in this
tutorial I'm going to show you how you
can achieve the results of every fold in
a careful cross-validation set up you
can go back to my tutorial 12 where I
used the experimental tool of Weka
the quick a GUI instead of wake up I to
get the results especially precision
recall f-score and error rates of every
fault in a careful cross-validation
environment so for this tutorial I have
written a class called
Weka test and this waked test has one
single method called main now I will go
through every line of this main method
and illustrate what that line actually
is doing the very first line creates an
object of instances class the name of
the object is also instances what this
means is I'm using the bufferedreader to
read the file called to dot ARF and the
structure of the to dot our file is
explicitly mentioned in my previous
tutorial so this line here is actually
taking every instance of to dot our file
in the instances object now we will move
to the second line in the second line
what I'm doing is exactly what I did in
my previous tutorial in this line I am
actually setting that class attribute
that denotes the positive and negative
levels of your dataset so I am calling
the method called
zip plus index of object instances this
method actually sets the class attribute
of your dataset and if you can recall
the class attribute of to dot our file
that I am currently using is situated at
the very end of the attribute section
so if my data set has ten attributes
numbered from zero to nine
then the class attribute is going to be
nine that's why I have deduced one from
the number of attributes method returned
there after I have declared four double
type of variables precision recall F
measure and error also I have assigned
them zero then I have number of
variables declared again the first one
is called size the size is defined as
follows I have called the method num
instances of instances object and divide
that by ten so the size integer variable
is actually the size of every fault for
example if I have 40 instances in my
data set then the size of size is going
to be 40 divided by 10 equals to 4 this
means we have four instances powerful
variables begin and end are declared to
control the test instances so the begin
is set to zero and end is set to size
minus one so if you can recall from your
machine learning or school book in a
10-fold cross-validation set up if we
have 40 instances in our data set then
we have fold one containing 0 1 2 3 fold
2 containing 4 5 6 & 7 instance and
false
10 will contain 36 37 38 and 39 instance
so as you said begin to 0 that means
fold one and in a sorry an end set to
size minus 1 that's 3 so that means fold
1 is going to be our test set for the
first iteration then we have a for loop
this for loop is actually taking control
of all the faults we have 10 folds
that's why the loop starts from 1 and
ends at 10
I am also printing the iteration number
at the very first line of the loop then
I am creating instances object the first
one is called training instances and to
create that instances object I am giving
all the instances in my dataset in the
third line of this loop I am creating a
testing instances object so in order to
do that we have simply modified the
argument list than that of training
instances in this time we are sending
the instances all the instances in our
dataset we are also sending the begin
and end - begin as argument this means
that this line says states that create a
training instance with all the instances
of our toy dataset but the test instance
should contain only those instances from
beginning to end
- beginning and also from a 10-fold
cross-validation set up you should
remember that the test set should be
excluded from the training set that's
why we are deleting those entries from
the training instances that are inside
the test set then I'm creating an eye
base object that's my classifier called
tree and I'm sending all my training
instances
to develop or induce an eye base
classifier on them
I am also again creating evaluation
object of devaluation class this time
I'm sending the testing instances
because I am going to apply the night
based classifier on the test set
that's why to evaluate model I am just
sending the classifier we just developed
here and the testing instances and
afterwards with this evaluation object
are printing PRF and e of this
particular iteration next I am just
adding up the PRF and E because I'm just
adding up for iteration 1 iteration 2 up
to iteration 3 all the PRF and E and
then I will divide each of them by 10 to
get the average PRF and E the PRF and II
after summing up dividing by 10 if we
get them then that's my average PRF and
E for 10-fold cross-validation set up so
in the end we have numbers of operation
the first one is begin equal to n plus 1
again if we consider the example we just
we have just seen
then our full one contains 0 1 2 3 fold
2 contains 4 5 6 & 7 instance and our
fault n is going to contain 35 36 37 38
and 39 instance so for the next
iteration for the first iteration for
example fold one was our test set so the
begin was set to 0 and the end was set
to 3
the next iteration for two is going to
be our test set and the rests are going
to be our training set so we are setting
begin to n plus 1 that means we will
have a begin equal to 3 plus 1 that
equals to 4 and we are also saying that
n is going to be nth plus size so our
end card end is going to be sorry
our current end is 3 and we are adding
the size with it so the size is going to
be 4 and now next end is 7 also for the
last iteration we have to be careful as
we approach to the last iteration if I
equals to 9 at that time we don't have
to add end with the size instead we can
easily called the NAM instances method
and set that to end that means in our
very last iteration our end is going to
be set at 39 if we consider our data set
to be composed of 40 instances so in the
very end we are approached to the very
end of the program we have accumulated
PRF and E for every fault and we just
divide them by 10 each of them by 10 to
get the average precision average recall
average F measure and average error so
if we run the program and take a look at
the output then we can see that we have
the results for every iteration so you
can see that this is the result for
iteration 1 this is for titration 2 for
3 and this goes up to iteration 10 so
now you're having PRF and E for every
iteration and in the end you can always
see that we are having the average
precision recall net score the
accumulated precision recall EF score
and error rate and we divide that by 10
so I hope that helps see you in a week