Weka Tutorial 30 Multiple ROC Curves (Model Evaluation) Video Lecture -

oh and welcome to another youtube waker
tutorial in this tutorial i'm going to
show you how you can use multiple
numbers of ROC curves or multiple
numbers of classifiers so in previous
tutorial I demonstrated how you can
generate a single ROC curve for a single
classifier but in practical situations
most of the cases you'll need this
tutorial to learn how you can generate
multiple numbers of classic multiple
numbers of ROC curves for multiple
numbers of classifiers this is really
important to evaluate the performance of
your classifiers so to start with we are
going to the knowledge flow of wake up
because Explorer will not allow you to
produce multiple ROC curves for multiple
classifiers in order to produce these
ROC curves we have to go to the
knowledge flow application of Vega so I
click on that the knowledge flow
environment is not loading so when it's
loaded the first thing is to get the our
floater this will load the art file the
second thing you need to do is you're
trying to trying to point out which one
is your class attribute so in order to
do that you have to go to the class
assignor in the evaluation tab you
select that and you put that in
knowledge flow layout the next thing to
do is to choose a class value picker
here from the evaluation tab you click
on that and you put that in your
knowledge flow layout so I'm going to
connect these things later so at this
point we have the AR floater we have the
class assign it to see to say waker
which one is your class attribute the
as value picker I'll talk about this
more shortly the next thing to do is
you're going to set up a cross cross
validation fold maker because we're
doing a 10-fold cross-validation so we
put that in the knowledge flow layout
the next thing to do is to classify the
items using classification algorithms in
this example I'm just choosing
classifiers produced by ninety base and
also I'm going for a three type
classifier which is random forests
there you go
I choose it and I put it here okay so as
as these two algorithms will produce
classifiers I need to know their
performances so I go to the evaluation
tab and you can see we have an option
their classifier performance evaluator
so I click on that I put it there I
click on that I put there so I need to
performance evaluator for two algorithms
present here and finally I want to
visualize the ROC curves from the
performance evaluator so I need to pick
up the model performance Sharpe from the
visualization tool I click on that and
place that on the layout so I have
pretty much everything except the
connections to connect the our floater
with the closest einer we have to right
click on that and we click on data set
we have an age and we connect these two
next thing is from the closest einer we
right click on that again we choose the
data set and we connect it with class
value picker again we click on we right
click on the class value picker we find
the data set and we put that to
Crossville cross-validation fold maker
so after getting our data set up to this
point to the cross-validation fold maker
we have to provide the training and test
set to the two algorithms we are using
naive Bayes and random forests so we
right click on the cross-validation fold
maker we choose training set and put it
to night base and again right click on
the CV fall maker we choose the test set
and put it back to night base again in a
similar way we are right-clicking on CV
fall maker choose choosing the training
set and connect it with random forest
and again right-click on the CV fall
maker choose the test set
put it back to random forest in this way
we are saying okay we are having a cross
fold a validation here we are making ten
fold or for example in general we are
making careful's and these cables are
served to these two algorithms as
training and test set next we are going
to give the data from naive Bayes as a
batch classifier to the classifier
performance evaluator we did for naive
Bayes and again on random voice we right
click and we find the best classifier
there to connect it to the classifier
performance evaluator the next thing to
do is right click on the performance
evaluator and we're getting the
threshold data we are pointing that to
model performance sharp this is flowing
from 90 days part and this one will be
flowing from the random forest part that
we arrived right clicking on the
classifier performance evaluator we're
choosing the threshold threshold data
and put it back to model performance
chart okay now to configure we can say
that which one is our class designer so
we right-click on that and we click on
configure and you can see that it's not
still there because we didn't load any
kind of data here so put on a cancel and
we are now trying to load the art file
right click on the our floater we go to
the configure and we are choosing the
diabetes data set from the way calc
positive then we are right clicking on
the class designer and we are saying
configure and you see that you can put
any class there so these are the
attributes in our our file diabetes dot
our file and we are saying that ok my
plus file is the last one so select that
one and click OK the next thing to do is
the class value picker so if you don't
know what what what is the purpose of
class fellow picker you can go back to
my tutorial
and there you'll be able to see that
whenever you're producing a rock carve
that rock curve can be produced for each
items each class labels okay so that's
what you are going to tell here in the
class value picker so we are going to
right-click on the class value picker we
click on configure and you can see that
we are having two types of class values
there tested negative and tested
positive
so the ROC curves can be produced either
for tested negative class or tested
positive class so just for the sake of
the tutorial we're saying okay we are
interested to produce the ROC curves for
tested positive class value and we're
taking ok and last thing to configure
here is the cross fault that the cross
validation pole maker we right-click on
that we click on configure you can see a
false or 10 so that's that's okay
because we are expecting to make this
experiment add in a 10-fold
cross-validation environment so that's
okay across that so in order to put this
experiment into practice what we are
going to do here is we are going to
click on the AR floater right click on
the arp loader and we click on start
loading so you can see that something is
going on here and everything is finished
the result is now on model performance
chart so we right click on model
performance chart and select show chart
so now you can see that we are having
two - ROC curves there one is for 90
days the other one is for random forests
and the random forest ad naivebayes both
are present there and from these ROC
curves you can see which one is better
and which one which one is not to
interpret the ROC curve as I told you in
my previous tutorial that the carp
should be the perfect curve is just like
this so it starts from 0 it stays with
the y axis and then expands
to the x-axis the more the curve closer
to one the better the performance of
your classifier is I hope that helps in
this way you'll be able to plus you'll
be able to understand or report the
classifier performances when you are
using multiple numbers of classifiers on
a single data set thank you very much
and I'll be back with more tutorials
shortly thank you don't forget to like
my tutorial if if this was really
helpful for your experiment and don't
forget to subscribe my channel to get
more updates and tutorials why