Weka Tutorial 39: Cost Sensitive Learning (Classification) Video Lecture | Weka Tutorial - Data & Analytics

all right welcome to another worker
tutorial in this tutorial I'm going to
demonstrate how you can use
cost-sensitive learning using wake-up
call sensitive learning is a very useful
technique whenever your data set is
imbalanced let me give you an example of
imbalance data set say you have 1,000
instances and out of this 1,000
instances only 10 belong to positive
class and 990 belong to negative class
so say you have just two classes
positive and negative so in this case
you are having negative skewness in your
data set your data set is said to be
highly negatively skewed so in scenarios
like these evolution measures like
accuracy or precision recall and score
they sometimes don't reflect the
accurate results of classification by
your classifier so machine learning
people have invented a technique called
cost-sensitive learning that whenever a
classifier is learning you're going to
say that okay I'm going to punish you
for making false positives or false
negatives by default this punishment is
set to one which is saying that okay I'm
not going to punish you for making false
positive and false negative just work
what with whatever you have learned from
the data so cost-sensitive learning is a
very very well defined procedure in
machine learning and you can apply that
for class imbalance problems or for
datasets that are having class imbalance
problems rather so this is how you do it
in wake up to do that we go to Explorer
and in the Explorer we are going to load
a data set for this particular
experiment I am choosing the day's data
set called diabetes dot R and you can
find it in the wake installation
directory under data directory so we
select diabetes store are so when it is
selected we go to classify tab and for
simplicity sake we're just selecting
naive Bayes classifier and we are just
starting the classification so now you
can see on the right hand side in the
confusion matrix that we
have two classes a is tested negative
and B is tested positive and this
confusion in this confusion matrix you
can see that instances that are
belonging to a class and classified as a
are 422 instances belong to a class but
classified as B are 78 instances belong
to a sorry instances belong to B Class
and identified as B 164 and instances
that are dead belong to B class but
identified as a are 104 so in a nutshell
we can say from the perspective of a
class or tested negative class of
diabetes or our file the true positive
is 422 to negative is 164 and false
negative is 78 and false positive is 104
we can also see it as a cost sensitive
learning with weight set to 1.0 to false
positive and false negatives so this is
your default classifier with weight set
to 1.0 to false positive and false
negative
you're not going to punish your
classifier for generating false
positives and false negatives but in
cost-sensitive learning you're just you
can just tell the classifier that I'm
going to punish you if you make false
positive and false negative and how much
are you going to punish then comes the
notion of weight and we are just going
to see how you can put weight in a few
seconds so this time we are interested
to choose a classifier which is called
constitutive classifier you can find it
in the middle classifier options in the
classifier options of Waker we go to the
properties of the cost sensitive
classifier you can see that we have a
option of an option for classifier we're
going to choose naive Bayes as our
classifier because we are interested to
experiment with diabase now we can also
see that we have an option called cosmic
Trix and by default the matrix dimension
is set to one by one
so we click on that and you can see that
on the 0.0 is set to their set there
because we our dimension of the matrix
is set to one by one but here you can
also see that the default number of
classes is set to one but in our data
set we have seen that we have tested
negative and tested positive two classes
so we are going to change and we are
said that we are going to set that to
two so as we set that and click resize
then you can see that the matrix size
has changed the dimensions have changed
so the top left corner is the true
positive top right corner is the false
negative lower left is false positive
and lower right is true negative you can
see that the false positive and false
negative cells of these matrix they are
set to one so that's the default I as I
said that the weight is set to default
one point zero for false positive and
false negative for any kind of
classifier so that's kind of constants
ative classification with weight equal
to 1.0 so this time we are going to be
happy with this setup and we are going
to cross this window and click OK to the
next window and we start classification
now from these confusion matrix you can
see that the number of false positives
and false negatives they have changed
they have not changed sorry they have
not changed so this is what we have got
in our confusion matrix for by using
consensus made a classifier using naive
Bayes as the best classifier and this is
the ninth base itself so the results are
the same so now we go back to our cost
sensitive classifier options and in the
cost matrix now we are going to change
the weights now we are going to click on
double click on the upper right corner
and we are setting that to two that in
this way we are saying that okay I'm
going to punish you double if you are
making false negatives
and then we press tab it's set to 2.0 we
cross it we click ok we come back and
start classification but this time you
can see strangely we are having a less
number of false positive it's it's now
down to 46 but our false positives
having increased and that's the catch of
cost-sensitive learning you have to
decide by yourself that if you are
saying that I'm going to punish for
making false negatives then false
negatives are going to decrease by your
classifier in some sampling some method
but also the catch is that your false
positives may increase so again let's
make an another make another experiment
we go to cost matrix this time we are
going to readjust the false negative
weight to one point zero the default but
we're changing the false positive weight
to two in the similar fashion we have
done it before we cross it we click OK
we come back and press start but this
time again you can see that strangely we
are having less number of false
positives but our false negatives have
increased because we have put weight
equal we have set weight equal to two
point zero for making false positives by
our classifier so our classifier is
really afraid to make false positives
that's why it was learning very well how
to deal with false positives so it
decreased the number of false positives
now it is down to 82 but also the number
of false negatives have increased and it
is not 115 now if you can remember from
precision recolor leaf score formulas
false positives they're related to
precision and false negatives they're
related to recall so if you put a higher
weight for false negatives then you are
going to have better recall your F n or
false negatives are going to be
decreased and also similarly if you put
higher weight for false positives then
your precision R is going to be improved
because you are now having less number
of false positives but your F score
I cannot I cannot remark on that you
have to do it experimentally but this is
how con sensitive learning is done in
Wicca it's a very simple step to follow
and it is very well known method to
fight or deal with class imbalance
problem in your data set if you are
benefited from these tutorials in my
channel please subscribe to get more
updates on the videos upcoming and if
you have liked this tutorial press like
and if you if you understand every
concept of it then please share with
your friends so that they can understand
the concepts of cost-sensitive learning
as well and this is how this channel is
going through all the odds thank you
very much for watching and listening I'm
hoping to come back with more tutorials
on Waker and other technologies stay
tuned thank you very much