Weka Tutorial 04 Systematic Oversampling (Class Imbalance Problem) Video

hello and welcome to the fourth Wicca
tutorial in this tutorial I'm going to
show you how you can manage imbalanced
data set what's the problem of
imbalanced data set well if you have a
training of test data set that consists
of positive and negative instances and
the classification levels before largely
that means for example in your training
set you can have 500 positive instances
but 50,000 negative instances so in that
case if you use some classification
algorithms then those algorithms can
underperform or they can experience some
overfitting problem because you have so
many imbalance in balanced data in your
data set so in order to remove that
there are several techniques and
personally I prefer one technique desk
that's called smart that's an
abbreviation for sampling a synthetic
minority over sampling technique SM OTE
and by using smart you can easily
increase the number of minority group or
minority instances in your data set that
means if you have 500 positive in 500
positive instances and 50,000 negative
instances then you can increase your
positive instances in every iteration if
you want to increase that 200% then in
the following action first iteration IH
would produce 1,000 positive instances
against your 50,000 negative instances
and you can experiment on and on until
you get good results with your
classifying algorithm or until the the
data set becomes balanced in terms of
positive and negative instances so Waker
has
a fantastic option to use smart so in
this tutorial I'm going to show you how
you can use smart option of wake up
smoke is actually filter in wakeup and
now your how you're able to balance your
data set so that you're classifying
algorithms can perform better they and
they do not experience overfitting
problem so let's get to wake up and I
have already loaded at an experiment
file of mine and this is a training file
that I'm currently using my research and
you can see that if you go to there are
several attributes the number of active
groups is 35 including the classic 5
classifying feature class and you can
see that the blue ones are es class and
the red ones are the no class and you
can see that we have only 372 positive
instances against almost 6,500 negative
instances so if I run this data if I run
a classifying algorithm on this data to
classify unseen documents with a 10-fold
cross-validation for example then I will
have some underperforming classifier
designed and that classifier also can
experience some overfitting problem
because of the imbalance dataset so in
order to balance it you can you can see
that you are in pre-processed tab you
can choose a filter from here in Waco
filters we have to choose supervised
filters and because we are trying to
increase the number of instances so we
have to choose the instance filter from
here and you can see that as mode is
present therefore a choice so we select
smart and if we want to make it default
then we can use the default settings but
if you want to change according to your
plans or according your task
then you can always have the option to
change the parameters of smoke so if you
click here and you can see the class
value is zero this means that wake up
automatically understands which one is
your minority class and in this case we
have our minority class at a zero plus
zero and that contains three hundred and
seventy two of course you can change it
to one but in that case the the
synthetic examples of synthetic
instances will increase for the no class
and your dataset becomes more imbalanced
so we want to choose zero here it's
default and wake automatically
calculates it nearest neighbors means
that the smoke algorithm works in this
way if you choose five nearest neighbors
then what is going to do is it
calculates or it synthetically produces
a synthetic training instances on test
engine instances between by calculating
five neighbors and put them within those
five neighbors and then it takes us
number six and number eleven and
calculates synthetically again to
produce the synthetic instance and put
them within the sixth and eleventh
instance so you can always have the
option to choose nearest neighbors this
is an open option force mode and for for
a a for instance if you know you can if
you choose ten this will obviously
change the classifying algorithms
performance if you chooses fifteen again
the performance may change so it depends
all on your tasks and all on your demand
so you have to be careful in choosing
nearest neighbors because you're
classifying algorithms performance
definitely is going to change just
because of selecting years neighbors
it's all up to you percentage means how
much of the data or how much of the
instances of the minority class
is going to increase in this iteration
so 100% means in the next run it will it
will be 372 multiplied by 2 because if
chosen 100 percent and those newly
created synthetic instances will be
inserted into your data set and randoms
it means it's just a case or random
value by which it increases it increases
the number of synthetic instances so we
are going to use the default one for
this case that because it's just a
tutorial we select ok and we then click
apply and this may take several times to
run because in every iteration in your
data instant in your intensity of the in
your data set are going to increase so
in that time it's it's increasing the
number of data every time 100% of the
minority class so sometimes it is OK in
waking Explorer to get slower and
sometimes it's a it gets it gets out of
memory as well so don't worry about that
you have to be careful in processing
large number of training features or
training attributes so you can see that
if you choose class after the first
iteration our es class this is the
minority class has increased from 362 or
something like that to 744 but the no
cast has not changed so in this way you
can increase the number of instances in
your minority class in your dataset and
now you can also save this data set
because a right after increasing the
number of instances in the minority
class and I'm going to save that and
test small file and if I go to a
directory to see what's written there
it can go at the bottom and these bottom
values are actually produced by smoke
algorithms and because you can see that
they're all yes only yes because the 362
or something something like that the
increased number of instances in this
training set they're all pushed at the
bottom of your training set you can see
that this is my original data set you
can find lots of knows and then some
yeses because we have few years and more
knows but at in the end after applying
the smoke algorithm you can see that
they're all yes now we have a problem
because in some cases if you have if you
if you want to apply 10-fold
cross-validation and you want to learn
you want to develop some learnt model
with some classifying algorithms at that
time if you have so many nodes at a time
or some in a fold and so many ESS in
another fold then we have a problem of
overfitting there so to remove that you
can choose another filter from here and
it's going to be unsupervised filter
we're going to deal with the attributes
I think and ok it's not going to be
actually be lighting it's going to be
instance and you can choose randomize so
after choosing randomize you can press
apply and you again save that file test
underscores mode we want to override
that and if you go to the North pad plus
plus you can see at the bottom now we
have some changes they're not all the SS
or all knows use result we have some ES
class here as well as we have some no
classes here now the problem has been
solved what was the problem I'm stating
that again if
you want to use some classification
algorithm to classify or to develop
model of your classifier with the data
set with if you and if you want to apply
10-fold cross-validation on that data
set in order to classify that but the
problem is that if every fold has a
large number of VSS and which if one
foal has large number of yeses and
another fold has large number of nose or
negative instances at that time you have
a possibility of overfitting your learn
learned model and so in order to remove
that when we use mode we have seen that
what small chance is small pushes the
synthetic minority instances at the end
of your file so if you want to apply
10-fold cross-validation at that stage
then this is very possible that your
learned model is going to be over fitted
so in order to remove that you have
chosen the randomized filter from wake
up your process tab and apply that so
after applying that you are going to
save that file and you can see that your
values are known randomized so that's it
for today I hope that helps and we have
to deal with lots of imbalanced data set
and you can use smoked in Waker explorer
to solve this imbalance imbalance
problem and it can make your dataset
balanced so that it's no overfitting in
your learning and learning algorithm and
there is no under performance as well
thank you very much