Weka Tutorial 10 Feature Selection with Filter (Data Dimensionality) Video

so welcome to the 10th way Commission
learning tutorial and this is a
continuation of machine learning
tutorial 9 so in this tutorial I'm going
to explain how you can use the filter
method in order to select the features
from your feature vector
so unlike repre method what filter
method does is it uses an attribute
evaluator that develops the attributes
or features and the rank R to rank all
the features in your dataset so you are
having a list of features that have some
ranks given by a particular algorithm or
anchor algorithm and in association with
an attribute evaluator so the number of
features you want to select from your
feature vector can always be defined
that's why I have bold and italicized
all in red because all means you can all
you can always have the options to
select all the features to rank them and
also you have the options to find out ok
you can say that I can I get the best
ten features according to their ranks
out of my 50 features so that's all as
possible for you if you're working with
wicker workbench and after having the
ranked features you can omit the
features one at a time that have the
lower ranks from the bottom of your
rankings and you can see the predictive
accuracy of your classification
algorithms with the ones that are not
omitted that are retained that have the
higher ranks well the plot one problem
with filter method is the weights put by
the ranker algorithms in order to rank
the features in your feature vector that
can be different very different than the
weights put by classification algorithm
so you always have the possibility of
being over fit your model can be really
over fit because the weights put by rank
your algorithms and the way to put the
classification algorithms they differ so
the rank or algorithms are used to rank
your features
and by omitting one feature at the time
from the rank list provided by this rank
or algorithms you are now using a
classification algorithm like naive
Bayes or begging to see how how the
classification algorithms are performing
on your data set with the features that
are not omitted to demonstrate we can go
to the Weka Explorer and I'm using the
iris of iris data set again so in select
attributes you can choose the attribute
evaluator I'm going to choose info gain
attribute evaluator this is my favorite
when I'm using the ranking so we can
choose that and you can see that base
first was selected as the default but
best first search search technique that
you don't have to use that you are not
allowed to use that for filter methods
because it because in filter you are not
actually searching for any subsets you
are already known that you are going to
evaluate every single feature in your
feature vector so it says you must use
the ranker search method in order to use
the info gain attribute eval and should
I set the rancor search method for you I
gladly accept that so you can see the
search method is now turned into rancor
from best first and you can click on the
search method you can see that we have
numbers of options now to select minus
one means give me all the rank list we
have in the P process tab we can see
that we have four features plus one is
that class feature so that minus one
means okay rank all the four features so
if you want to select the best two you
can always put two in num to select our
field here so if you want to know more
about rank or algorithm you can see that
where it ranks attributes by their
individual evaluations using conjunction
with attribute evaluation
evaluators like relief f gain ratio
entropy etc so it provides a good good
start good start with the ranking so the
starts Eddie I'm just I'm just ignoring
them because this is the default as I'm
using them in my research so I'm just
saying that okay now I'm to select a
minus one and I'm thinking okay if you
click on info gain evaluate your tab
that you can see that binarize numeric
attributes you're saying false okay I'm
not I'm not going to need that missing
March you're not going to need that as
well because this is a simple tutorial
to demonstrate how filter method works
so you can see that the info Gainey
attribute evaluator evaluates the worth
of an attribute by measuring the
information gained with respect to the
class and we have some informations here
as well how this info gain is calculated
so you click OK and we are using the
full training set and we are trying to
classify the instances on the basis of
the nominal class the number five
attributes of this data set that's
called class so we click start and you
can see that we have we have our
features now ranked so this is the score
the ranking score rancors algorithm some
inner mechanisms to give scores to the
features you don't have to be worried
about that we have some rags and we have
the attribute number and attribute names
here so the most prominent feature here
is number three
then comes four then comes one and and
at the last position we have several
weight so now what you can do in order
to in order to get the best set of
features you have the ranking here you
can see that two is at the bottom you
can go to the pre-process tab and you
can omit two and you can remove that you
go to the classify tab we're using an
eye base and this is the accuracy or
precision recall if score and the
results we
doubt the number two attributes you can
see that a number two has changed now we
have now three features and the class
attribute then comes number one so you
can go to the pre-processed tab and you
can omit number one from them and you go
again to classified tab and use the nine
basic we can now you can see that we are
having a different result so this is by
omitting one feature and now this is by
omitting the second feature so by this
way you can always take a look at the
incorrectly classified instances this is
four percent if I omit one feature
according to the ranking we limited the
feature that that was sitting at the
bottom of the ranking list and this one
in this case we have the incorrectly
classified instances also at four
percent so now what you can do is you
can plot the learning curves by
excluding one feature at a time from
this ranked list and by examining the
learning curves according to my research
I can say up to what point I can omit
other features from this ranking list
because we can have some ups and downs
in learning curves and you can always
reach to the global minimum in your
learning curve and then after reaching
that global minimum the error rate or
this information here incorrectly
classified instances that goes high and
high and it never comes back to the
global minimum so at that point you can
say after reaching to the global minimum
your data set is now getting overheated
your classifier is now getting
overheated if you omit a single feature
from your rank list so I stop there and
up to that point the numbers of features
who I omitted from the rank feature list
I omit them and the features that are
remaining after reaching the global
minimum that's my best possible feature
set so in this way I select my
that performs the best with a particular
algorithm now the problem I was talking
about is in select attributes you can
see that we use ranker algorithms to
rank and these are the rankings but for
naive Bayes this this is very different
the night based in naive Bayes we
evaluate the features differently than
the algorithm in rankers so this is very
possible from my experience I can say
that if you're using a classification
algorithm to get the predictive accuracy
of your classifier and you're using
rancors algorithm to rank the features
then this is always possible that you
can experience an overfit in in the
classifier you are designing or even
trying to develop so I always preferred
the first one wrapper method for this
particular test and I also can tell you
that wrapper method is useful for
machine learning tasks whereas the
filter method that's useful for data
mining tasks because in data mining you
can have thousands or millions of
features and at first glance you just
want to reduce that one hundred or
thousand or million features set to down
to fifty so in order to do that your you
always can run a filter method technique
the one I showed you here and you can
omit the features that are sitting at
the bottom of the rank list but in
machine learning test when you are when
you are aware of the features that are
really prominent in that case all you
can do is you can use the wrapper method
to find out the best subset of features
that performs the best with a particular
classification algorithm and then you
can evaluate a evaluative evaluate the
performance of a classifier that you
develop from that particular
classification algorithm with a subset
of features that you have produced so
that's it for today I think that helps
you in selecting attributes or selecting
features from a large number of feature
vector
thank you