Weka Tutorial 35: Creating Training; Validation and Test Sets (Data Preprocessing) Video Lecture | Weka Tutorial - Data & Analytics

hi in this tutorial I'm going to
demonstrate to you how you can split
your original data set into three
different sets the first one is training
set the second one is cross-validation
set and the third one is test set so
this has been a lot this has been a
question for a long time that how you
can do that in wakeup but I really
searched online and I did not find any
proper answer and then I figured out
that Wicca doesn't have a direct
functionality to split your original
data set into three different sets like
training cross-validation and test and
that's why people are getting baffled
and they're in no place to say that this
is the way to do that but you can always
play with some filtering options in Wake
and it's kind of tricky answered that
I'm going to demonstrate in this
tutorial and if you follow this tutorial
then you'll be able to split up your
original data set into three different
sets so before going into detail I'm
going to show you how I have organized
my original darf file that I'm going to
split into three different sets
so if I click open to see how the
instances there are you can see this is
a modified I restored our file just to
make this demonstration easier for you
because the organ I restart our file it
contains more than 100 instances but I
just kept the first ten instances of
this art file you can find it in the
data set folder of wake up so you can
see that these are the first ten
instances of of I restored our file and
I just chopped off the rest of the
instances from the original data set and
I'm going to split these ten instances
into training set cross-validation set
and test set without overlapping that
means the sets will not contain
duplicate items and what my design is
that the training set will contain 60%
of the data and the cross-validation and
test set will contain the rest of the
data that means 40% of these ten
instances and the cross valley
will contain the 50% of that 40%
instances that means our test set will
contain the rest of the 50% of the 40%
instances that are not included in my
test set training set sorry so in a
nutshell my training set will contain 1
2 3 4 5 6 data for example this is not
going to be the exact these these
instances but I'm just demonstrating
here that the training set will contain
60% of the data I have 10 instances so
the training set will contain 6
instances and the tests cross-validation
set will contain 50% of the rest of the
40% data that means that the 40 that
means that my cross validation set will
contain 50% of the data for example just
these two and the test instances will
contain the rest of the data that means
50% of the 40% instances that are not
included in the training set the and
tests that may contain these two
instances just work for example I'm
going to demonstrate that in wake up so
to do that in wake I I'm going to
Explorer and I'm going to open the file
called original dot our file that I've
made that I've created so this file
contains 10 instances so to split these
10 instances into training
cross-validation and test set we're
going to use a filter which is wakers
unsupervised filter and as we are
dealing with instances we are going to
choose an unsupervised instance filter
which is called resample so we see like
that then we need to edit the properties
of resample filter if you click that and
this window pops up and the options are
invert selection we're keeping it false
for the first time we have several sets
of operation coming up the no
replacement option should be true to all
time because we don't want our sets to
contain duplicate items and that's the
standard rule for machine learning that
it should not have duplicate items in
your training cross-validation and test
sets
now as I told you that I need
I'm planning to have 60% of the
instances 60% of these 1010 instances
here in the modified iris file will be
in my training set so I just keep the
sample size percentage option 260 we can
click OK and then I click apply so you
can see that the number of instances now
went down to 6 and this these 6
instances are my training instances so
I'm going to save that as train and I
click Save begin now now we have to deal
with the instances so that we have the
rest of the instances which are not in
the training set in that go into into
the cross-validation set and the test
set so to do that we have to revert
these operations we are clicking undo
and now we are we have our 10 instances
back here and again we have to
left-click on resample filter to edit
the properties so in this case the null
placement option will be true random
seed is 1 that's fine it up to it's up
to you to change this random seed so
that they did meet this means that how
you want to shuffle your instances in
your dataset so we're just keeping that
to 1 for this moment so you can see that
before we have selected sample size
percent 60 percent and we have got 6
instances that are included in my in our
training data but now we are trying to
get the other 40 percent of the
instances of these 10 instances so in
order to do that we have to change this
invert selection option and we need to
set it to true we click OK and we can
click apply this gives us 4 instances
and these 4 instances are not in our
training set to say up now - we have to
keep the 50% of these 4 instances in our
cross-validation set and the rest of the
50% of these 4 instances will be going
to our test set so to do that we again
click on this resample filter option and
we get inverse selection to true we are
just changing that to
because this is our first operation on
these four ins on these four instances
no replacement is still true random
Stage one is fine but we need to change
the sample size percent to fifty because
we're trying to get fifty percent of
these four instances we click OK and we
click apply this gives us two instances
so these two instant instances will be
going to our cross-validation folder and
we're going to save that and I'm just
renaming that to CV and click Save now
we need to deal with the test set
instances to do that we again click on
undo this gives us the four instances
that are not included in our training
set now
I'm again clicking on this resample
filter to get the options and this time
I need to change the inverse selection
option set to true because now we are we
have to find the 50% of these four
instances which are not included in the
cross-validation set so no replacement
is true random series 1 and sample size
% is 50 that's fine I'm just I just
changed the option inverse selection to
true this time click OK and I'm clicking
apply so this gives us two instances and
these are the two instances that are not
included even in training set or even in
cross validation set and we're saving
that as our test set so I'm just remain
renaming that to T for example and I
saved that so to demonstrate if this is
really working or not so I'm just
opening all the files that I just saved
with wake up the first one is original
darf and the second one is the train the
tarp one is the cross-validation and the
last one is the test set so if you
carefully look at the instances so this
is your original dataset you can see
these are the ten instances included in
that data set and if you take a look at
it carefully then you can
see that your training set contains 1
sorry 1 2 3 4 5 6 instances and your
cross-validation set will contain 2
instances and you can find that these
instances are not present in in your 2
Indians they're not duplicate that they
cannot be found in your train dot arf
and also your test set contains 2
instances that cannot be found in cross
variation set or in your training set so
I think that helps you to split up your
original data set into 3 different data
sets called train cross-validation and
test if you liked this tutorial then
don't forget to click on the like button
on youtube if you want to find out more
about the upcoming tutorials then please
subscribe to my channel and if you have
any questions regarding this tutorial or
the other tutorials that are posted so
far on youtube just just put some
comments on the comment area and I'll
try to get back to you as soon as
possible thank you very much