20 Auto Model - Clustering & Outliers Video Lecture | Rapid Miner: Learn and Master - Data & Analytics

[Music]
in the video on classification with the
auto model feature we provided an
introduction to the tool and its usage
in conjunction with classification in
this tutorial we want to introduce the
other two options in which to use it
clustering and outlier detection
together with classification there the
main tasks solved with machine learning
for this demo we will use again a
dataset which is shipped with rapid
miner studio of course you can also load
or import your own data sets the data
set is called sonar sonar stands for
sound navigation and ranging and is a
technique used for hunting and
orientation for example by bats and
dolphins humans are using the sonar
technology mostly underwater or for geo
exploration the technology is as simple
as emitting an acoustic signal and then
interpreting the echo which is returned
with respect to duration and
characteristics the data in the sonar
example set our sonar returns collected
from a metal cylinder and a cylindrical
shaped rock positioned on a sandy ocean
floor we have a total of 208 return
signals out of which each is described
with 60 different data points
representing the response amplitude of a
certain frequency band the original idea
of the experiment was to identify the
different objects via applying a neural
network model for classification the
classification was then given as rock
versus mine
originally called a metal cylinder
however classification is not our task
in this tutorial but we want to find
some clusters in our data all attributes
or columns are now checked if they're
acceptable for our usage and the result
is indicated with a traffic light color
to understand the colors and the quality
bars in more detail please refer to the
information panel and the auto model and
classification video as you can see here
from a pure data quality and
applicability perspective the data of
our label could also be used for
clustering but we will exclude it as it
is not relevant for our clustering
now we're presented with the option to
configure the general overview and with
the configurations for two different
clustering algorithms k-means clustering
always requires the input of how many
clusters are to be formed up front we'll
just check the results if we look for
example for three different clusters in
our data
check out five minutes within go in our
YouTube channel to
great explanation on how K means work of
course there's also some additional
details in the information panel X means
allows us to simply define an upper and
lower limit for the number of clusters
the default is set from 2 to 20 which
accommodates a large number of clusters
so we can keep that as well
so the first thing we can look at is the
correlations as you can see there are
close correlations between the
attributes for the obvious reason that
they're adjacent frequency bands and as
such not independent further you get a
nice overview for the results of each
clustering method for K means we have
clustered the data into three clusters
cluster 0 cluster 1 and cluster 2
cluster 0 contains 85 out of the 208
examples which is a bit more than 40% of
all examples with an average distance of
just a little under 30 cluster 0 is much
denser than the other two especially
cluster 1 is relatively loose the
characteristics of the clusters are
shown here for example cluster 0 is
prominently below average in these three
attributes the clusters 1 & 2 have both
values significantly above average in
three dominant attributes each the same
as a parent here again in the heat map
which shows all the nine attributes
mentioned before with red indicating
values above and green below average you
can also look at the clustering in the
form of a decision tree it is showing
that cluster 0 & 2 were formed around
these relatively clear paths
whereas cluster 1 basically consists of
the remainders for the rest let's jump
to the results of the X means algorithm
which has identified four clusters here
you can look at the centroid chart to
see the average characteristics across
all attributes and of course we also
provide the data for those centroids the
scatterplot tab shows a two-dimensional
representation using the two most
discriminatory attributes cluster three
for example has a strong characteristic
in attributes 16 and 17 and you can see
the points are nicely congregating
around here last but not least we show
down here the main result of the
clustering which is the clustered data
now we use the same data to identify if
we have any outliers in our case this
might be for example cases with bad
signal recording we could then remove
those upon identification and redo the
clustering to get a potentially better
result just for consistency all
again the class we've discussed it
before so I'll turn off the correlations
this time with our small examples that
we can increase the percentage here a
bit too too
and here you see the output for each of
our two outlier detection methods as
Flags and here even with the absolute
values you can now investigate the ones
which are indicated as mentioned also in
the other auto model video it is
important to note that nothing here is
done in a black box but rather the
opposite so in case you want to edit the
underlying process you can do that just
by opening it here you have now all the
different steps displayed and annotated
nicely so you can swiftly find your way
around to check and change as you please
with this we end this tutorial on
clustering and outlier detection with
the auto model feature thank you very
much for watching
[Music]