Clustering Introduction - Practical Machine Learning Tutorial with Python

what is going on everybody and welcome
to the 34th part of our machine learning
tutorial series and the start of a new
section which is clustering and
unsupervised machine learning so up into
this point everything that we've covered
has been what's known as supervised
machine learning which means we the
scientists have told the Machine what
the classes of the feature sets were
with unsupervised machine learning the
role the scientist begins to be further
removed so to begin we're going to be
covering clustering which comes in two
major forms you have flat clustering in
hierarchical clustering with both forms
of clustering the machine is simply
given a data set which is basically the
feature sets and just the feature sets
and then the machine itself is going to
actually search for groups or clusters
and that's what it actually does and
then each cluster in theory is given
like an ID or something which usually is
going to be like cluster 0 cluster 1 and
so on so what's the difference between
flat and hierarchical clustering well
with flat clustering you're going to
tell the machine as the scientists
you're going to say hey I would like you
to find two clusters or three clusters
with hierarchical clustering the Machine
figures out the groups and how many
groups there ought to be so the first
one that we're actually going to be
covering is going to be using the
k-means algorithm so with clustering
we're going to be first talking about
k-means so this one's actually a really
simple algorithm it's going to be a nice
a nice Segway coming from the support
vector machine at least and it's quite
simple and the only parameter that
you're going as a scientist going to be
imposing is just simply you choose what
K is and K is the number of groups that
you want or the number of clusters that
you want so here I have a nice beautiful
graph and with some data points
and basically the difference between say
flat clustering and hierarchical
clustering would be the following so
maybe maybe you say to the Machine okay
I want to do k-means and we're going to
say K is equal to 2 so this will be flat
clustering so when K is equal to 2 the
cluster the algorithms going to go
through and try to find 2 clusters is
probably I don't know maybe going to
find this and then it's going to say
okay this is a cluster something along
those lines probably not actually it
won't actually this is just a simple toy
example but actually we'll talk about
why this probably actually will not
occur especially with with k-means but
anyways for now for the sake of concept
k equals 2 it finds those two groups
instead what if we ran a hierarchical
clustering algorithm may be something
like mean shift which is the other one
we're going to talk about it might say
hmm I found this group this group and
this group right it's going to say ok I
found 3 clusters actually so something
like that but for this tutorial we're
going to be talking about k-means and
let's say we've got k equals 3 the way
k-means works is the following
basically you'll take your entire data
set and you're going to set randomly
basically you're going to choose some
centroids and those are just going to be
randomly chosen to start so centroids
are just the Centers of your clusters so
to start I just typically take the first
K values in the feature set whatever
those happen to be but you can also use
random or something to actually randomly
select or you could shuffle your data
then take the first 3 and all that in
the end it really shouldn't matter but
if you're you know if you're not
optimizing for whatever reason it might
make sense to like try to shuffle the
data and then and then try again so what
you'll do is like let's say you took
your first two feature sets and let's
just for example say it was this this
one and this one now what you're going
to do is you're going to calculate the
distance of each feature set to the
centroids
and then you're going to classify each
feature set as let's say sense K
actually K is equal three so let's say
we chose these three okay so so what
you're going to do is you're going to
take each feature set from here and
calculate the distance we already know
how to do that we're just going to use
norm or Euclidean distance and you're
going to classify each of these new
feature sets as whichever centroid
they're closest to so let's just say
this is centroid 0 this will be centroid
1 and this will be centroid - ok and I'm
just going to eyeball this just to get
an idea but we're going to say we're
going to say we'll start with centroid 0
clearly let's do this one this one this
one obviously centroid that is 0 this
one in this one are all closest to 0 for
1 we're going to take this one this one
and that's probably it and then for 2
you would say ok all of these little
feature sets here belong to centroid 2
and again these centroids could be
totally randomly chosen you just needed
to just pick 3 and it could go you could
have you could have picked a totally
different 3 it would have been fine the
end result will still be the same so
we'll have classified all of these and
then what you're going to do is you're
going to take all of the feature sets or
data points or whatever you want to call
them you're going to take all of those
and then take the mean of those so find
the center of all of those so I'll put
the center in orange and let's say the
center is going to be I don't know
probably there hopefully you can see
that there here and then probably here
ok so these are our new centroids I'll
circle them just to make them even
easier to see in the video so those are
your new centroids and then you repeat
the process you just keep repeating that
process until the centroids no longer
are moving and once the centroids are
completely done moving
you've got your clusters and you can
already see obviously I eyeballed this
you're so
hopefully we would only need to take one
step but you should see that we've
pretty much already found the clusters
right we've already clustered on with K
equals three that would be this as a
closer this is a closer and this is
closer so we're actually already done in
one step so the final step is just
simply to repeat step two and three
until you're optimized the way you know
you're optimized is by checking how much
that centroid moves if at all and again
just like we did with the SVM you'll
probably have some sort of tolerance and
then you'll also probably have some sort
of Max Max itter for max iterations just
to say okay if we haven't found our
answer in 500 iterations or I think
scikit-learn uses 300 as the default if
we haven't found that the optimization
by then maybe we're not finding it or
we're not going to or for whatever
reason okay so this is actually super
simple it's easy enough to implement the
one downside to k-means is when you have
k-means basically wants to cluster data
in relatively equal sized groups and so
that's kind of like one of the major
downsides is just that it's it's going
to have a hard time picking like
differently sized groups because of this
this adherence to distance rather than
some other sort of adherence so a
popular data set is something like like
this it's it's referred to as the the
mouse data set and looking at it
visually you would probably agree that
you know you've got a cluster here here
and then here right and that it makes a
you know like a Mickey Mouse looking
thing okay but because of the way
k-means works and even some of the other
clustering algorithms work it's going to
you know it'll find the cluster center
is here here and here but when working
out the distances this cluster might
actually be this part this cluster might
actually get you know this whole area
let's say and then this cluster gets
this area but as you can see what what
should have been basically this for this
bigger cluster
is not because of the kind of adherence
to Euclidean distance so that can be
kind of hard and there are some things
and some clustering algorithms use
different what are called kernels which
is what we've already covered but as
you'll see even if you changed the
kernel to maybe a Gaussian kernel or
something like that
you are still going to have this problem
so this is probably the main issue
actually with k-means other than as is
common with most of the algorithms that
we're going to cover scaling so scaling
again you have to compare each point to
all other points luckily though like the
support vector machine once trained once
you have those centroids you can
classify new points based on those
centroids and you actually don't need to
train anymore and that's kind of the
neat thing about clustering is that you
have what's called semi supervised
machine learning where you start off
with a clustering algorithm you say okay
I want to find two groups and once it
finds two groups you can then translate
translate that to supervised machine
learning and say okay these are the two
groups that we have this is group zero
this is group one then you can actually
use like a support vector machine or
something like that on the data anyway
what we're going to do now is we're
going to pop on over to scikit-learn and
actually apply this to a really quick
example we're familiar already with
scikit-learn and and graphing and all
that so what I'm going to just stuff it
into the end of this video since there's
really nothing new but we want to at
least kind of cover the cover the exact
cover an actual example and show it at
work alright so before we actually get
to applying to a real-world example I
want to go ahead and show a very simple
example that can actually be visualized
so we can see that the k-means algorithm
actually works so with both of the
clustering algorithms that we're going
to be covering here it's not so much of
a accuracy or visualization capability
that either these algorithms have
they're more soda like for research and
finding structure so for the most part
when you're actually realistically using
them you're not going to actually see
that or have any sort of degree of
confidence as far as whether or not
these algorithms even work so we're just
going to see it visually real quickly so
we can have some degree of confidence
before we actually use it on an actual
so anyways let's go ahead and import
matplotlib pie plot as guilty from
matplotlib we're going to import style
we're going to style got use GG plot
important um pie as NP and then from SK
learn cluster we're going to import
k-means now we're going to define some
starting X values that'll be n PE array
and then we'll have one two three four
five and six elements in here you can do
something you want but you probably
should make it too obvious groups just a
suggestion but you don't have to you can
see what happens if you don't so I did
one to one point five one point eight
five eight eight eight one zero point
six and finally we'll do a 911 now PLT
dot scatter and then we're going to
scatter x colon comma zero x colon comma
one this is all the zeroeth elements in
the X ray these are all the first if
elements in the X ray will say the size
is 150 two's make a really big inline
whit's five so that's also use it and
then we'll show so this is just gonna
show us the data we're working with SK
lene lene version of SK to learn oh my
god wow that's a lot of typos we're not
making it very far okay let's see what
happens here alright cool so we made it
this far good stuff let's see size 150
line widths probably cuz I'm not using a
marker so it's just like this fat yeah
so maybe if I maybe I did this we don't
have that weird halo around there we go
nice okay so that's our beginning data
set and obviously if we say K equals two
we're expecting that this would be a
cluster down here and this will be a
cluster up here okay I think everybody
agrees so now I'm just going to comment
these two out try as hard as I can
anyway to do that and then we're just
going to define our classifier CLF
equals capital K and means we're going
to say n clusters that's one of the
parameters let's probably the only main
parameter you're ever going to need to
use besides maybe max iteration and
clusters equals to I believe the default
is eight why eight you ask I really
don't know I do not know why eight is
the default anyway number of clusters
too and I I don't even know what would
happen if we used a tuning of eight data
points but if you use like six it would
find all those closures I really do
wonder what happened maybe it would like
have two centroids on the same I don't
know if I if I think about it maybe
we'll do that at the very end she left
up it it would be for fun you'll see how
that fit X and then and then basically
you fit and then we're going to access
some of these attributes so we're gonna
say the centroids equals C left cluster
underscore centers and then we're going
to say the labels equals CL f dot labels
so so labels will just be an array
purely of the labels of our features and
they will have the same index as our
features so you can think of X as X
labels is your lowercase Y okay to
compare it to two previous tutorials now
we're going to have a colors list you
can make this a pretty long list if you
wanted I'm just going to go ahead and
put two things in here because I know
we're using K actually shoot meaning
probably more next we're gonna hmm well
what we'll do this for now so so green
period red period and let's go ahead and
add a few here red will do cyan period
be period K period so at least gives us
let's try to have let's see if I can
think of one more I think
orange oh so old period so these are
just like the color you can kind of get
away from get away with this when you do
like a color and then a period is like
the marker so it'll be like a dot
hopefully okay so now we're going to do
is we're gonna say for I in range of the
lens
of X capital X though before you get
ourselves in trouble there for I so this
will just the I will be our index value
right so we're going to PLT dot plot
we're going to plot X capital I 0 and
then we're going to pop x i1 okay so
that's the data and then we want to have
the color we're just going to say color
and we can actually get away with this
we can say colors so we can just have it
past so we can say colors and that's
going to be labeled I so colors labels I
and then so this will be literally like
the index so the label is going to be a
0 or a 1 since we have two clusters so
we're gonna we're going to use that
value this will be a 0 or 1 which
references the index of color so it'll
be a green or a red ok all that's going
on there and then we're gonna say marker
size equals 10 10 and then that's all we
need to do there then finally we're
going to do plot scatter and we're going
to do centroids colon comma 0 centroids
colon comma 1 not 12 just 1 the more say
the marker here yeah we're gonna say
marker is an X lower case I'm not sure
capital case would work their size will
be 150 line widths will be 5 and that's
all we need to do there and then finally
we'll do a plush up ok la much ado about
nothing hopefully no type is sure enough
okay so uh did we see a marker size 10
let's do maybe we will do 100 it's just
got a small okay maybe not a hunter you
let's do those do 25 you guys are being
outrageous ok so here we go so there you
have you alright this is a one of your
centroids this is one of your other
centroids and then these are just
colored by their group now just so that
works hopefully you can just see that
that was successful so now you can tell
it's successful by the way that it is
now we're going to do is let's do first
of all just 6 hopefully it would be like
every point
we'll see if it throws out sort through
an error it's probably angry about this
I thought we can do orange let's see our
I don't know if P maybe P is purple
let's try that you're killing me okay
fine we'll just get rid of that and then
we'll say mmm we'll make this list ten
times as long so that'll be like plenty
of color choices to choose from all
right okay so six clusters we get you
know six unique obviously data points
along with centroids that are kind of
behind him now let's see what happens if
you have eight oh yeah okay so this just
has like a simple warning that it's like
hey you're a number of samples is six
and you should be greater than or equal
to eight okay
just curious okay was that exciting or
what all right so now the next but in
the next tutorial what we're going to do
is actually apply clustering to an
actual data set where we believe because
like basically what you might find
yourself using this with is like
consider that you're made you work for
let's say Amazon and Amazon has some
some data on on some users and Amazon
thinks that each you know this data sets
really representative on whether or not
a user is going to be a buyer or a not
buyer okay and so they give you this
data set and then so you're going to use
k-means just to see if it separates the
users based on that really valuable data
that you think defines whether or not a
user is a buyer or not and you're just
going to see it does is that what
k-means actually separates them into
okay and then later on you'll see with
like hierarchical clustering maybe
rather than flat clustering thinking
like oh buyers are just buyers and non
buyers maybe use hierarchical clustering
to find out no no it's a spectrum where
you've got people that are you know
totally not buyer slightly likely to buy
may be likely to buy highly likely to
buy and certain to buy okay or something
like that so anyway so we're actually
going to apply this to national data set
that we think
we'll be descriptive of something but we
want to get some sort of confirmation
that it is so that's we're going to
talking about the next tutorial if you
have any questions comments concerns
whatever up at this point if ever you
leaving below otherwise as always thanks
for watching thanks for all the support
of subscriptions and until next time