Mean Shift Intro - Practical Machine Learning Tutorial with Python p.39 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody and welcome
to part 39 of our machine learning
tutorial series in this video in the
coming videos we're going to be talking
about mean shift and mean shift is
another clustering algorithm but this
time rather than the flat clustering
methodology of the k-means clustering
algorithm mean shift is what is known as
a hierarchical clustering algorithm now
among other things what this means for
us is that unlike k-means where we had
to tell the machine hey we want two
clusters or five clusters or whatever
with means shift and hierarchical
clustering the machine actually figures
out how many clusters there ought to be
and where those clusters actually are so
to exemplify this first I'll just draw a
really simple data set and then we're
going to have some data points here okay
and if you recall with k-means for
example when we started the algorithm we
just we said okay we've got K let's say
K equals 2 we're going to select
randomly to or select randomly K feature
sets and call those centroids or cluster
centers for now okay with mean shift
instead what we do is we say every
feature set is a cluster Center so this
is a cluster Center this is this is this
is this is this is and this is and what
you have here is a cluster nevermind
anyway so you've got all of these as
Center clusters so then what's the next
step so mean shift has one thing that
the scientist might supply but as we go
through it I'll show you a way that you
can kind of determine it automatically
and that is called the radius so the
radius also sometimes referred to as the
bandwidth but they really are not
totally interchangeable so basically
what's going to happen is you're going
to have a radius around a data point so
we're just going to start with 1 but
understand that this is going to apply
to every single data point so let's
we're going to work first with this data
point here but imagine this process is
applied to every single feature set data
point whatever you want to call it
so every data point has this this
bandwidth around it so let's say this
this feature set that we're working with
has a bandwidth and it's like like this
it would be a perfect circle so that's
your bandwidth and then really your
radius is basically this distance right
all the way around that's radius okay so
the bandwidth is really everything
within that radius and the radius is
that distance okay so for example this
starting cluster has three feature sets
within its bandwidth which is determined
by its radius so again people like to
interchange those I really feel like
that's not interchangeable but anyways I
could be wrong
so so each keep in mind though again
every single cluster Center goes through
this process so this is just for one
cluster Center okay
so then once you have your bandwidth and
all the data points within the bandwidth
you're going to remember with like
k-means for example you're then going to
take the mean of all the data points
that are within that cluster so in this
case you've got three data points what
would be their mean well it's probably
like right here okay
so now your new cluster Center is
actually this point right there okay and
then with it comes a new bandwidth okay
so there's your new bandwidth okay and
then again we take all of the feature
sets or data points within that
bandwidth and then we take the mean of
all of those so that would be basically
right it would be this one this one this
one and this one and then we take their
mean and I don't know it's probably
something like here okay so then you got
your new cluster Center which is there
okay and then again you take the
bandwidth which would probably not
change too much but it would be like you
know like this okay and then again at
this point it would be all of the same
data features within that bandwidth in
that I guess this time maybe within the
radius and so that cluster would not
move and when that cluster Center does
not move anymore
it is it's optimized okay it's done
the same thing is going to happen though
remember we started this entire process
with this data point right we said that
was a cluster Center so then if we were
to do this exact same process with this
data point knowing what we know from
what we went through with like k-means
and and and what you just saw now what
is that new the second data point right
what is that data point where is that
going to have a cluster Center well when
you run this it's going to most likely
within one step reach this exact same
cluster Center and what you have there
is convergence
I guess I'll write it all out
convergence and when you have
convergence you're done right so when
you have convergence you just say okay
well both of these became the same
cluster centers so they are the exact
same closer centers so just even though
you started as all data points are their
own cluster centers as you continue to
run the mean shift algorithm you're
going to be whittling away cluster
centers and then you're also at the same
time going to be slowly moving the
cluster Center to a point of convergence
and basically if you recall kind of like
with k-means once you have a bunch of
cluster centers that are done moving
they're fully converged nothing new is
happening you're done you're optimized
and you're good to go okay so as you
might guess as you did this to basically
all of these other cluster centers they
would all at least down here go to that
exact same cluster Center and then up
here they would probably all converge
about right there and they would all be
within their own little radius there and
you're done you've got convergence now
sometimes that's not going to work like
just with a simple radius and bandwidth
and that's kind of what we're going to
be showing when we actually go through
with Python but just as a quick example
let's say you've got a data set like
this okay this obviously doesn't really
appear to have any clusters but I just
wanted to show a really quick example
but with the bandwidths you're going to
have it you might have different levels
of bandwidth so maybe the first level is
this
and then you've got a secondary level
which is going to be this in a tertiary
level like this and so on and then the
the if you find feature sets within this
first level then maybe they have a
weight of three assigned to them and
then out here these are data points that
exist but they only have a weight of two
and then these will have a weight of one
and so that way your your ear
encompassing basically all data points
for the most part or a larger range of
data points but you are penalizing for
distance away you could also penalize in
some sort of squared error fashion or
something like that all right so I'm
just going to show a really quick
example here there's really nothing new
being covered here besides maybe 3d
graphing I'm not really sure if we've
used that yet but if you want to know
more about 3d graphing we haven't talked
about it you can check out the
matplotlib series and learn more about
it mostly I just want to show a really
quick example and some sample code but
really there's nothing new here so I
think it'd be kind of a waste of time to
be rewriting this script when we've
covered basically all of this stuff so
anyways we're just creating some
starting centers here so that we can
create some sample data these centers
are not used in the rest of the code or
the mean shift algorithm or anything
it's just simply so we can create this
starting sample data also it'll be
interesting to compare since this is
where the data started from we can grab
the cluster centers and compare what
means shift found to be the cluster
centers with what the random data was
generated from and they should be pretty
darn close right otherwise something
went wrong so that's that basically
that's pretty much the only thing I
think worthy mentioning because
basically what we're going to do is we
just we fit to X whatever X is from make
blobs from scikit-learn
we get the cluster centers which are
attributes the number of clusters
hopefully three plenty of colors to
choose from we're just multiplying this
list by ten to cycle through so in
theory we can find
like 70 colors creating the graph 3d
graph graphing it scatter show cool okay
so let's go ahead and just pop up an
example real quick and see what k-means
does for us so sure enough we get this
beautiful 3d graph and we've got three
clusters here beautifully colored and
everything got this one will outlier
interesting in one room
I wonder which cluster generated this
little guy k meter mean shift says it
was the blue group but man does that not
look closer to the red in basically all
examples maybe in this one I don't know
anyway you guys probably aren't seeing
the same thing okay so then we can
compare so here are the cluster centers
so maybe not in the same order but we
can see this first one right we got 1 1
5 5 or 1 1 1 5 by 5 3 10 10 well this is
obviously the 1 1 1 pretty close there's
a little bit of deviation from that but
not more than would be expected because
this is just these are centers to start
from and then they're just randomly
created from there and then you've got 3
10 10 obviously and then this is your 5
5 5 so pretty close to where the random
data was you know generated from and
probably if we made like I don't know a
thousand samples it'd be even closer to
those starting numbers so if you're on a
laptop or something like that I wouldn't
recommend running running this is going
to be like a pain on your on your
computer but anyway so that in this case
we've got now a thousand examples and
here are new clusters so even closer to
one one one closer to 555 and even
closer to 310 ten and probably we could
do like 10,000 or something but even
this is going to be like probably really
hard for my computer to shift around
sure enough yeah okay anyway so there's
a really quick simple example of mean
shift at work but again like kind of
like I was trying to explain before the
clustering algorithms are really not so
much for you might use them for
visualization but really they're for
like some sort of research structuring
data that kind of thing so in the next
tutorial what we're going to talk about
is applying mean shift to that titanic
dataset they work
because we are under the impression that
hopefully it will find two groups but
maybe that's the wrong the wrong
I suppose thought on that data set maybe
that data set actually has five groups
or something like this that we haven't
really you know in our head we know it
should be two groups but really maybe in
reality these are separable into more
groups like for example we already know
right out of gate that there are three
classes of passengers on the ship first
second and third class right okay that's
for sure
so so maybe three classes is actually
what the true the true number of groups
is on the ship and then from there we
can do some more interesting research
and so that's what we're going to kind
of start toying with in the next videos
so if you have questions comments
concerns up to this point but for you
even below otherwise as always thanks
for watching thanks for all the support
subscriptions and until next time