Classification w/ K Nearest Neighbors Intro - Practical Machine Learning Tutorial with Python p.13 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody welcome to a
new section on the machine learning with
Python tutorial series this section
we're going to be talking about
classification and a handful of methods
for a classification so as we dive in
the first classification algorithm that
we're going to be covering is K nearest
neighbors and but really all
classification algorithms boil down to
the same things so if you recall with
linear regression the objective was to
create a model that best fits our data
and with classification and general
purpose is to create a model that best
divides or separates our data so let's
go ahead and show a quick example so
let's say you've got a graph and then on
that graph you've got some data points
like these and the objective is to
figure out how to separate these into
obvious groups and even just looking at
this intuitively you could see that
there are two groups here one group is
this group and one group is this group
right you just know that's the case so
what we just did just now is actually a
clustering right like with our mind
there when we were just looking at this
and we decided that these were two
groups we actually did clustering
classification is actually even more
simple than what we just did here so
what what classification is going to do
is the following
so with classification you're gonna have
a data set that looks more like this
where you've got a group that you know
are pluses and a group that you know are
minuses and the objective is to create
some sort of model that fits both of
these groups right that that properly
divides them so almost like some sort of
model that defines the pluses and some
sort of model that defines the minuses
so what if you had an unknown
dot somewhere right like what if you
have a data point that's like here
looking at that just visually which
group would you assign that to would you
put it with the minus the bloom Isis or
the green pluses most likely you would
put it with the green green pluses and
then I ask you why why would you have
done that right like what what made you
think that that was the case so I think
about that and then what if we had a
point over here where would you assign
that point well in this case most likely
the blue minuses and again think about
why might you choose that and then
finally what if we had a point maybe
right here in the middle almost now how
would you classify that and it turns out
the way that you would classify that
might actually vary depending on the
algorithm that you're using but in most
cases I think that if you have a dot
like like this one you're gonna classify
that based on proximity to the other
points I think most people when looking
at a graph like this would go based on
proximity than anything else so you're
thinking to yourself well this this
point is closest to this point for sure
this point in this point and what you're
doing when you think of that because
those three points are much closer than
the closest blue - which is all the way
here right that's pretty far so what are
you doing when you do that well turns
out you've just done is nearest
neighbors so with nearest neighbors you
are just checking to see basically who
are the closest points to this new new
point on the data in this case we've got
two dimensional data but you can have
three dimensional ten dimensional and so
on so obviously visually for you looking
at this is super simple but what if you
had yeah like ten dimensions or a
thousand dimensions suddenly you can't
do this by eye anymore and that's where
the machine begins to shine
so that's nearest neighbors but this is
actually most people use
k-nearest neighbors so what the heck is
k-nearest neighbors well it turns out
that if you just try to start thinking
about okay how does this process
actually gonna work do you actually need
to compare it to every single point in a
data set to get your answer and most
likely you don't need to do that but so
with K nearest neighbors we'll just add
a ki suppose here but that's all
together right K nearest neighbors you
decide what the number of K is gonna be
so let's say K was equal to 2 what you
would do is you would find the two
closest neighbors to K and I'm gonna say
visually that is this one and honestly
I'm not really sure which one is closer
of these two I would probably guess
maybe this one my orange line is
definitely shorter but what it doesn't
quite go the whole distance but let's
just say it was closest to that second
one there so with K 2 you've got two
points that are the closest so we've got
basically two points are saying yep this
is a plus but what if you had a point
that was maybe here you might have a
case where what are the two closest
points by K 2 well you would have
probably this point here and this point
here right those are the two closest
points and when K you know within when
the nearest neighbors go to basically
place a vote on the what the identity of
this point is we have a split vote okay
so in general when you do k deers
neighbors you're probably not going to
want to have K equals 2 or any other
even number you're gonna want k equal to
some odd number when in this case we'll
do three so what if we did three what if
we said okay we need one more point well
we would say okay it's this one so then
basically the vote would be negative
negative and positive that's a two out
of three so we would say it's the class
is actually a negative class that's what
we would end up going with here and so
that's basically how cake
neighbors works it's it's a super simple
algorithm and the other thing you have
to think about too is in this case we
had only two groups but what if you had
three groups is k3 gonna be a good idea
turns out no because you could have a
total split amongst all the groups what
about four no because you could have a
totally even vote so if you had three
groups you need at least five total you
know K equals five to avoid any sort of
split vote you can also code something
into just randomly pick if there is a
division what's neat about K nearest
neighbors though is not only can you get
an actual classification for the data
point that you pick you can get what we
were talking about before both accuracy
in the model so that you can train and
test the model for the models overall
accuracy but each point can also have a
degree of confidence so for example what
if you get you you're using K equals
three and you get a vote that is like a
negative negative and a positive well
that's a two out of three right so
that's a you know sixty-six percent
confidence in the score or in the in the
classification of that data but not only
is it sixty six percent at is the
confidence sixty six percent but you can
also have the entire K nearest neighbors
model that you've trained you can have
that accuracy so this would actually be
more like confidence and that's why I
wanted to change when we were doing
linear regression why I didn't want to
call it confidence I wanted to call it
accuracy because confidence with K
nearest neighbors is something you can
actually value and it can indeed be very
different from the entire models actual
accuracy so so it's kind of cool with
with K nearest neighbors now what are
some downfalls of K nearest neighbors
well as we're gonna see in order to find
out who are the closest neighbors what
we're using to measure that distance is
just simple Euclidean distance is what
we're going to be using here
and to to do that to find the Euclidean
distance the most simple method is
actually to measure the distance between
any given points and all of the other
points and then you just say okay what
are the closest three or whatever K is
and as you might guess on a huge data
set that's a very very long and tedious
operation there are a few things that
you can do to kind of speed it up but no
matter what you do to speed this up
you're going to find that the larger the
data set the worse this algorithm runs
because it's just not not as efficient
as other algorithms and then and so once
we cover this and then we get into maybe
like the support vector machine you'll
see that the support vector machine is
much more efficient when it comes to
actual classification also with K
nearest neighbors you're basically
there's never really a point where
you're totally training anything like
the training and testing is basically
the same the same spot or the same
basically the same thing because when
you go to actually test you're comparing
it to all the points there's really no
good way to train a simple K nearest
neighbors algorithm there are also some
things that we can do down the line but
we'll probably won't be getting into
that ourselves but anyway just keep in
mind that the scaling is not so good and
we'll point out exactly why and then
when we get into support vector machines
you'll see why support vector machines
scale so much better than K nearest
neighbors that said I don't mean to brag
on K nearest neighbors too much it's
actually a more than fine algorithm for
many classification tasks so if you're
even if you're working up to maybe a
gigabyte worth of data K nearest
neighbors can still be calculated quite
fast and it can also be easily
calculated in parallel since any any
point you're trying to predict can be
calculated regardless of the other
points that you're trying to calculate
so it's actually you can you can thread
it and still scale relatively well but
if you're working with you know billions
of data points it's not gonna do very
well so anyways that is the theory and
intuition behind K nearest neighbors and
now we're going to actually be diving in
to a real world example of K nearest
neighbors then after that wash
right our own k-nearest neighbors
algorithm so stay tuned for that if you
have any questions or comments leave
them below otherwise as always thanks
for watching thanks for all the support
and subscriptions and it till next time