Writing our own K Nearest Neighbors in Code - Practical Machine Learning Tutorial with Python p.17 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody and welcome
to part 17 of our machine learning
tutorial series we've been working on K
nearest neighbors and let's get into it
don't want to waste any more time here
so we've started creating this K nearest
neighbors function all we've done so far
though is just warn the user when
they're trying to do something stupid so
now we actually need to create the K
nearest neighbors part calculate who are
the K nearest neighbors what other class
was the class of whatever we're
predicting based on the comparison to
the data okay so what we need to do how
how how could we do this how can we
compare a data point to the other data
points to find out who is the closest
data point and therein lies the problem
with K nearest neighbors to do this we
have to compare the prediction point to
all other points in the data set that's
the problem with K nearest neighbors now
there are actually some additional
things we can do call you can do what's
called a radius so you can look within a
certain radius of a point and then you
can kind of like forget the outliers and
that can save you a little bit on the
calculation of like Euclidean distance
you can find out much easier as a point
inside or outside a radius then you can
to calculate the Euclidean distance okay
so anyways we're not going to get into
that but just keep in mind that this is
the problem with kick nearest neighbors
so what we're going to do is we're just
going to grab a basically a list of
lists of the distances so what we're
going to say is you might say something
like this like you might say distances
equals a list and then you're going to
say a for group in data and so that
would be like for each keep in mind data
is whatever you pass through here but in
this case it would be this data set so
for each group so for each class
basically so for a group in data for
features in group or rather features in
data group your data group
so this was now iterating through the
features here what do we want to do well
we need to calculate Euclidean distance
right and I got rid of the Euclidean
distance so um in order to save y'all
time I'm not going to write it out I'm
just going to copy and paste here
because we're not going to actually use
this version but going with the version
that I wrote previously we would say
that right square root a feature 0 minus
predict 0 squared plus features 1 minus
predict 1 squared and then take the
whole square root of that right that is
the equivalent of what we wrote earlier
unfortunately that is not that is that's
the algorithm but it's not fast it's not
a very fast calculation so I mean on our
data set it would be lightning-fast and
probably on the breast cancer set it
would be pretty fast but in this case no
so so we definitely want to change it so
what we're going to change it to is for
example this would be a faster version
also the other problem here is what if
you have this works with two dimensions
but if you had three dimensions right
this will not change this is hard-coded
to only do K nearest neighbors on to to
feature dimensions bummer all right
there are a few things that we could do
to counteract this specifically with
like a for loop would do it like a
one-line for loop could figure it out
but to counteract for a dynamic number
of features you do something like this
so first of all we're using the numpy
square root we're using num pies version
of sum and then we're also using numpy
arrays where we can actually just
subtract numpy arrays square those
number eye arrays and perfect we get the
same value for euclidean distance here
as well but we're going to go ahead and
not use that one either because turns
out that numpy even has a more simple
version and that would be the following
I guess this one all right out because
you're good this is one you're going to
need to write out so it's Euclidean
distance equals NP limb alg or it's
basically linear algebra log right dot
norm okay it's just another name
basically for what we're going to do
here which is Euclidean distance so this
is the norm
of MP array and we're going to say
features right this is features
corresponding with features - um wait do
we do yeah - MP array predict okay
and this obviously no longer does it
look like that the Euclidean distance
formula and that's why I did not want to
use this initially because this is
you're just you're kind of cheating
using this like you this is much higher
level
so anyways but we're going to use it
because it's faster but I did want to
show you the the true way to write it if
you if you needed to okay so we've done
that and now what we're going to say is
um let's do whoops hit my mouse
let's do over here so we've got Oh
actually what we need to do is now we
need to do distances dot append and
we're going to append the following
we're going to pend the list so we're
going to say whatever the Euclidean
distance happens to be and then we're
going to pend the group so it's the dis
distance it's the distances and then a
group so it's going to be a list of
Lists where the first item in the list
is the actual distance second one's the
group this way we can sort that list
take the three things in that list and
then get the first if element and those
are the groups that's my thinking so now
we're going to come down here and we're
going to say the votes equals so I guess
let me explain how I'm going to do this
first so again I'm going to use a
one-liner for loop but I'll explain
I guess I'll explain I'll write this one
out and then I'll explain it so
basically it is going to be the votes is
equal to I 1 so again I 1 is the group
once we know what the top 3 distances
are we don't actually care what the
distance is we just care about it
because we want to be able to rank the
distances after we've done that and got
in the top three are the best closest
distances we don't care what the
distance was so I want for I in sorted
distances 2k
right because after we've sorted the
distances we only care about the
distances to K so it would almost be the
votes are equal to four this is I I won
but this just allows us to populate
votes as a list that is just simply the
the the groups right are the categories
labels whatever you want to call it
class so there you have votes and then
we're going to say the vote result is
equal to counter of the votes not most
underscore common and how many of the
most common do we care about we just
care about the first one the one that is
the most common and then we're going to
take zero zero so most common it gives
you like a first of all it's it comes as
like an array of a list so you take zero
first you get the list and then you take
the zero with the game because that list
tells you what the most common how many
there were or the most common group and
then how many there were so there's two
things in it I think it's actually a
tuple of an array I can't remember
anyway it's like that so we're going to
take those so in fact I mean we could
print just so you can see counter votes
most common just so you can not take my
word for it okay so so then you've got
vote result we return vote results and
we've done K nearest neighbors so now
let's say we actually want to run this
darn thing and we're going to say
results equals and let's just copy this
copy paste we're going to pass through
data set we want to predict on the new
features and we're going to say k equals
three no problem okay
and now let me print results so we'll
also get this other print out but
that'll be okay and then well maybe get
an error even if we're lucky nope no
you're cool so it's actually a list of a
tuple
okay good thing I printed it outside I
guess I was making it up so what we get
here is that the return is that it's a
the most voted thing was our turns out K
equals three so all three were R and of
course the vote result
are so we get that the class back over
this data point right here is an R type
which as we already saw like we kind of
expected that to be the case so good so
now there's a couple of things that we
want to do first of all we might want to
compare it to psych it learns version
the other thing we could do I mean like
we could print or like we could graph
this just for example so let me do we
could take this cut copy come down here
paste and I'll take this away and new
features so instead well yeah we can
pull out that and then is it color yeah
color equals results I think we'll get
away with that so if we do that right we
just need to change the size but that
was the new point right and so indeed it
classified it in the correct group so
now what we want to do is we want to
compare that to psych it learns version
of K nearest neighbors okay so we're
going to we're going to use this this K
nearest neighbors algorithm against that
breast cancer data and we're going to
see how well we compare two scikit-learn
so that's we're going to be doing in the
next tutorial so if you have any
questions comments concerns or whatever
leaving below otherwise as always thanks
for watching thanks for all the support
and subscriptions and until next time