Creating Our K Nearest Neighbors Algorithm - Practical Machine Learning with Python p.16 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody welcome to
part 16 of our machine learning with
Python tutorial series
in this tutorial actually in the
previous tutorials we've been talking
about K nearest neighbors we talked
about the intuition which is basically
the class of any given point is based on
the vote of K let's say k equals 3 so
the three closest data points to it and
then we talked about in the previous
tutorial that that closeness is measured
by Euclidean distance and then also we
showed the accuracy overall of K nearest
neighbors is actually very impressive
so in this tutorial we're actually now
getting to the point where rect we're
going to write our own K nearest
neighbors algorithm so the first thing
we're going to go ahead and do is I'm
going to remove this stuff here and
we're going to add some new imports here
so first I'm going to say import numpy
as NP and we're going to be using numpy
especially here we're going to actually
change this Euclidean distance formula
to use numpy numpy it turns out actually
has a built-in function that will
calculate Euclidean distance for us the
reason I chose not to use it is it's not
as obvious as this this is very clearly
the square root of x1 minus x2 squared
plus these squared value of y1 minus y2
squared right well that is it's very
obvious this way but with numpy it's not
so obvious but we're going to use numpy
X it's faster but again the whole reason
I wanted to break down the formulas for
you all was so you have a better
understanding of how it actually works
and using numpy right out of the gate
you wouldn't probably as easily
understand how it works so anyways
that's why I did it that way but we'll
keep that there for now but we are going
to bring in numpy here pretty quickly
then I'm going to say import matplotlib
pipe lot as PLT from matplotlib but
we're going to import style from
the collections we're going to import
counter which is how we're going to do
the votes basically then we're going to
do style dot use and we're gonna say 538
also let's we're going to import
warnings important warnings and this is
so we can warn the user when they're
attempting to use a dumb number for K so
now we're gonna get rid of this and
we'll leave Euclidean distance here for
now but um in fact let's just I I don't
know what I want to do with it I think I
was going to delete it for now we'll
rewrite it anyway so now let's say we
have a data set where and we're going to
make this a dictionary and I'm going to
kind of cheat a little bit so we can
make things actually kind of easier down
the line but we're going to say this is
a class we're going to say the class of
K the K class is the following it is a
list of lists and the list in here is
actually coordinates or I hate we're not
going to call them coordinates I
apologize these are features we're going
to call them features this is a machine
learning tutorial I'm thinking about
plotting them anyways so this one has
and these are going to be
two-dimensional features so the K the
features of K are 1 & 2 2 & 3 & 3 & 1
okay so those are features that
correspond to the class of K now we're
going to have another value which will
be R that's the the class that's the
label and that label whoops is not a
dictionary
it is a corresponds again to the same
number of features a lot three features
for R and those features are six five
we'll do seven seven and eight six okay
so we have two classes here and their
features okay so and then let's say down
the line we have a new point or new
we'll say new features I'll probably
forget that later on but new features is
probably better better worded five seven
so that's the new point five seven now
looking at this visually you can
probably are
any surmise which one this this better
belongs to so you can take a wild guess
but and don't worry soon enough will
actually will write this algorithm and
then we'll apply it to a more realistic
data set this is just simple data set
for now and I'm going to write a
one-line for loop that's going to graph
this so let's say we're going to say PLT
dots scatter uh and we're going to
scatter let's say peel t scatter and
we're going to say I I 0 actually you
know what I'm gonna do first let me
write it out so let's say you had for I
in dataset for III in dataset I
so we're iterating through you know for
I end dataset I corresponds to K NR and
then for I I and data set I that would
be for each feature set basically so for
I I we're going to do PLT dot scatter
III 0 comma III 1 s equals 100 color
equals I and that is why I use those so
so so how might you break this back down
to be a one-liner for loop you would
just you could do basically this so PLT
dot scatter that I color equals I and
then do this for I I in data set I and
then for I in dataset okay
that's usually how I build a one-liner
anyway so this should do what we want to
do and then let's go ahead and peel T
dot show to see our data are you there
it is okay so there we have our data and
again it's pretty simple to see kind of
what we're working with and in fact we
can even do here PLT dot scatter
I think we'll have to do this zero one
okay
that was pretty small we should have
done size equals 100 but you get the
point so just looking at it visually
we're probably thinking okay that
belongs to the red group right so so we
have our data we are scattering that
data just so we could visualize it but
now let's get started with defining the
K nearest neighbors algorithm so I'm
going to comment this out for now but we
at least wanted to see our data so we
know that eventually we want to have K
nearest neighbors as if we'll do a
function and in order to calculate K
nearest neighbors we need to pass
through data right this is the training
data basically and then we need to pass
through whatever we're trying to predict
right and then we need a value for K
okay and for now we're going to default
K 2 equal to 3 which is the same I
believe as psychic learns K nearest
neighbors it might be 5 but we'll look
at that when we come to it so then what
we're going to say is for example we
know that eventually what we want to
have is if Len data is less than or is
greater than or equal to K I if we have
more because we're going to pass through
a dictionary right in the length of a
dictionary like the length of this
dictionary here is 2 it has to basically
keys okay so that would be 2 so if the
length is greater than or equal to the
value of K that we choose we're going to
just send the user a warning so we're
gonna say warnings not warned and then
we're just going to say you're dumb
never mind where is that k is set to a
value less than total voting groups
something like that idiot just kidding
okay so anyway you can leave you there
if you like and then and then we're
going to do you know special KN and algo
and then we're going to return the the
results and it's probably going to be a
vote results so we're just a vote
results okay so that's the starting
point of our
k-nearest neighbors algorithm and i'm
going to go ahead and cut it off here in
the next tutorial we'll continue
building this to the point where
hopefully we can pass out data and
actually get that vote result and
probably bring at least this one to a
close and then we can test it on real
data so anyways if you have questions
comments concerns whatever post them
below otherwise as always thanks for
watching thanks for all sports
subscriptions in until next time