Applying our K Nearest Neighbors Algorithm - Practical Machine Learning Tutorial with Python p.18 Video Lecture | Machine Learning with Python - AI & ML

what is going on everybody welcome to
part 18 of our machine learning with
Python tutorial series in this tutorial
we're going to take the K nearest
neighbors algorithm that we wrote it
appears to be working and then we're
going to be testing it on some
real-world data and we're going to use
that exact same data set that breast
cancer data set and then when we get our
accuracy back we're going to compare our
accuracy to the scikit-learn accuracy to
see if we did about the same and what I
what I want you to think about is should
we or should we not get either identical
or almost identical results or will the
scikit-learn classifier do much better
than us under the same let's say k
equals 5 parameter so think about that
as we go so the first thing we're new is
we're gonna clean up some stuff we're
going to get rid of this information
here we're gonna get rid of the
matplotlib stuff we're not going to be
graphing there are way too many way too
many dimensions for that one also do we
ask assay Otho how could we know have
known pi we're going to add after
collections we're going to bring in
import pandas as PD and we're gonna also
import random pandas so we can load in
that data set random so you can shuffle
it so we're going to shuffle that data
set because we're not using scikit-learn
it all here we are doing this ourselves
from scratch ok so except for the pandas
part that's going to that would take way
too long but the algorithm ok anyway no
one is amused anyway we'll get rid of
that too so it's just the function in
the imports so here the first thing
we're going to do is DF equals PD read
underscore CSV whoops CSV and don't
forget that CSV let me just copy and
paste it's that breast cancer Wisconsin
data copy paste and don't forget the dot
txt like I did that one time
now we're going to do D F dot replace of
course just like before we get rid of
the question marks and we'll replace
that with negative 99999 now that you
understand K nearest neighbors hopefully
you understand what I was explaining
before about that significant outlier
that
that distance is quite large so chances
are under these circumstances the only
time something would compare to
something like that is if they shared a
missing data point so anyway but we'll
keep it there anyways oh and we need in
place equals true CDF for place in place
equals true now we're going to DF drop
and we're dropping the ID column same
reasons as before that's a worthless
common if you recall accuracy went down
to like 56 or something percent or was
it 51 I can't remember it it's very
close to you know a coin toss so big
deal there full data we're going to say
is D F dot as type float dot values dot
two lists and the reason I'm doing this
is for some reason this data frame like
if I go print lets you print D F dot
head and I'll just comment this out for
now hopefully we'll get when I'm trying
to show you I'm not seeing it but it
exists for some reason some of these
were coming through as quotes maybe
because I've updated maybe it won't but
I'm pretty sure it will so we just want
to make sure that we've converted it to
float everything in this data frame
ought to be an int or a float it happens
to most everything here will be an int
but if you wanted to reuse this code it
would need to be float most likely so
anyway we're going to convert it to a
float and then dot values dot to list so
now we've got the data now we're going
to shuffle the data and now keep in mind
in this case we can shuffle the data
because what we've done is we've
converted this to a list of Lists so for
example let me just print full data um
and we'll do the first ten and I think I
hit run here we go right okay so as you
can see there's the first elements and
keep in mind the two is if I recall
right benign and a 4 would be malignant
but I don't see a four at the moment
and just let me do this real quick I
just wanted you don't have to follow
this I just want to see because I knew
this yes so so converting it to lists
here you can see like this one is in
quotes it's it's being treated as a
string for some reason so this comm
column for whatever reason is treated as
a string probably because it had a
question mark in it um but then again I
don't know because it's been replaced I
really don't know why it's doing that
but anyway that's why we're saying as
type float I values it to list
so anyways there's our data so at this
point we can shuffle this data and we're
not losing the relationship of the
features to the label it's all part of
the same list right so we can shuffle
this and not lose anything so now we're
going to say random shuffle full
underscore data and just to show print
let's do print full data we'll do two to
five and then we'll print full data
again to five after twenty pound signs
just to exemplify something so I just
wanted to show that shuffle applies you
have to redefine so the first one starts
with 5 1 1 1 2 and this one is 5 2 3 and
so on so the shuffle works that was
something that always confused me
initially I would always try to do the
following I would try to redefine the
variable like full data equals randomly
shuffle data that's that's not how it
how it works anyway so that so we've
shuffled the data now and this is going
to be our version of train test splits
and we really high quality code so we're
going to say test size equals 0.2 and
then we're going to say the train set is
a 2 colon empty list for colon empty
list and then test set equals 2 colon
empty lists we should have just copied
this for colon 2 list anyway train set
test set and then we're going to say
train train data is equal to the full
data
whoops not parentheses brackets to the
negative integer value of the test size
times the Len of full data so we're just
we're multiplying the whole test size
point two or we're using that to create
an index value and then we're just
slicing it based on that index value
we've converted it to an int so it's a
whole number and all that fun stuff so
we've done that and let's just copy this
paste and now rather than : - it would
just be a - int - then basically - let's
see - here so this would be everything
up to the last 20 percent of data and
then this will be test we need to rename
this test data will be the last twenty
percent of the data okay so now so we've
shuffle the data we've sliced the data
and now what we need to do is populate
the dictionaries because our we built
this to want a dictionary so now we're
going to populate these dictionaries and
populating them super quick and easy
because all we have to do is the
following
so we're say for I in train data we
could make a one-line for loop here we
really ought to but I'm not gonna train
underscore set I basically this will be
I negative one what are we doing here so
we're saying train set I negative 1
which is negative negative first element
in those lists remember the last column
is the class column that's why we're
using negative 1 that's the last value
so that is either a 2 or a 4 right and
recall 2 is benign 4 is malignant so
that's how we're identifying which one
of these in the dictionary we want to be
a part of so train set I negative 1
dot append append i all the way to
negative one so now we are appending
lists into this list and that list is
elements up to the last the last element
so again you wouldn't want to have the
one of the attributes being the class
because you would get it right every
time most likely and k-nearest neighbors
actually might not but but yeah you
don't want to do that
so now we've done that now what we need
to do is basically the exact same thing
only for the test data so let's take
this copy paste change train to test
train set to test and you're good now
and again you could make this one line
but I didn't want to do that simply
because of the I negative one that whole
stuff that was kind of confusing
probably so anyways we're done with that
whoops what has happened come down here
so we've populated our our dictionaries
so what's left really nothing we just
need to pass the information through to
K nearest neighbors so basically what
we're going to say is we're going to say
let's measure will say correct equals
zero in total equals zero we're just
going to create a simple counter here
we're gonna say it's for group in test
set what do we want to do we're going to
say for data in test set group so for
each group in the test set so this is
test set so for each of these two and
four we're testing these and then we're
going to say for the data in test set
group so just that list of features
right so that's what we're about to feed
through to predict and we're doing this
just so predicted is these lists from
the test set right and then as you might
be able to guess what we're going to
pass through data which goes here which
we iterate through every single point
and calculate the distance is going to
be the dictionary from the train set
okay so for data in test set group we're
going to say the votes is equal to K
nearest neighbors and we passed rain set
that data which is the features and
we're gonna say K equals five simply
because if you look at the scikit-learn
documentation for K nearest neighbors
they are using a default value of five
so we're going to copy that then we're
good all we have to ask at this point to
know if we were right or wrong is if
group equals vote right if if the group
that they came from for them from the
test set because the test that we know
what the answer is so if that group is
equal to the vote we got from our K
nearest neighbors classifier
congratulations
plus equals one for you otherwise are
all so what we need to do is total plus
equals one okay so now we're basically
done so now we would just print maybe we
would say accuracy : comma and then
accuracy is just the correct correct out
of the total so let's save and run that
and see we forget any errors oh we
shouldn't be printing this out LS is
disgusting
okay it went pretty quick anyway
accuracy 0.9 seven eight
so ninety-seven point eight percent
accuracy boom look at us okay I'm gonna
I'm gonna become out okay so um so
that's we've applied it and now what we
want to do is compare that let's run it
one more time without nasty output we're
going to compare that so we ran it again
95.6%
accuracy okay so now what I want to do
is have us compare this to scikit-learn
so we're going to do that and then also
we're going to calculate confidence and
we're going to do that in the next
tutorial so if you have any questions
comments concerns whatever up to this
point feel free to leave them below
otherwise the next tutorial that's what
we're going to do also thanks for
watching things for all this
subscriptions until next time