Final thoughts on K Nearest Neighbors - Practical Machine Learning Tutorial

what's going on everybody and welcome to
part 19 of our mushroom learning with
Python tutorial series in this video
we're going to be talking about K
nearest neighbors one last time before
we jettison off into the support vector
machine what were we talking about here
is K accuracy and predictions first
question we might have is well what if
we if we increased K would accuracy
necessarily go up so just for fun let's
um let's say K is equal to 25 and we run
that the previous accuracy was about 95
now we got 88 or 98 rather sorry I can't
read for CREP
run it again okay 97 run it again 94 run
it again okay you get the point so we're
still we're still doing pretty good what
if we raise it to 75 95 percent okay
still doing pretty good 97 percent still
doing pretty good okay 93 is one of our
lower numbers anyway oh we don't really
seem I mean we're still kind of the
average around here is about 97 so um
you know we can keep going I forget how
many data points we actually have in
this set let's see which have 600 about
so if we said K is equal to 200 so one
of the closest 200 points we see
accuracy actually appears to be worse so
looking at more points and actually
probably what's happening there is
recall that we actually have a skewed
number only like 30 percent were
malignant where right yeah 30 percent
were malignant and the other like 70
percent were benign so this is you know
adding decay doesn't necessarily do you
any more of a favor a 5 is probably a
pretty good just guess but depending on
your data set and all that you you might
want to fiddle with K just to see if it
makes a huge difference and just test on
whatever data set you might have so
anyway that's K
next we're going to be talking about
confidence versus accuracy and k-nearest
neighbors can give us confidence so
accuracy is did we get the
classification right but confidence
actually can come from the classifier
and the classifier can say hey we have a
hundred percent of the votes in favour
of this class being such-and-such or
conversely it can go back and say hey we
have only 60 percent of the votes this
is what the vote was but our confidence
is only sixty percent right so for
example you could create confidence and
confidence would be basically this this
and instead of that zeroeth element it
would be the first two so that's how
many and then what would it be it would
be divided by whatever K was right so
you're hoping if you say K is five
you're hoping that this number is five
so anyways there's that and we'll just
leave it like this
simply because uh every other like
accuracy and stuff is the decimal form
so we're not actually x by 100 so you
can have confidence there and then you
literally could just do confidence okay
and we can let's see vote and then
confidence would be here so let me make
sure we get away with that right so
there you have that and then basically
what we could do is we could um you
wouldn't want to do this all the time
but you could print you know vote result
and foot result confidence so there we
go so almost all of these were a you
know 1.0 confidence this one was eighty
percent but this one was sixty percent
and the interesting thing would maybe be
to say vote confidence if that is the
case else print print the confidence so
every printout of confidence so we're
going to comment this out so all the
numbers we're about to see are the
confidence score
of the votes that we got incorrect so
run this okay so actually this is a
pretty good split here this wasn't these
were 100% votes that were incorrect but
then we also had some six votes that
were incorrect let's go up here and
let's maybe change the test size to 0.4
we're going to sacrifice a lot of data
so accuracy might actually go down but
the question was in my mind would you
see less 100% confidence if if that in
the incorrect valuations and sure enough
it appears with this at least this test
this was a pretty even split but the
ones that it was unconfident about was
60% so you might even get to the point
though where let's say you have someone
in your about and you're telling them
whether or not they have cancer okay
you're telling them whether or not they
have cancer if the confidence is only
60% you might have you might you know
not say anything right you might say
well the comments that the test was not
accurate or whatever or the test was not
confident right and so you know that
might be a reason why you might want to
take a little more care in your reports
anyways so uh that's Kay's confidence
and now comparison so what I want to do
is let's see what I want to do is take
this here and basically we're going to
take this and let's say for I in range
25 and we're gonna say for I in range 25
we're going to do well actually yeah I
will just will be sloppy about it
whatever tab that over and we're gonna
say accuracies equals a list and then
here we're gonna say accuracies dot
append
that number and then when we're all done
we're going to say paren we're gonna say
Len no some some accuracies / Len
accuracies and let's just do 10 well do
five first just to make sure logic oh we
shouldn't have we have to get rid of
these printouts right okay
so we'll get rid of the pronounce and as
we can see at least out of five tests
where we average ninety six point two
percent accuracy and the other thing we
need to make sure yeah k is indeed five
we'll stop printing accuracy and we will
stop where's the other thing we were
printing oh it's here else print
confidence let's just delete that okay
so now now we've got that and we are
getting different accuracies that just
to show the whole process is being
repeated over again so let's run this 25
times may not be enough for you but
we'll run that 25 times and then let's
go to the other K nearest neighbors
let's say you don't have to follow along
if you don't want to by the way you can
just see what the result is I believe
that this is the other K nearest
neighbors from part 14 what we could do
is do basically the exact same thing
here right so let's take this here tab
over paste for iron range that I'm going
to tab I'm going to stop this accuracy
stuff and in fact we don't need to do
predictions either I'm going to leave
them there just in case I because this
is the old code actually so so I usually
upload this to github so anyway we'll
save that and then accuracy up and
accuracy and then we print some
accuracies / Len
accuracies accuracies okay so out of 97
urban out of 25 examples we ran a ninety
six point four percent accuracy on the
version we wrote now let's test K
nearest neighbors okay so this is the
average accuracy for K nearest neighbors
ninety six point eight percent the only
thing I will stress though is the
following keep in mind on both of these
we're basically doing the exact same
thing we're loading in the data frame
we're passing the data we're training
we're testing and we're doing 25
iterations so so again let's run this
one and it's still running and I'm going
to go it's still running I'll run this
one too and it there's your answer there
and then we're still waiting on the war
on the version we wrote and we're still
waiting oh man
anyway um so so what's the difference
okay there's a couple of differences
here one is K nearest neighbors has a
default parameter actually I think the
default end jobs is equal to one but a
couple of things one K nearest neighbors
can be threaded okay so you do not have
to test each prediction point or like
like in tests you had twenty percent of
the sample right so when you go to test
each one each like set of features is
its own unique snowflake so you don't
actually need to test linearly you can
test all test each one on their own
basically so you can heavily thread K
nearest neighbors on a bulk of
predictions so first of all you can do
that but I actually I think that the
default is they they do not let me pull
up scikit-learn scanners neighbors for
you guys so here we go and neighbors
default 5 sure enough radius this is
most likely where they're they're
winning they're beating us because
they're using that idea of that radius
to ignore points that are outside the
radius most likely n jobs this is what
we're talking about so how many parallel
jobs do you want to run for the
neighbors sir
the default is one but if you set that
to be negative one where are we here we
are then it will do as many jobs as
possible so that's another way to
actually speed it up but as you can see
it was actually already pretty fast but
this is not a high performance tutorial
and Python this is a machine learning
tutorial so if you want high performance
you'll most likely be using someone
else's algorithm but our accuracy was
actually very similar to theirs and I
would wager that if if we ran like a
million samples we would find that
accuracy was identical it's just 25 is
not actually a decent sample size so
anyways that's all for K nearest
neighbors a very valid algorithm to use
you can use it on pretty like a lot of
people say it doesn't scale well but it
scales to very large data sets
it just doesn't scale to like terabytes
of data it's just not going to run very
well on terabytes of data but you can
use a radius you can thread it and it
actually still will run pretty well and
the accuracy is pretty good so one of
the other upsides of the K nearest
neighbors classifier is that it can work
on both linear and nonlinear data so
you'll see that that makes a big
difference especially in something like
the support vector machine which is what
we're covering next but to use
algorithms that we've already covered
you actually can use regression to do
classification so long as you're using
linear data so for example I've got a
graph here and if I was to draw the best
fit line the Y hat line the regression
line for these two data sets I would
probably do something maybe like like
this okay so it would be for the blue
dots and then I might do something like
like this for the orange dots and then
if I had an unknown data point like
let's say maybe this point here without
using K nearest neighbors instead what I
could do is I could measure the squared
error between this regression line and
this regression line and whichever one
had a lesser squared error that would be
what class that new X plot belonged to
which in this case would be the orange
the orange class right so
with regression if the data is indeed
linear you can still do classification
right you don't just have to forecast
out now what about a dataset like this
though this dataset both of these the
orange dots and the blue dots do have a
best fit line I have no idea what they
are but they do have a best fit line but
even if they did even if when we when we
did draw the best fit line the squared
error the coefficient of determination
would be so poor so the actual you know
confidence of this algorithm would be
pretty bad overall so you can't really
do classification on a data set like
this this is nonlinear data but you can
do K nearest neighbors on nonlinear data
so you know we could still we can
classify this point using kada nearest
neighbors because then we're just we're
measuring the distance between this
point probably this point and this point
and those are our closest three points
we would get if K was equal to three we
have two to one vote so we would vote
that it's an orange and orange dot class
right so anyways that's one of the other
upsides to the K nearest neighbors
classifier that's it for K nearest
neighbors the next topic we're gonna be
talking about is the support vector
machine so that's what we're gonna be
getting into if you have any questions
comments concerns whatever feel free to
leave them below otherwise as always
thanks for watching thanks for all the
support subscriptions and until next
time