Mega-R5. Support Vector Machines Video Lecture - Artificial Intelligence

the following content is provided under
a Creative Commons license
your support will help MIT
OpenCourseWare continue to offer high
quality educational resources for free
to make a donation or view additional
materials from hundreds of MIT courses
visit MIT opencourseware at ocw.mit.edu
hope everyone had a great Veterans Day
break yesterday spending it of course as
I checked my tests on Ian's spending it
mostly doing your opera research or
psets but at least one person got to
watch TV so at least one person got to
have a real break and that's something
truly amazing and special so now we're
going to talk about svms
they're pretty much the hardest thing in
6:54 however in recent in recent years a
few shortcuts have popped up that will
sometimes allow you to solve the
question depending on what they're
asking for without solving some vast
ugly set of equations with the vast
Uglies number of unknowns so I'm going
to show that to you guys
I'm also going to try to explain all the
alphabet soup that's in neural nets and
what all the letters stand for because I
took me a few times going to North go to
the S campus it took me a few times
going through SVM's to actually find out
for sure what all those letters stood
for and if you guys figure it out first
try that's going to be great and you
guys will be just fine so let's take a
look at the problem is perhaps most
optimized for using some of the
shortcuts to solving it and not putting
up all the equations then I will because
I am not because I'm sadistic but
because I'm being nice I will force you
with me to solve some of the things they
didn't ask for us to solve so that you
can see that we can't get away with
everything without doing some of the
harder stuff and
of course definitely ask questions as
always but this time even more so you
guys well if you were looking around you
saw nobody in this entire lecture hall
raise their hand that they're already
set and ready and no svm so if you have
a question maybe everybody else does
so let's go
you'll start right here as always
pretend that I can draw and therefore
all the pluses and minuses are only on
integer coordinates so we are asked in
this problem to circle the support
vectors draw the edges of the street and
then the dotted line in the middle that
separates them the separator as a dashed
line and then to give both W and B so
what are W and B well there's a few
important equations in in svms that we
really hope and I'm going to tell you
we're lucky in this because we don't
have to but we really hope that we don't
have to use because they provide a huge
number of variables so one of those
crucial equations is that for a spur a
plus support vector W vector dot X plus
the plus support vector plus B equals 1
WX minus plus B with minus 1 and W dot
that dotted line I don't know we'll call
it dot dot plus B equals 0 so what does
this mean there are a lot of vectors
well I mean we're usually in
two-dimensional space so we can
basically just say that there's two
components of this W vector W 1 and W 2
and they're just two coefficients and a
linear equation so to get for instance
what we're interested in finding this
that dot line we'll just call that X
with nothing on it actually maybe
that'll be easier this is equivalent to
saying W 1 X 1 plus W 2 X 2 plus B
equals 0 where X 1 is this and X 2 is
this we would possibly call them x and y
so one way to think about it is you know
W 1 we call a a x+ called W to be B Y oh
don't call it B
well a X plus CY plus B equals put this
on parenthesis this is basically an
equation like this or you know y equals
negative a over C X minus B over C it's
basically y equals MX plus B does
everyone see that this thing that we're
looking for this w dot X plus B equals 0
is the equation of a line on in
Cartesian coordinates it just looks
uglier normally when we're doing all
this solving for W and B we'd have to
put in equations with the you tons of
equations plug in all the support
vectors in there and we have to use
these little devils called alphas which
alphas essentially if it wasn't clear in
the lecture which it usually isn't
completely clear to everyone wasn't
clear to me completely alphas way I'd
like to think about them the Alpha is in
this problem is they are the weight of
how significant any particular point on
the graph is towards creating the
boundary this the higher the alpha is
the more that that point narrows in the
boundary the lower the alpha is the less
at that point narrows in the boundary
the wider the road can be and if that
point
do anything if that point is irrelevant
and could be removed it wouldn't affect
the boundary the Alpha is everyone zero
well that was one person but you can
suffice for everyone the Alpha is zero
and that means if it's not a support
vector if it's not one of the vectors on
the boundary lines it will always have
an alpha of zero because it doesn't
affect so keeping that in mind there's a
few fun and important equations about
alphas that will mean if we're solving
many equations for many unknowns which
hopefully we won't have to do the sum
over the positive alphas equals the sum
over and the alphas of the negative
points this is true over all the points
but since all the alphas are 0 except
for the support vectors it also means
the alphas of the positive support
vectors are equal to the alphas of the
negative support vectors additionally
our buddy the W vector is equal to the
sum over the all I that R plus vectors
of WI alpha I minus M over J minus
vectors of W J alpha J now all these
equations can be used in a bloody mess
to figure out the answer to what we're
trying to find which is circle as well
actually they can't be used to circle of
support vectors and draw the draw the
dotted line but once we do that all
these equations can be used in a bloody
mess to give us the but the next thing
that we want which is W and B so
fortunately there's another way to get W
and B if you guys really want at the at
the end of the hour we can also try to
get try to drive W and B using the many
equations and many unknowns but it's a
bit painful we'll try to do it the cool
way so let's start
this is the one we're looking at we need
to find what the support vectors are so
first thing we need to do is simply I it
fortunately on the test there will
always be ones that you can I if you're
supposed to circle the support vectors
there's obviously some number of pluses
and some number of minuses I say
obviously but maybe not but hopefully
obviously and we'll find out because I'm
going to call them random people so give
me a positive support vector yeah
perfect that's good all right that's
what ah not give me a native support Oh
No ah that is a good question
question is what is a support vector how
many other people are will meant to
having this question see you're not
alone okay before I go on I'm going to
assume guys make sure I'm correct money
was uh just being served so I can tailor
based on this Monday was the support
vector machine lecture but it was also
very difficult to follow that's what I
that's what I usually expect so um what
is a support vector well all these
pluses and minuses if we were if we were
me and we if I guess yeah if we were me
and I was describing this problem the
one that we work out in class I would
call them points because they're on the
graph their points their data points I
would cut but however in more difficult
versions of this problem that have n
dimensions where n is some ridiculous
number of dimension
that you're never going to graph like
say you know some of the research I'm
doing now I could use support vector
machines on some of these articles that
I'm reading about cyber events to try to
figure out if there's a real event or if
it's just someone complaining about how
we're really vulnerable or something
like that and no event actually happened
so the reason why they call these guys
vectors is when you're not able to graph
them on a Cartesian plane there's still
this long you know vector of many
different dimensions right now though
these points represent the vectors it's
very simple it's easier to view them
this way but for instance that plus at
negative 1/2 represents the fact that
there's a vector going in the direction
of negative 1/2 with the magnitudes such
that it reaches negative 1/2 so all
these points are just a point
representation in vector you probably in
any class that worked with vectors saw
this saw vectors being represented as
points question hi the question is all
respect to the origin answers
economically when directions are
represented at points yes it's always
good respect to the origin so that's the
basic idea is that all these points are
vectors so what are support vectors well
we could call them support points for
this case but the reason we call the
support vectors is again in the
generalized case that you might be doing
in the real world with real AI you're
going to have the giant vector it's not
just going to be points on a graph well
usually so the support vectors the
support points we found one of them
correctly
it's this guy they're going to be the
ones that again they don't have an alpha
of 0 they're the ones that bind in the
as petracles of the road the boundary
lines they're going to be on the edge of
plus whichever direction we draw it this
plus is the edge of the plus region if
we made this the edge of the plus region
and everything on this side is plus
everything on this size - we'd be
screwed because there's two pluses on
the on the other side of that generally
when trying to find
support vectors you do something a
little bit similar to my crazy method of
doing nearest neighbors and try to find
a plus my excuse me a plus minus pair
that's close to each other sometimes
though it's not just two points because
sometimes if you try to draw the
simple-minded thing which is sort of the
perpendicular bisector of the two points
you get screwed because there's another
point in your way so now that I've given
away a clue let's go
and hopefully that made sense to you
guys the support vectors are the ones on
the edges they're just barely a plus for
sure but barely - for sure let's go back
can you give me a negative support
vector the one on Paulo yeah and does
anyone think that there's a third
support vector well let's simple-minded
Lee try the thing that remember support
vectors always attempt to have the
widest possible space between the pluses
and the minuses that they can so let's
simple-minded Lee try to do the
perpendicular bisector and see if it see
if it screws us over so when we
simple-minded Lee do the perpendicular
bisector it goes through like here like
this and it's just fine so these are the
only two support vectors
and there's our divider line so we're on
the homestretch but we have to find W
and B in olden days we would find W and
B by plugging in W dot the plus support
vector plus B equals one that's very
crucial these W dot X plus X minus our
only true equaling one or negative one
are only true for support vectors it's
always true that W dot any positive
point plus B will be some positive
number but it won't always be supportive
it don't won't always be one in fact it
will always be the greater than one up
over here it will always be less than
negative one down over there
so in olden days we would plug in
negative 1/2 into this equation we plug
in 3 negative 2 into this equation we
plug in alpha plus equals alpha minus
it's the sums and since there's only one
plus or minus we know that they were
equal and then we fidget around with
this W equation however there is a
better way to do it and so let's use
this cheap strategy to solve to solve
this version of the SVM here's how first
and I know I didn't draw these
completely straight sorry but can anyone
by looking at the fact that this is 3
negative 2 and this is negative 1/2 can
anyone tell me what the equation uh if
you can do y equals MX plus B can anyone
tell me what the equation of that dotted
line is supposed to be if I was a good
at drawing you said Y minus 1 and I say
yes why I'm just - why so therefore the
pluses would be y is greater than or
equal to X minus X minus 1
indeed so we've already seen that WX
plus B like somehow can be converted
into this form right so therefore if we
have y equals X minus 1 then we know
that
we have in w1 that we know that we have
W X plus B equals zero let's do that
here so we know that W 1 W 1 X 1 being
even call it X and why I think it'll be
fine no one will come after us
w 2 y plus B equals 0 but we also know
that y equals x minus 1 which means that
if y equals X minus 1 then that means
that according to this thing that we
have over here then negative w1 over w2
equals so we know that negative w1 over
w2 and we have negative negative B over
w2 so y equals x minus 1 and if we solve
this equation to make it look like this
we would have y equals negative w1 over
w2 minus B over w2 so we know that in
some way shape or form that we know that
Nick did then therefore w1 over w2 is
some scalar multiple of minus 1 and we
know that B over w2 is in fact some
scalar multiple of positive one scalar
multiple scalar multiple well why is it
a scalar multiple why is it it just
going to be negative 1 or positive 1
because just because in this equation we
can multiply the entire equation by any
number and it will still have the same
boundary line you guys see that if you
know if X here if we multiplied
everything since it's all divided by
- if we double w - but also w BB and w1
it would be the exact same question do
you guys agree so there's in fact the
infinitely many possible equations you
say well great mark you figured out what
form it is so you figured out that w1
over w2 equals on some scalar multiple
of negative one so it's negative one
times what's everyone's favorite letter
okay
negative 1 times K and we figured out
that B over w2 is I guess we can just do
negative K is positive K but what's K
how are we going to figure it out well
it's a good question I will tell you how
I'll assert the following fact as true
without proof then I will not prove it
1 over the magnitude of W which is this
vector here duck with w1 w2 equals this
where this is that line that I just drew
the line from here to this point 1 over
the magnitude of W equals this therefore
since 1 over the magnitude of W equals
this and this equals I believe 2 root 2
because we're going over to down to
Pythagorean theorem 2 or 2 so therefore
flip everything over magnitude of W
equals 1 over 2 root 2 so therefore
magnitude of W equals root 2 over 4 but
why are we okay well how do we calculate
the magnitude of W do people know in
general magnitudes of vectors generally
for these vectors we do it by thing the
square root
of the sum of the components squared so
the square root of W 1 squared plus W 2
squared equals root 2 over 4 but that's
not all that's not all we say because we
know from this over here that the ratio
of W 1 and W 2 is yeah the ratio of the
ratio of W 1 and W 2 is going to be
actually sorry I shouldn't put a K here
I realize I probably should have been
confusing you guys a lot w1 over w2 is
just negative 1 B over w2 is just 1
that's just the fact there's no K the K
is to determine what W 1 and W 2 is w1
and w2 are so W 1 equals like negative K
and W 2 equals positive K and then B
equals also positive pay by the way
here's a question for you could I have
put the K the negative sign on a
different I'm going to put the negative
sign on W 2 and B instead of on W 1 so
maybe people said yes that's a very
strange actually not because of the fact
that the pluses are on the negative
x-axis it's just a little trick I picked
up when one of them is negative in the
other one is it follow the pluses so we
know that W 1 is negative K W 2 is
positive K and B is positive K w1 over
w2 is negative 1 B over W 2 is positive
1 so what do we know about the ratio of
W 1 and W 2 it's equal to negative 1 and
that means that when we square it W 1
squared equals W 2 squared so therefore
this is the square root of 2 W 1 squared
which equals root 2 W 1 equivalently
well actually
no it doesn't equal root 2 w 1 because W
1 is actually negative so it we you know
it's negative 2 W when it doesn't matter
the point is that if that equals root 2
over 4 then W 1 is everyone negative 1/4
bingo
and if W 1 is negative 1/4 everything
else falls into place
butter W 2 and B everyone positive 1/4
we got it we've done with this part of
the problem however bonus let's do
conquer the alphas which they didn't ask
you to calculate actually you know what
we'll do the office if we have enough
time since they didn't actually ask you
to calculate them however my
recommendation is since there's only one
alpha plus and 1 alpha minus they must
be equal from this equation
since this is some of the alpha plus
equals the sum of the alpha minus and so
therefore some of the W or say W equals
the sum of W sorry should be an X of
course there's not a million w's in this
equation the sum of the points the
positive data points times their alpha
is minus the negative data points times
their office so we're looking at here on
negative 1/4 1/4 equals um what do we
got here positive point negative 1/2 so
we've got alpha plus or alpha of that
point negative 1/2 minus alpha of that
minus point and what is that it's 3
negative 2 3 negative 2 so if both of
the alphas which are equal were 1 we'd
have negative 4 and we'd have negative 4
4 but we want negative 1/4 1/4 so
actually both of the alphas are 1/16 and
that's the answer
we'll do that boring gap if we have time
but we won't so let's do number two so
let's go into faster mode number two
very similar to number one in many ways
but as you can see one of the main
things that they added an extra minus
sign at two negative one so I think we
can all agree that this will still be
our plus actually add another plus sign
there too so maybe this plus sign is a
split vector but it's not this plus sign
is the support vector what do you guys
think about the new negative sign will
it become the support vector since it is
like strictly closer to the pluses yup
you're right
okay so this is a very beautiful
division because if I do this correctly
which I didn't but if we pretend that I
did then the dotted line is y equals x
okay so with the dotted line at y equals
x then we know that just like we did up
here we know that if y equals x plus 0
we know that first of all B equals 0
second of all we know that we've got
just with y equals x then we know that
negative w1 over w2 equals 1 the plus
are still on the left and up so we know
that w1 is some negative number negative
K w2 is some positive number K great how
are we going to figure it out well let's
call that this D for distance or
whatever you want to call it so on over
W equals D D in this case is not on 2
root 2 can everyone tell what D is here
it's actually so it goes over two and
one so it should be one and a half root
two
since this with distance which is twice
as much goes over 3 and 3 which is 3
root 2 so it's one and a half root 2 I
don't like putting in decimals and stuff
there so we'll say that 2 over negative
W equals to D equals 3 root 2 so
therefore right high-diving 1 2 3 1 2 3
3 root 2 so therefore magnitude of W
equals let's see switch them over we
should get root 2 over 3 and it's
magnitude of W is root 2 over 3 we can
do our same trick from before square
root of 2 W 2 squared equals root 2 over
3 and this is just root 2 times W 2
equals root 2 over 3 so therefore W 2 is
1/3 and W 1 is negative 1/3 bingo we've
got W 1 we've got W - we know that B was
0 because obviously at 0 its y equals x
and we're done
ah that was fast the alphas might have
taken longer actually the alphas on this
one or more of a pain in the ass than
anywhere else because let's take a look
at this one if you can see it we've
added yet some new points we've got this
point up here this point down there so I
think pretty clearly these plus and
minus are the closest to each other but
what happens if we take the
perpendicular bisector between these two
and dude like this this plus is in the
middle so we're going to have 2 feet so
therefore this plus is going to have to
also be support vector so we can't just
draw this line we have to include this
what's our best division
vertical lines that's right vertical
lines just saw and that means that the
equation of our equation of our boundary
the dotted line here y-axis so the
equation of the boundary of the y-axis
then B equals zero and how W 2 equals
zero so the only thing that is not W 2
equals 0 so W 2 equals 0 B equals 0 but
W 1 is not equal to 0 because it's just
the equation of the y-axis so we
therefore know that the equation is just
W 1 times x equals 0
so if w1 times x equals zero and we know
that that just means essentially that X
is going to be some K it's also going to
be negative because of the fact that
that plus is our store on the left then
we're going to figure out what that K is
we'll use our old trick
by this point old hopefully one over
magnitude of W equals D this time D is
just 1 so therefore magnitude of W
equals one there's only one component in
W so therefore W 1 is minus 1 because
the plus is on the left people see that
not too bad
this one's easy to calculate a W but
it's not as easy to get all the alphas
but let's move on to a new and even more
fun maybe not
um question which is this guy as you can
see this is a lease this well maybe not
this is a one-dimensional vector these
vectors only have a single dimension so
it just looks like a number line here
that dimension varies from it looks like
negative 9 to positive 9 either just has
one component you don't have to worry
about any of these crazy magnitudes with
two components just everything as a
single component however it's obvious
that a linear basis line is going to
completely screw us up here since lines
at this point are just like glump all
these are pluses all these are Mises
well not great that doesn't get them off
so how are we going to do it well we're
going to use what is usually perhaps the
hardest thing in SVM's but in this case
is not going to be too bad for us we're
going to use a kernel now based on how
little I understood kernels the first
time I've took this class I'm guessing
that you guys would like to have some
explanation on these kernels you
probably saw them you remember the
kernels from Patrick's lecture vaguely
they're like this Phi and then there's
this K and then they get really
complicated okay so here's how the
kernel works the basic idea is this
and I'll write it over here
oh wow there's more stuff okay I'll
write it right here the basic idea is
this we're taking the normal space which
is you know this number line or it could
be any kind of normal space and we're
going to take a vector we're going to
put it into it of a function called Phi
and Phi of vector X brings X into some
new dimension Phi or fee if you like it
better is usually a nasty piece of work
and something that you'd never ever want
to look at sometimes it's not too bad
Phi is the function that brings it into
the new dimension okay and when you
brought the data into a new dimension
sometimes you can just cut a straight
line in that dimension and you'll just
be happy however something that was
noted by the very very smart inventor of
support vector machines is that you
don't actually need to to work with
function Phi
even if Phi is an absolutely horrible
monstrosity because of the fact that you
never need to know what the vector X
what all these vectors X actually are in
the new space at least not directly in
none of these equations up here do we
ever use X by itself however we do use X
being dot product with something else so
he figured out a very sneaky and
excellent shortcut okay so I shouldn't
use x1 x2 L use X and Z so if you have
two vectors X and Z which are in a
regular space you put them into this
function called the kernel then it will
tell you Phi of X dotted with 5z
and if you have that you don't need five
does everyone see that
does everyone see why we don't need five
look at all these equations up here we
never look at X by itself in these in
these vector equations at least now
calculating alphas yeah that gets a
little bit fuzzy but in the vector but
in the vector equations also you get you
may ask like why would you do this you
can't complete the alphas it turns out
we're um that actually other than for
these very simple linear problems human
minds cannot calculate the alphas in
fact you run a very complicated
quadratic optimization in fact finding
out the best alphas is the thing that
you will climb on when you're doing
SVM's
in the real world you say alright I'll
run my algorithm and I know there's only
one peak which is very very good because
it's quadratic optimization let me
figure out the office so in fact it
doesn't matter that you can't do that
use these alpha places to figure out the
alphas um if you've gone into if you
only know the kernel function out the
Phi function because normally the
computer figures out the alphas for you
with quadratic optimization just in
these simple problems we know you can
calculate the alphas so we have the
kernel which replaces that basically
gives us the dot product of the things
in the new space so being that as it may
I'll give you the kernel here I'd like
you to give me the Phi so a kernel for
that someone got an idea whose name was
Susan Q random student apparently she
got an idea that if we had a kernel for
X and Z actually they're not vectors I
guess they're just single components and
the kernel equals cosine PI over 4 x
times cosine pi over 4z plus sine PI
over 4 X plus
pi over 4z so that is the new dot
product cosine PI over 4x
oh wait sorry I put one of the disease
not inside the parenthesis that was
still with me so cosine of the quantity
PI over 4 x times cosine is upon T pi
over 4z plus sine is upon T PI over 4 X
plus sine of the quantity PI over 4 z is
the new dot product so that begs the
question in this one this is an easy one
so we can calculate the 5 what is Phi of
X we're actually taking it from one
dimension and we may be playing around
with it a lot to get this this thing has
become a new dot product it replaces dot
product now remember the dot part for
scalars would have just been multiplying
the two numbers together so it actually
makes it a little bit more complicated
does anyone think they know the Phi
ah we got one what do you think you need
to come back two dimensional absolutely
that's exactly correct how would you
solve this on the next whole quiz if
you're not our brave volunteer
well that K if you squint at it not very
much actually it's pretty much a dot
product between cosine PI over 4 s and
sine PI over 4 is I mean look at it dot
product is just remember if the dot
product of X and Z vectors is X 1 is X 1
Z 1 plus X 2 Z 2 so that basically is X
1 Z 1 plus X 2 Z 2 o this should have
been many times
yeah there's trillion times sorry
there's a plus there anyone who missed
it because of that my bad that should
have been a x there should've been times
up there the plot only plus is its
cosine PI over 4 X cosine 2 pi over 4z
plus sine PI over 4 x times sine PI over
4 Z so yeah it's basically the dot
product between cosine PI over 4 X and
sine PI
for bingo all right last thing and then
we well we're not done yet because we're
going to maybe ask some questions and
then we're going to see if we can
conquer those alphas but last thing
let's graph in this new dimension all
the points
so obviously cosines and sines so we're
going to get results between 1 and minus
1 let's see maybe I can graph it um can
I write on all these wait maybe this one
oh no people do weird stick figures
there ok
oh yeah this one's kind of messy but
we'll do it on this ok so we've got this
is 1 negative 1 1 half and 1 or so 1/2
negative 1/2 1 negative 1 negative 1/2
1/2
ok so given that let's try to graph all
these points on this number line into
this new brave new dimension by using
their cosine sine PI over 4 so all right
great so let's do the pluses first what
about the plus X 0 is cosine 0 sine 0 so
what is that that's 1 0 that's right
in fact the 8 and the minus 8 are also
that times 3 the eight of the minus 8
are also that because then it's just 2
pi n minus 2 pi which well cosine and
sine are periodic ok great
what about the one well that's cosine PI
over 4 sine PI over 4 what's that
yeah it's root 2 over 2 root 2 over 2 so
it's something like here we'll say and
in fact the 9 and the 9 and the negative
7 are also that so there's three of
these two okay what about the negative
one that's cosine negative PI over 4
sine PI over 4 that's right the x value
is positive root 2 over 2 and the y
value is negative so and again there's
three of them all right great now let's
do the minuses
there's the minus at 3 which is also the
same as I minus at negative 7 minus at 3
is cosine 3 PI over 4 sine 3 PI over 4
which one is that
so that's in a second ah yeah that's
going to be the second quadrant we're
going to have the cosine is going to be
negative but the sine is going to be
positive and so we get three points here
and as you may predicted the other one
the 5 PI over 4 is in the third quadrant
we get three points here why are the
support vectors
but I understand where you're getting
three of these of plus the person's
already lonely to keep ah there's two
negatives here and there's two natives
here good cop doesn't change the problem
in fact if we just grapple points there
might have been more but there's a very
subtle and important distinction there
are two negatives but otherwise yeah
these are grasped correctly does anyone
see where the support vectors are some
popular minus and plus will try to do
the perpendicular bisector let's see it
that works but guess what these guys are
on the same line so we'd better circle
them so it's actually the question is
what isn't a support vector only this
only those big question this is one
affection I mean you just proved that
actually just show that those land on
the same line so you really need like
the cosine term and a side
just the cosine alright classic is so
then we've done this in one dimension
all we do is do only the cosine so if we
did only the cosine then they would have
still been divided easily divisible the
answer is absolutely we could have
however the question did not because
Susan Q random student decided to do
cosine inside but yes if we had said you
654 student find a fine that will work
for this you could have found a Phi that
was just cosine that would be easier
however it's important to be able to
work with what somebody else gives you
in this case they gave you they give you
that transformation which yeah was a
wasteful with an extra dimension you
didn't need the sine because you didn't
need the y-axis really here you just
needed the X does everyone see this how
this works can maybe transform
dimensions the main hardest part is
they'll usually give you a K and ask for
a fire or give you a fire ask for a K
it's not too bad just remember if they
give you a Phi do a dot product with it
and if they give you a fire that's just
one component dot product of one
component just multiplying it together
easy enough so they give you a K treat
it as a dot product and try to
reverse-engineer it's usually something
like this that's easy to
reverse-engineer I really haven't seen
it where it's not so it may look it
often looks like the scariest problem
but it's usually not too bad to go
between five and KS does anyone have any
questions on anything that we did on
spectrum change
but what is some what's the intuition
behind the building like we solve it and
figure out numbers integrations what is
it but what is intuition what is w w is
the dividing line it's the drop-dead
dividing line when I say the drop-dead
dividing line you like those big bold
solid lines over there those are you're
pretty certain lines everything passed
that was a minus and you're doing your
training is that everything has to plus
the big bold line there was a plus in
your training separate the dotted line
is the one you're really going to use in
the test data in the test data when push
comes to shove you might get something
that's inside of that gutter and if it's
on the say of that one up there it's on
the up left side that dotted line you're
going to call it a plus so that dotted
line is your decision boundary and that
is that is basically the idea and in
fact the way that the algorithm would do
it on the computer is it would contract
the optimize the Alfa's which messes
around with the dotted line and by
quadratically i'll math-amazing the out
those alphas you see how the alphas add
up to a w it just checks it around and
eventually it finds Oh making the alpha
of this one zero makes that that makes
it a better optimization you're trying
to get the widest possible road it would
eventually come out to this this is
trivial for a human to eyeball but some
real problems with two hundred data
points that have to get one or two of
them wrong classified and so you may be
using a quadratic kernel or something
like that
you can't do that you just can't well
maybe you can in which case you should
be getting a MacArthur Fellowship or
something like that but the computer
can't and the basic idea is when it
comes down to it it figures out the
Alpha is to give it the bet the best W
for widest road in the W is your
decision about it good question any
other questions about our old friend SVM
I have a question for you um after
seeing this and let's pretend that they
only ask you to solve for WS B's you
know these kind of criminals and Pfizer
typical things how many people now think
that you can go through and work an SVM
problem alright we've got we've got a
few we
a happy few band of brothers maybe eight
people raised their hand there that's
good
how many people know what a support
vector is now that's really good because
that if that's all you learn from
today's minor sensation is still good
it really is I'm telling you I had to
take two classes on this and then ta it
before I really really understood it so
you guys are ahead of me all right take
care have a great weekend and we'll see
you for boosting and vampires next week