AI & ML Exam > AI & ML Videos > Machine Learning with Python > SVM Optimization - Practical Machine Learning Tutorial with Python p.27
SVM Optimization - Practical Machine Learning Tutorial with Python p.27
Text Transcript from Video
what is going on everybody and welcome
to another SVM or support vector machine
from scratch tutorial as well as part 27
of our machine learning tutorial series
where we left off we're building our
support vector machine class and we're
working on the fitment the training and
better known as the optimization for the
support vector machine we got all the
way to while not optimized and we're
passing at the moment and we're going to
go ahead and pick up here so the first
thing we're going to do is we're going
to iterate through B's so not only do we
have to check and step through the W's
right we also not so basically not only
to want the minimum magnitude of vector
W we also want to have the maximum be
most bias possible so what we're going
to say is for B in MPA range so MPA
range is very much like just pythons
typical range only it does a couple
things for us mostly though it allows us
to say how much of a step do we want to
take at a time is what we're why we are
you going to use it also is probably
more efficient than range anyway MPA
range and we want B to go from a range
of negative one times basically negative
one times self and let's do this
negative 1 times self dot max feature
value times B range multiple so that is
the first value in the range so that's
basically it'll be negative let's say 50
okay so that might be negative 50 then
what we're going to say and actually
I'll just hit enter here so that's the
first value the next value will be self
dot max feature value times B range
multiple so now it's negative 52 50 next
you can say in this range like normally
pythons range just wants to go by one so
if you said range 0 10 it would go you
know 0 1 2 3 4 5
910 and so whereas here what you can add
is how much of a step do you want to
take each time so we're going to say the
step will just be step by B multiple and
again the reason why we want to have
this be multiple now is we don't need to
take as small of steps with B as we do W
right so so we're just gonna multiply
that step by five because this is a very
costly step that we're going to take and
the reason why it's costly is we are not
going to give be the same optimization
treatment with big steps smaller steps
as we are doing with W you could add
that in if you want it but that would
just add more complexity to this and I'm
trying to keep it as simple as possible
yet not as absurd you know we don't want
the SVM to take you know five minutes to
classify these six samples that we have
so that was my main objective which is
not to take forever so anyway so we'll
stick with this but just know that you
could you can use the same stepping idea
with B we just aren't going to do it
here so anyway there's that next let's
see four be there and actually B let's
see this is actually not going to go
right under this is an actual for loop
so : at the end of that enter and then
you begin you may want to put that all
on one line to make it clear I just
didn't want it running off the screen so
now we're going to say is for trance
formation in transforms we're going to
transform W now so we're going to say W
underscore t4 transform W basically um
is W getting low excited W times
transformation' so we're let's see step
instead sighs so W transferred where's
our transformation there there okay so
here you have your beginning of
transformations that are happening here
and so then for each of these
transformations we're just applying that
to our original w-which we're acquiring
from here where the original W is just
the maximum value that we can get
basically latest optimum and then we're
multiplying that by ten so for example
did we define our sample data we sure
did so that the largest value here is
clearly eight so it's eight times ten
eighty so then W we're going to run
through a bunch of WS that could be
anywhere between negative eighty and
positive eighty so there's that next
what we're going to do now is basically
we're going to test it and first we're
going to say found option equals true
innocence until proven guilty basically
we live in a civilized society so what
we do here is for eye and self data and
here is the weakest link weakest link in
the SVM fundamentally we'll say s mo
attempts to fix this a bit but s mo can
only fix it so much right so if you have
15 terabytes of data and you're only
able to get it to the minimum sizes and
chunks of 92 gigabytes
okay you still got a load 92 gigabytes
in memory so anyway but this is the
weakest link not in my code necessarily
but in the SVM fundamentally is that we
now have to run this function or this
calculation on all of the data to make
sure it fits so for I in self data we're
gonna say for X I in self dot data I
because so if the data is a a dictionary
there we go
so I is the class we could even say for
Y and self that data but we'll just do
this for now for X I in self data I what
do we want to do well we'll say I'll
just say why I equals I just so it's
totally obvious and then we're gonna say
if not and remember what is the
constraint what is that constraint
function right that constraint function
is y sub i multiplied by X sub I dotted
with W whoops dotted with W plus B needs
to be what greater than or equal to one
that's the function so if not y sub I
times and P dot dot this just so we can
dot these things WT for that
transformation X I plus B greater than
equal to one if that is not the case
found option equals false so what we do
is for everything within the data set we
initially say it's true but if even one
of the samples does not fit this
definition the whole thing is thrown out
okay and you could even probably throw a
break here and then maybe another check
like here if not or something like or if
found option anyway you could
that's one spot to speed it up I'm going
to keep it the way I had it initially
but basically as soon as you find it
false you might as well just break that
like there's no reason to continue on
with checking any more samples from that
class or the other class anyway we'll
keep it for here for now so then what
we're going to say is for transformation
because we at this point we don't like
we'll test each transformation
separately so if found option so
basically if everything checked out what
do we want to do well we're gonna say
opt dict CNP MPL in algae or m-- so
again this is how you can do the
magnitude of a vector it's you you know
the a squared B squared Z squared all
that's fun stuff
that's the magnitude of the vector so
and then that is going to be equal to
and we already said we're going to do WT
and B I'm gonna I'm sorry I'm still
thinking about this break thing I'm
thinking
add a break here later because I really
think that'll that'll save a lot of
processing actually because probably a
lot of the variations the first one will
violate that that and so you wouldn't
actually have to test you wouldn't be
wasting your time pasa well you'd still
waste I want be but anyway I'm going to
have to think about that break so now
what we want to do is we're going to
come down and we want to be in line with
this four B's so once we've finished
running through every B option in every
transformation option what we want to
check next basically from here is if if
W zero so this is really either of the
W's because they're identical but we're
gonna say if W zero is less than zero
what do we want to do well we're going
to say optimized equals true that
doesn't necessarily mean we found the
best value but this is or may be better
put step optimized or something like
that we'll keep it as optimized though
and then what we're going to do is we're
going to print optimized optimized a
step and the reason why we can say less
than zero is because we're transforming
here so if B is five five we've tested 5
5 negative 5 5 negative 5 negative 5 5
and negative 5 so we don't need to go
any further past zero at this point
so we optimized a step else if it's not
less than zero then we need to take that
step so W equals W minus step and just
for clarity someone get something more
familiar with the math fundamentals of a
vector could couldn't can tell us but
currently W is equal to something like
let's say five five and then step is a
scalar value so step might be one right
step might equal one and when we say W
minus step with under these
circumstances the answer is four four
I'm not sure mathematically that's valid
I know it's multiplication that can work
but I don't think that works with - but
so I can someone can tell me for sure
but that's what's happening so maybe
officially step would be more like it
should be more like W
- step step or something I just know we
can get away with this so uh so yeah
someone can correct me but I don't think
that's actually supposed to be the case
normally in normal math like if you
wrote that on a math test or something
you'd probably get in trouble so anyway
once that's done the next thing that
we're going to do is we're going to
break out of the the entire basically
while loop um so at the point where
because that's what while loop and then
you still want to take your next step
but for this entire step in theory if
this is triggered we have hit our
optimization value so let's assume we
hit that so you know if you come down
you're in line with the else
one more delete in your in line where
you want to be we're going to say norms
not roms norms equals sorted and we're
going to say it's the sorted value of n
for n in opt dick so we're just simply
sorting a list of those magnitudes
optics the key is the magnitudes so
we're just sorting them and when you
sort them it's from lowest to highest so
which one do we want
well our opt choice is going to equal
norms and we want the zeroeth element
there lose my that first one so now we
have our opt choice and finally we can
set some some values so we can say self
dot W equals opt underscore choice zero
and then we can say self dot B equals
opt choice one so uh let's see norms
zero sorted yeah we thought something
was weird yeah so opt dict and then we
need to do norms zero so this up choice
is opt dict norms zero there we go okay
so the logic here just so I didn't lose
anybody these are just the norms or the
magnitudes or whatever we take sorted
and that's just a sorted list of all the
magnitudes then we're going to say the
optimal choice is going to be optics
where the magnitude is the zeroeth
element of norms so the smallest norm
and then that dictionary if you recall
its magnitude of W : W be so self W zero
if there it is self top B is the first
tooth element and we have our our stuff
so once we've done that the other thing
that we want to do is we're also going
to set the latest optimum optimum just
want to call it optimum equals opt
choice zero let's see op Jois 0 0 plus
step x 2 so the latest optimum we come
back up to here basically where we
initially set that value for to
basically be your step we're resetting
that so later on when we go to reference
that latest optimum like when we go
through the first step it's this value
when we go through the next step and in
fact we could even we probably should
not be hard coding that - we should
match it to whatever this is but
whatever anyway we go through the next
step and now we're in the you know 1
percentages for each step in that
smaller value though but it's at 1
percent of the original value so that's
why we're not like growing in size
anyway so we'll step me make that step
but we're modifying this each time we go
through so so if you wanted to get more
and more precise the only addition or
the only change you would need to make
is by adding yet another step and just
to recap to remember in the fundamental
tutorial maybe like I think it was the
very last one before we started coding
well how would you know that you ought
to take another step right what was the
logic that we used to take another step
well the the logic was that support
vectors let's go up here support vectors
we'll be equal to y sub I X X I dotted
with W plus B and we'll will equal 1
because it's y sub I x all that so you
will know that you have found a really
great value for W and B when in both
your positive and negative classes you
have a value that is close to 1 how
close to one can be up to you but you
can eventually just set something to be
you know I want to at least be less than
2 or something and or maybe you want
something that's like one point zero one
or better and until you hit that value
you just keep stepping and then maybe
you hit the hit a max step and each step
is an order of magnitude smaller
basically or divided divided by where
basically that just means each step size
is an order of magnitude smaller anyway
you can keep doing that if you wanted to
be totally dynamic but some problems
can't be optimized so at some point you
do want to like stop because some
problems you're just not going to get at
least with a linear kernel and then some
something really just can't be optimized
but anyway we'll talk more about kernels
or in another tutorial but so at this
point we've we have finished our
optimization algorithm and we'll
probably come back to explaining how
everything works there but that is done
and so now well all we have to add
pretty much at this point is we're gonna
adjust where's our prediction do we not
I thought we wrote the prediction yeah
it's done here so that darn white space
from send X anyway uh so yeah so we got
a prediction that's pretty much done we
are gonna add something there and then
we got to do the visualization and then
we'll actually run this and we can see
what our lovely work has done or we'll
see some errors and we'll fix them so
anyways any questions comments concerns
or whatever up to this point feel free
to leave them below otherwise as always
thanks for watching thanks for all the
support and subscriptions and until next
time
to another SVM or support vector machine
from scratch tutorial as well as part 27
of our machine learning tutorial series
where we left off we're building our
support vector machine class and we're
working on the fitment the training and
better known as the optimization for the
support vector machine we got all the
way to while not optimized and we're
passing at the moment and we're going to
go ahead and pick up here so the first
thing we're going to do is we're going
to iterate through B's so not only do we
have to check and step through the W's
right we also not so basically not only
to want the minimum magnitude of vector
W we also want to have the maximum be
most bias possible so what we're going
to say is for B in MPA range so MPA
range is very much like just pythons
typical range only it does a couple
things for us mostly though it allows us
to say how much of a step do we want to
take at a time is what we're why we are
you going to use it also is probably
more efficient than range anyway MPA
range and we want B to go from a range
of negative one times basically negative
one times self and let's do this
negative 1 times self dot max feature
value times B range multiple so that is
the first value in the range so that's
basically it'll be negative let's say 50
okay so that might be negative 50 then
what we're going to say and actually
I'll just hit enter here so that's the
first value the next value will be self
dot max feature value times B range
multiple so now it's negative 52 50 next
you can say in this range like normally
pythons range just wants to go by one so
if you said range 0 10 it would go you
know 0 1 2 3 4 5
910 and so whereas here what you can add
is how much of a step do you want to
take each time so we're going to say the
step will just be step by B multiple and
again the reason why we want to have
this be multiple now is we don't need to
take as small of steps with B as we do W
right so so we're just gonna multiply
that step by five because this is a very
costly step that we're going to take and
the reason why it's costly is we are not
going to give be the same optimization
treatment with big steps smaller steps
as we are doing with W you could add
that in if you want it but that would
just add more complexity to this and I'm
trying to keep it as simple as possible
yet not as absurd you know we don't want
the SVM to take you know five minutes to
classify these six samples that we have
so that was my main objective which is
not to take forever so anyway so we'll
stick with this but just know that you
could you can use the same stepping idea
with B we just aren't going to do it
here so anyway there's that next let's
see four be there and actually B let's
see this is actually not going to go
right under this is an actual for loop
so : at the end of that enter and then
you begin you may want to put that all
on one line to make it clear I just
didn't want it running off the screen so
now we're going to say is for trance
formation in transforms we're going to
transform W now so we're going to say W
underscore t4 transform W basically um
is W getting low excited W times
transformation' so we're let's see step
instead sighs so W transferred where's
our transformation there there okay so
here you have your beginning of
transformations that are happening here
and so then for each of these
transformations we're just applying that
to our original w-which we're acquiring
from here where the original W is just
the maximum value that we can get
basically latest optimum and then we're
multiplying that by ten so for example
did we define our sample data we sure
did so that the largest value here is
clearly eight so it's eight times ten
eighty so then W we're going to run
through a bunch of WS that could be
anywhere between negative eighty and
positive eighty so there's that next
what we're going to do now is basically
we're going to test it and first we're
going to say found option equals true
innocence until proven guilty basically
we live in a civilized society so what
we do here is for eye and self data and
here is the weakest link weakest link in
the SVM fundamentally we'll say s mo
attempts to fix this a bit but s mo can
only fix it so much right so if you have
15 terabytes of data and you're only
able to get it to the minimum sizes and
chunks of 92 gigabytes
okay you still got a load 92 gigabytes
in memory so anyway but this is the
weakest link not in my code necessarily
but in the SVM fundamentally is that we
now have to run this function or this
calculation on all of the data to make
sure it fits so for I in self data we're
gonna say for X I in self dot data I
because so if the data is a a dictionary
there we go
so I is the class we could even say for
Y and self that data but we'll just do
this for now for X I in self data I what
do we want to do well we'll say I'll
just say why I equals I just so it's
totally obvious and then we're gonna say
if not and remember what is the
constraint what is that constraint
function right that constraint function
is y sub i multiplied by X sub I dotted
with W whoops dotted with W plus B needs
to be what greater than or equal to one
that's the function so if not y sub I
times and P dot dot this just so we can
dot these things WT for that
transformation X I plus B greater than
equal to one if that is not the case
found option equals false so what we do
is for everything within the data set we
initially say it's true but if even one
of the samples does not fit this
definition the whole thing is thrown out
okay and you could even probably throw a
break here and then maybe another check
like here if not or something like or if
found option anyway you could
that's one spot to speed it up I'm going
to keep it the way I had it initially
but basically as soon as you find it
false you might as well just break that
like there's no reason to continue on
with checking any more samples from that
class or the other class anyway we'll
keep it for here for now so then what
we're going to say is for transformation
because we at this point we don't like
we'll test each transformation
separately so if found option so
basically if everything checked out what
do we want to do well we're gonna say
opt dict CNP MPL in algae or m-- so
again this is how you can do the
magnitude of a vector it's you you know
the a squared B squared Z squared all
that's fun stuff
that's the magnitude of the vector so
and then that is going to be equal to
and we already said we're going to do WT
and B I'm gonna I'm sorry I'm still
thinking about this break thing I'm
thinking
add a break here later because I really
think that'll that'll save a lot of
processing actually because probably a
lot of the variations the first one will
violate that that and so you wouldn't
actually have to test you wouldn't be
wasting your time pasa well you'd still
waste I want be but anyway I'm going to
have to think about that break so now
what we want to do is we're going to
come down and we want to be in line with
this four B's so once we've finished
running through every B option in every
transformation option what we want to
check next basically from here is if if
W zero so this is really either of the
W's because they're identical but we're
gonna say if W zero is less than zero
what do we want to do well we're going
to say optimized equals true that
doesn't necessarily mean we found the
best value but this is or may be better
put step optimized or something like
that we'll keep it as optimized though
and then what we're going to do is we're
going to print optimized optimized a
step and the reason why we can say less
than zero is because we're transforming
here so if B is five five we've tested 5
5 negative 5 5 negative 5 negative 5 5
and negative 5 so we don't need to go
any further past zero at this point
so we optimized a step else if it's not
less than zero then we need to take that
step so W equals W minus step and just
for clarity someone get something more
familiar with the math fundamentals of a
vector could couldn't can tell us but
currently W is equal to something like
let's say five five and then step is a
scalar value so step might be one right
step might equal one and when we say W
minus step with under these
circumstances the answer is four four
I'm not sure mathematically that's valid
I know it's multiplication that can work
but I don't think that works with - but
so I can someone can tell me for sure
but that's what's happening so maybe
officially step would be more like it
should be more like W
- step step or something I just know we
can get away with this so uh so yeah
someone can correct me but I don't think
that's actually supposed to be the case
normally in normal math like if you
wrote that on a math test or something
you'd probably get in trouble so anyway
once that's done the next thing that
we're going to do is we're going to
break out of the the entire basically
while loop um so at the point where
because that's what while loop and then
you still want to take your next step
but for this entire step in theory if
this is triggered we have hit our
optimization value so let's assume we
hit that so you know if you come down
you're in line with the else
one more delete in your in line where
you want to be we're going to say norms
not roms norms equals sorted and we're
going to say it's the sorted value of n
for n in opt dick so we're just simply
sorting a list of those magnitudes
optics the key is the magnitudes so
we're just sorting them and when you
sort them it's from lowest to highest so
which one do we want
well our opt choice is going to equal
norms and we want the zeroeth element
there lose my that first one so now we
have our opt choice and finally we can
set some some values so we can say self
dot W equals opt underscore choice zero
and then we can say self dot B equals
opt choice one so uh let's see norms
zero sorted yeah we thought something
was weird yeah so opt dict and then we
need to do norms zero so this up choice
is opt dict norms zero there we go okay
so the logic here just so I didn't lose
anybody these are just the norms or the
magnitudes or whatever we take sorted
and that's just a sorted list of all the
magnitudes then we're going to say the
optimal choice is going to be optics
where the magnitude is the zeroeth
element of norms so the smallest norm
and then that dictionary if you recall
its magnitude of W : W be so self W zero
if there it is self top B is the first
tooth element and we have our our stuff
so once we've done that the other thing
that we want to do is we're also going
to set the latest optimum optimum just
want to call it optimum equals opt
choice zero let's see op Jois 0 0 plus
step x 2 so the latest optimum we come
back up to here basically where we
initially set that value for to
basically be your step we're resetting
that so later on when we go to reference
that latest optimum like when we go
through the first step it's this value
when we go through the next step and in
fact we could even we probably should
not be hard coding that - we should
match it to whatever this is but
whatever anyway we go through the next
step and now we're in the you know 1
percentages for each step in that
smaller value though but it's at 1
percent of the original value so that's
why we're not like growing in size
anyway so we'll step me make that step
but we're modifying this each time we go
through so so if you wanted to get more
and more precise the only addition or
the only change you would need to make
is by adding yet another step and just
to recap to remember in the fundamental
tutorial maybe like I think it was the
very last one before we started coding
well how would you know that you ought
to take another step right what was the
logic that we used to take another step
well the the logic was that support
vectors let's go up here support vectors
we'll be equal to y sub I X X I dotted
with W plus B and we'll will equal 1
because it's y sub I x all that so you
will know that you have found a really
great value for W and B when in both
your positive and negative classes you
have a value that is close to 1 how
close to one can be up to you but you
can eventually just set something to be
you know I want to at least be less than
2 or something and or maybe you want
something that's like one point zero one
or better and until you hit that value
you just keep stepping and then maybe
you hit the hit a max step and each step
is an order of magnitude smaller
basically or divided divided by where
basically that just means each step size
is an order of magnitude smaller anyway
you can keep doing that if you wanted to
be totally dynamic but some problems
can't be optimized so at some point you
do want to like stop because some
problems you're just not going to get at
least with a linear kernel and then some
something really just can't be optimized
but anyway we'll talk more about kernels
or in another tutorial but so at this
point we've we have finished our
optimization algorithm and we'll
probably come back to explaining how
everything works there but that is done
and so now well all we have to add
pretty much at this point is we're gonna
adjust where's our prediction do we not
I thought we wrote the prediction yeah
it's done here so that darn white space
from send X anyway uh so yeah so we got
a prediction that's pretty much done we
are gonna add something there and then
we got to do the visualization and then
we'll actually run this and we can see
what our lovely work has done or we'll
see some errors and we'll fix them so
anyways any questions comments concerns
or whatever up to this point feel free
to leave them below otherwise as always
thanks for watching thanks for all the
support and subscriptions and until next
time
About this Video
4.85/5 Rating
Apr 18, 2026 Last updated
Related Exams
Video Description: SVM Optimization - Practical Machine Learning Tutorial with Python p.27 for AI & ML 2026 is part of Machine Learning with Python preparation. The notes and questions for SVM Optimization - Practical Machine Learning Tutorial with Python p.27 have been prepared according to the AI & ML exam syllabus. Information about SVM Optimization - Practical Machine Learning Tutorial with Python p.27 covers all important topics for AI & ML 2026 Exam. Find important definitions, questions, notes, meanings, examples, exercises and tests below for SVM Optimization - Practical Machine Learning Tutorial with Python p.27.
Introduction of SVM Optimization - Practical Machine Learning Tutorial with Python p.27 in English is available as part of our Machine Learning with Python for AI & ML & SVM Optimization - Practical Machine Learning Tutorial with Python p.27 in Hindi for Machine Learning with Python course. Download more important topics related with notes, lectures and mock test series for AI & ML Exam by signing up for free.
Description
Video: SVM Optimization of Machine Learning with Python have been curated by the AI & ML experts, helping you revise the topic quickly for exam preparation. Watch on EduRev.
Information about SVM Optimization - Practical Machine Learning Tutorial with Python p.27
Here you can find the meaning of SVM Optimization - Practical Machine Learning Tutorial with Python p.27 defined & explained in the simplest way possible. Besides explaining types of SVM Optimization - Practical Machine Learning Tutorial with Python p.27 theory, EduRev gives you an ample number of questions to practice SVM Optimization - Practical Machine Learning Tutorial with Python p.27 tests, examples and also practice AI & ML tests.
Related Searches
Extra Questions, Exam, ppt, Viva Questions, study material, Previous Year Questions with Solutions, Important questions, past year papers, Objective type Questions, SVM Optimization - Practical Machine Learning Tutorial with Python p.27, Free, shortcuts and tricks, video lectures, Sample Paper, Summary, practice quizzes, mock tests for examination, Semester Notes, SVM Optimization - Practical Machine Learning Tutorial with Python p.27, SVM Optimization - Practical Machine Learning Tutorial with Python p.27, MCQs, pdf ;