Mega-R6. Boosting Video Lecture | Artificial Intelligence: A Fundamental Guide - AI & ML

the following content is provided under
a Creative Commons license your support
will help MIT OpenCourseWare continue to
offer high quality educational resources
for free
to make a donation or view additional
materials from hundreds of MIT courses
visit MIT opencourseware at ocw.mit.edu
good morning everyone today we're going
to talk about boosting boosting is
pretty awesome and it's not as easy as
the it's not as hard as it might seem
it's actually pretty easy as long as you
do it right so let's take a look at this
boosting problem there's really mainly
just like with ID trees you may have
noticed there's like the ID tree problem
where there's a graph and all of the
tests are X and y-axis tests on that
graph and then there's the ID tree
problem where there are a lot of crazy
different classifiers about sort of
characteristics that are discrete and
then all of the sort of ID tree stumps
if you will are built out of those
street qualities this is an example of a
boosting problem of the of the second
type with a bunch of discrete qualities
like evil emo transforms sparkly and it
has a numerical quality number of
romantic interests so it's one of
basically the two kinds of boosting
problems you might see another is a is
the two axis Cartesian problem a good
example of that is the hours of sleep
versus coffee problem which I for one
I'm planning on doing in tutorial on
Monday so that you guys sort of get a
sense of both types of problem this one
looks big I think it looks I mean it has
10 different vampires or non vampires to
classify it has a whole bunch of
possible classifiers but if you do it
right you can do it fast so here's the
prompt for the problem let's see after
graduating MIT you get a job working for
Van Helsing and Summers a famous vampire
hunting consulting agency gabriel van
Helsing one of the two founders once
attended several 6.34 lectures as a
guest and he remembers professor
Winston's vampire identification tree
lecture he assigns you the task of
creating a superior classifier for
vampires by using boosting on the
following data so we've got the ID
number which we can use
just write things out in shorthand we've
got the name of several vampires and non
vampires then you see whether they're a
vampire or not that's whether their
classification is like a plus or a minus
for the quality vampire after that
there's a bunch of possible ways to
classify whether they're a vampire or
not there's whether or not they're evil
whether or not they're email whether or
not they transform and whether or not
they're sparkly as well as the number of
romantic interests that they have so for
instance on the one hand you have
Dracula who's evil but he's not emo he
can transform into a bat or a cloud of
mist he'd nut he does not sparkle and he
has five romantic interests those three
vampire chicks at the beginning
Wilhelmina Murray and Lucy Westenra
so on the other hand you have like
squall Leonhart who's the protagonist of
Final Fantasy 8 is extremely emo and
doesn't have any of the other
characteristics and he's not a vampire
however he's a nice counter example for
a possible role that all emo people are
vampires because he's very very emo and
he's not a vampire so how will we go
about tackling this problem with
boosting well there's a whole bunch of
different classifiers and if you think
this is all of them like evil emo
transform sparkly romantic interests and
true it's actually only half of them the
other half are the opposite versions but
we'll ignore them for now so if you look
at these you can probably figure out
what they mean evil equals yes means
vampire is what we're saying here except
maybe true you might be saying why is
there one that just says true the one
that just says true says that everybody
is a vampire you might think oh that
sucks but it's not that bad since seven
of the 10 symbols or vampires the key
crucial thing about boosting is that for
any possible classifier like classifying
on the evil dimension which actually
sounds like some kind
weird place that you go in a comic book
but classifying on the emo dimension or
whatever as long as it's not a 50/50
split of the data you're guaranteed to
get sub to be able to use it in some way
for boosting if it is a 50/50 split it's
like flipping a coin so it's useless
because if you say okay if you had some
other thing like gender equals male or
female and let's say that was 50/50 it's
not but let's say it was 50/50 between
vampire and non vampire it's a useless
classifier because it would be just the
same as flipping a coin you get no
information now you might say wait a
minute what about classifiers that get
worse than 50/50 what about them might
not they be even worse than a 50-50
classifier I would claim a classifier
that gets less than 50/50 is still
better than a classifier that gets
exactly a 50/50 split is there a
question
let's reset
so the finance in the ID tree example
you might use fifty-fifty classifiers
and later rounds if for instance there's
a fifty-fifty classifier except for that
most of the things off of that side have
been already removed let's say there's
20 data points and there's a classifier
that splits it ten and ten splits it ten
ten and it gets half plus half minus on
both sides but all the pluses from the
right side have been removed by some
other classifier you might use it that's
true but in boosting you'll never use
something that's a fifty-fifty
classifier something's you never use
something that has exactly a 50/50
chance of being correct because if it
has forgiven your chance of being
correct it's useless and if it has a
50/50 if it has a 50-50 chance of now
sorry let me let me specify again you'll
never use something that has a 50/50
there's a 50/50 chance of giving you the
right answer given the weights that's
very very important and I that may be
what you're on what your question was
getting at as I'm about to show you and
as Patrick told you in the lecture in
later rounds of boosting you change the
weights of each of the ten data points
at first you start with all the weights
being one tenth the weights have to add
up to one in this case you never ever
choose a classifier that gets five of
them right and five of them wrong in
later rounds you'll never ever choose a
classifier that gets half of the weight
wrong exactly half of the weight wrong
but half of the weight may not be half
of the data points so it's possible to
choose a classifier that that gets half
of the data points wrong if it doesn't
get half of the weight wrong and that's
similar to an ID trees when you've
already gotten things right before
because you'll see that the weight is
going to go to the ones you got wrong so
I'm not saying that you should throw out
right away anything that gets five of
the punch wrong hell you shouldn't even
thought right away something to get
seven of the points
it's possible possible that you can get
seven of the points wrong while getting
less than half of the weight wrong if
those other three points are really
really annoying to get right and we'll
see that later on but for insight at
every step along the way we're willing
to choose we're willing to choose any
classifier that doesn't get fifty fifty
however we want to choose the classifier
that gets the most of the weight right
by most of the weight at first we mean
most of the points right later we will
mean but exactly what I said most of the
weight and if you don't understand that
it's sometimes hard to get it right away
when Patrick just lectures through it
introduces a new concept if you don't
understand that you'll see what I mean
when we go through all right so my point
I was making before is what about things
that get less than half of the weight
right well those are always okay because
you can just flip them around use their
inverse and that gets hat more than half
of the weight right it's sort of like
yeah it's sort of like my girlfriend
always tells me that she is more than
50% likely to choose the wrong direction
when you're trying to go between two
places which I'm kind of skeptical of I
said if that's really true then we can
just go wherever you didn't say it to go
and it will be more likely to go the
right way so you're actually really good
at finding the place that we want to go
and then she's like no that won't work
because then I'll know that you're going
to do that and I'll say the well double
say the wrong way and then you'll go the
wrong way again but that's not
withstanding you can see you can see you
can apply the same concept to boosting
and that's why underneath of this I have
all the opposite versions of all these
tests so what should we be doing to to
solve this problem more quickly first
let's figure out which data points are
misclassified by each of these
classifiers in other words if we say all
the evil things are vampires and all the
non evil things are not vampires what do
we get wrong and we're if we do that for
every classifier then that only get
faster later on because later on we're
going to go through classifiers
and we're going to have to add up the
ones they got wrong so this chart over
here is going to be extremely useful and
on the test that this appeared in they
even made you fill in it to help
yourself out so let's see if we said
that all the evil equals yes are
vampires and all the evil equals is no
are not vampires then I'll do the first
one for you
so we get all the non vampires correct
because they are all evil equals no but
unfortunately we get uh we get angel
Edward Cullen sigh Otonashi and Lestat
the Liancourt wrong because they are
vampires and they're evil equals no
apparently Lestat is iffy but I never I
never read those books and the Wikipedia
article said that in the end he wasn't
that evil so there we go evil equals yes
misclassifies two three four and five
alright so let's try emo equals yes I'll
have someone else do it so see if you
guys got it so if we say that all the
emo people are vampires all the non evil
people are not vampires what do we get
wrong one six seven nine that's exactly
right and fast good we get one six seven
and nine wrong one six and seven are
wrong because they are not emo but
they're vampires nine is wrong because
squalls emo and he's not a vampire good
okay what if we said that exactly the
transforming characters are vampires and
the ones that do not transform are not
vampires which ones we get wrong
is the next one over so long transforms
we've down getting off this she's not
this is on the essentially on there yeah
yeah yes things out oh yeah oh man you
didn't see the chart you were just like
me you saw the black Jesus in him which
of these are the transforming characters
okay that's pretty hardcore
but yeah three four five and eight no no
it's definitely given to you that would
be like the worst test ever for for
international students if you don't know
these ten characters that mark found as
vampires you lose alright so what about
that sparkly equals yes is a vampire and
if it's not sparkly it's not a vampire
this is not this is guaranteed not to go
well what do you think it's gonna get
wrong yes mark Lincoln's guests are the
only good yo hmm yep Andy yes that's
right it gets one two four five six
seven eight wow that's pretty awful but
damn it it gets ever gone right and he's
hard to get correct due to the fact that
he's not really much like a vampire he's
more of a superhero who says he's a
vampire okay so next number of romantic
interests greater than two
so if they have more than two romantic
interests their vampire and otherwise
they're not a vampire so which ones
would does get wrong would that get
wrong
just green that's right because Searcy
had Odysseus she had Telemachus actually
she had that guy she turned into a
woodpecker
she had that other guy who was AC God
who caused her to turn Scylla into the
nine-headed thing and probably leaves
one other person so seriously it gets
wrong and it also gets Edward Cullen
wrong because he only has one so three
in ten you can tell I thought about this
problem when I was writing it up I wrote
this one all right number of romantic
interests greater than four so it's a
little bit different this time now you
have to have at least four romantic
interests or actually greater than four
but there are none there exactly four to
be classified as a vampire which ones do
you think it's going to get wrong yeah
it is going to get three four and ten
wrong because now you run into the fact
that SIA has that blonde Chevallier guy
Haji and Kyle so the last of the
positive ones because I who I claim I
can derive all the negative ones of you
guys give me the positive ones lots of
positive ones
everybody's a vampire who does that get
robbed yes okay now I can derive all the
negative ones from this without a sweat
evil equals no well it's one six seven
eight nine ten without looking at the
chart raise your hand if you don't see
why raise your hand if you see why it's
one six seven eight nine ten without
looking at the chart raise your hand if
you don't nobody okay wait one hand okay
I saw another one back there too later
they were just more tentative okay it's
the compliment of a because a is equal
equals yes as a vampire it gets two
three four and five wrong so therefore
evil equals no as the vampires
guaranteed to get all the opposite ones
yeah we're looking here but we're not
looking at the big chart there oh yeah
if you can't look at anything then
you're screwed unless you've also not
only are used hardcore as this guy but
you've also memorized the numbers all
right so emo equals no is going to be 2
3 4 5 8 10 transforms equals no is 1 2 6
7 9 10 Sparkle equals no is 3 9 10
when take interest less than 2 is
everything except 3 & 10 1 2 4 5 6 7 8 9
1 2 5 6 7 8 9 and then finally
everything but 8 9 and 10 so 1 2 3 4 5 6
7 all right so we've started off we know
what everything gets wrong I then make a
bold claim because there is there an of
these which is 14 I make the claim that
there are only 6 that in your wildest
dreams you would ever possibly even
consider using ever and the rest you
would never ever use question yes
ah two or four but it should have been
negated with a less than or equal I'm
popping off of the quiz but yes I
noticed that this morning when I was
putting myself through my paste I'm like
there's not a less than or equal to wait
a minute and then oh wait it doesn't
have any to zur force actually I don't
remember writing them all as fives and
threes it's possible that somebody else
in the post editing process changed the
ball to be about the same number and
then change the less than or equal to be
less confusing it's possible I had
searcy it for and there was an equal to
somewhere and they're like forget it
as I can't think of the fifth romantic
interest for her so yes normally you
would have to negate it with a equal to
sign but there happen to not be any
things that are equal to four or two
here so they get away with it this time
but good practice it's good practice so
I'm claiming that our wildest dreams
we'd only ever want to use six of these
ever in the other eight forget it so
let's see I will call on people at
random to the first people obviously
getting it really easy to tell me which
of these you think that you might ever
want to use give me one you might ever
want to use ye be the best one yes
that's one that you might ever want to
use I'll circle the ones that you might
ever want to use e it only gets thing 10
wrong that's amazing it's like the best
classifier of all okay so give me
another one that you might ever want to
use ah let's see
F F is great it only gets three wrong do
people agree that you would ever want to
use F everyone's saying no why not it's
slightly something worse if you
guaranteed it every stat no matter what
the weights are to have a worse accuracy
than E it is definitely good if he
wasn't around it would be one of our
best classifiers of all but actually f
is not one of the six this is why I
wrote had them write on the test that
there are six because people might not
have found
six because people who did figure out
not to include F might not and figure
out to include some of the ones you want
to include
so why you can't use that okay so we
start off with one tenth wave for all of
our data points but let's say during our
time of boosting that all ten of the
data points have now different weights
so we'll call whatever the weight of
three is which you're going to get wrong
you want to minimize the error right so
that weight of three which goes into the
error of E is X the weight of ten can be
y so if you get if you're thinking of
choosing three you know your HIV feeding
loops using e your error is X plus y if
your thing of choosing F your error is X
plus y plus Z with Z is the error of
four and since you never gonna have a
negative wage X plus y plus Z is always
greater than X plus y that's a yes you
would you would probably shoot something
without the three into ten didn't get
the clear in the ten wrong but you would
certainly never choose f ever because
it's always worse than e in fact this is
exactly the process that will allow you
to find the correct six and I will I
mean ten and I can't I mean let's see if
you guys get it give me another one of
the six thing you might q okay is the
claim sparkly okay okay I'm going to say
we'll lose for the same reason as f it's
three nine and ten
it's essentially similar two three four
and ten so oh by the way we should not
be only going for the ones with the
fewest row with the fewest incorrect
where you need to be going for ones that
do not have something that is strictly
better in this case three and ten wrong
is strictly better than getting three
nine in ten wrong question you're going
to say transform you're going to be
correct transforms is one of the ones we
need see three four five and eight
there's nothing down here
it gets fewer than those wrong like
there's nothing that gets us three four
five wrong for instance there's no yeah
there's no way to get three four wrong
with that without getting either ten
wrong
or five and eight wrong what YG y naught
y naught t let's include G - we need six
no I just said give me any and someone
gave me the easiest one eat question why
don't be B looks great I love B let's
include B does someone else want to give
another one that they want to include a
why not a sure I mean it's hard to see
down here cuz there might be something
better on the bottom but yeah there's
not
so let's include a why not a I love a is
great okay so that is now that is now
five there's one more that we need is by
far the hardest one to find find me one
more that there's nothing better than it
there's nothing that has a strict subset
of the same one's wrong what
okay why would you shoot a minbari 7c
let's say H was a real problem for you
and you were just getting let's say that
three four and five they weren't that
bad they weren't that bad they weren't
that bad okay you got them wrong here
with transforms you chose C but sometime
later eight was just very far your issue
all right three four and five and eight
three four five were much smaller
weights than eight and then after you
got them you got three four five and
eight wrong three four five were still
not that bad and eight you still was a
high number and then sometime later down
the line you're looking at things and
you're saying you know what I really
don't want to get eight wrong again but
I don't mind if I get three four and
five wrong maybe I'll get it wrong with
two which I've never gotten wrong yet
actually none of the ones we've circled
here get two wrong so it's probably not
that bad to get two wrong so that's why
because it also it doesn't get eight
wrong if a was two three four five eight
you would never take it you see what I
mean oh did someone raise the head did
someone find it
the nine is icon dude elusive but he's
strictly worse because but usually shoot
me because three times get it only three
times better than getting three nine ten
one and universe UK you said I mean it
might not be that much worse it might be
only a little bit worse to choose K but
it's always worse so cushion okay okay
you planted by d DS you just want to
happen long but you don't want to get
screwed right you know what against and
unlike our other things you get three
and ten right which is which is B also
gets nine right D is the last classifier
you got it it is hard to choose one that
has seven of them wrong but D is the
last one you might pick those and it
turns out there's nothing better than
this we're classifying correctly those
annoying data points of Edward Cullen
and squall and also seriously who's not
that annoying but she tends to be a
problem when romance is concerned so
these are the six that we might use we
can now ignore the rest of them forever
or at least until someone reuses this
problem or something like that but we
can ignore everything except ABCDE G in
fact why do I even bring that up all the
ones we want around the front I'll bring
it back down
then I'll cross this off with reckless
abandon that even broke up a piece of my
chalk now these are the ones we're
actually thinking about using there is a
chart over here already prepared to do
some boosting with these six classifiers
alright so let's give it a try now
remember in boosting we always
try to choose whichever classifiers the
loose error is their question yes okay
you want to take every classify that you
generally want to take classifiers so
I'll tell you what lets you cross off a
classifier that may be a better way to
do it
you can cross off a classifier as
useless if if and by the way this is
only useful if you can do it faster than
just wasting your time looking at all of
them because if you can't cost cross off
some of them as useless usually the test
they won't make you you can just waste
your time and have 14 instead of six
possibilities every step of the boosting
but take a look at this one one two
three four five six seven then see do
you have anything here that has a strict
subset of these right wrong all up two
three four five is a strict subset this
can be crossed off one two five six
seven eight nine anything that's a
strict subset yes one six seven nine so
it can be crossed up one two four five
six seven eight nine let's see one six
seven nine is a strict subset three nine
ten three ten is a strict subset one six
seven nine a strict subset three four
five eight is a strict subset as these
two three four five one six seven nine
is a strict subset and up here three
four ten three ten is a strict subset
but the others don't have one even one
two four five six seven eight so you
look in general you want to keep them
you want to keep every classifier you
might use the only ones you'll never use
are ones that there's something else is
just better always by having a strict
subset of them wrong
hopefully that was more clear it's it's
tricky some out very few people realized
but we're brave enough to take sparkley
even when it got seven things wrong so
let's start out some boosting this
wasn't boosting this was setting
yourself up that it was setting yourself
up with the knowledge of how boosting
works less knowledge less search now we
only have to search six things I mean
more knowledge
search not nest on that search so we
start off with always being equal and
since there's ten data points all ten of
the data points are weighted one-tenth
okay we're now waiting all of them
equally
since we're writing all of them equally
when we want to find the classifier that
gets the least error we just want to
find the one that gets the fewest points
wrong which one is that
that's our friend E the first one that
people realize was a good one so we're
going to choose classifier E what's our
error it's just the sum of the ones we
get wrong so what's our error this time
it's one-fifth
we got points three and ten wrong they
both have a weight of 1/10 1/10 plus
1/10 is one-fifth so I'll put one-fifth
and alpha alpha is sort of a vote that
will be used at the very end to sort of
aggregate our classifier alpha is
one-half natural log of 1 minus the
error over the air however I have a
little trick for you it's not that
impressive of a trick but it's a little
fun so since error is one huh
oh sorry not air alpha alpha is one-half
natural log of 1 minus error over error
if error is in the form if error is 1
over X then alpha is 1/2 natural log of
X minus 1 that just follows from the
math it's a little shortcut if errors in
the form of 1 over X then it's just 1/2
natural log of X minus 1 which means
since this is in the form of 1/5
however on alpha is 1/2 Ln 4
okay one-half felon for so now we come
to the part in boosting that many people
consider to do the hardest part now I'm
going to show you how to do it more
easily this is that part where we try to
make all loans we got right
we've changed their weights to be in one
half and all those we got wrong we
change their weights to be one half here
is my automated process it's called the
numerator stays the same method here's
how it works here's our 10 data points
their current weight is one tenth all of
them we're about to rewrite them for the
next step so you agree they're all one
tenth they're equal to start off so step
one erase the denominators screw
fractions I don't like them there's
division multiplication it's a pain I
just want to add whole numbers that's
what we're going to do so which ones do
we get wrong
three and tenth circle those all right
add the numbers in the circles and
multiply by two why does that give you
four that's the new denominator you
always multiply by two add the numbers
not in the circles multiply by two but
does that give you 16 that's the new
denominator a final crucial step so that
we can do this again next round it is by
far the most mathematically complicated
thing here because we have to actually
do something with fractions but it's not
too bad is we then change everything to
be with the same denominator so the 1/4
become four sixteenths
all right
I can also unser pull these for next ah
I hit the that button alright alright
new weights 1/16 1/16 four sixteenth
1/16 1/16 1/16 1/16 1/16 four sixteenth
note the weights add up to one the ones
you got wrong add up to one half the
ones you got right add up to one half
you're happy so now that you get four
sixteenths of an error for getting three
wrong four sixteenths of an error for
getting ten wrong take a look at these
six I'm not going to call on someone
just whoever's good it now whoever's
good at math and can add these up more
quickly just three and ten count is for
all the others times one add them up
tell me which one's the lace what did
you say you go B it doesn't get three
wrong that sounds pretty good to me
does everyone else like B as well I like
it
I mean of our ones that don't get three
wrong or ten wrong we're only looking at
B and D and D just has D as seven B has
four so B is the best B gets four
sixteenths wrong does everyone see that
because even getting one of three or ten
wrong is as bad as all the ones that be
gets wrong because of the new weights so
cool let's choose B that's right and I
sort of gave it away what's the error
that B has it has four of the four wrong
each of which are worth 1/16 the error
is what four sixteenths or 1/4 whichever
is your favorite which means that the
Alpha is 1/2 Ln 3
bingo final round okay what do we get
wrong we got one six seven and nine
wrong oh yeah we can erase the
denominators all right what are the
numbers in the circles summed up
multiplied by 2 that's 8 1/8 and what
about the numbers not in the circle
summed up multiplied by 2 that's right
24 which means I'm going to have to
change all the numbers in the circle to
324 except I guess I don't because this
is the last round but if I was going to
do another round let's prepare in case
we were changed all of these to 324
besides it makes it easier to calculate
which one is the best classifier because
you can just use the numerators and sort
of add them up so while I'm writing that
up you guys figure out which one you
like for classifier and call it out to
me when I'm done
324th 124 for 24 124 124 324th 324th
124
324 wait I'm off by 1 here 3 1 4 ah ah
you're right
w1 is not assigned to any to anything so
w w2 is really w1 um yeah
yes that's true okay
well you get it H so what is the best
age you get it because it's right here
see the process is so foolproof even a
fool like me can get it right while they
have the chart wrong alright so what's
the best classifier you say see I say
that seems pretty reasonable only gets
three four five and eight wrong does
anyone else get a different answer
someone else gets a I like a who said a
a lot of people said a well let's figure
it out so a eight gets one five six
seven C gets four five six seven they're
in fact equal tiebreak goes to the lower
letter because that's what we said
in fact I didn't tell you but that's
what we said question ignore all
previous rounds the question is do you
only look at the current weights when
determining a classifier or do you look
at the previous rounds as well you got
to ignore the previous rounds trust me
they will be used later in the vote but
it's sort of like painting the jury a
little bit to use the previous rounds
when you're doing the current round
because you want to start fresh with
these new weights get a new classifier
and then later everyone will get to make
their vote so you only do it based on
the current weights
nope
all the weights taking consideration
when it's wrong it's going to increase
lesson question is could you
theoretically use a classifier answer
you absolutely can when that happens it
centrally gets extra weight because you
used it again but you can never ever use
it twice in a row here's why
let's say that we wanted to use which
was one we used last over there B let's
you want to use B again what is it give
us 5050 if we want to use B and then B
three six nine twelve wrong always
guaranteed to give you 5050 which is the
only is the only way that you can be
sure you'll never use it in fact that's
by design you could reuse it but never
not twice in a row it could be used
later on down the stream and it will be
used because if you do seven rounds one
of them has to be reused it just gives
more weight to whichever one is reused
but yes a wins against CC was a
perfectly good answer as well question
what okay if you could reuse why didn't
we pick e he gets 8 out of 24 wrong it's
one worse than a and C that's the only
reason next step we'll probably use a
struggle next step we'll probably use e
frankly although maybe not because we
picked we still we got three wrong on a
but net but pretty soon we would using
again because E is pretty awesome but
anyway here we use a and we said we got
720 for strong oh man we can't use my
little shortcut so the answer it has to
be 17 sevens or 1/2 natural log of 17
sevens okay so there we go
now we have to ask what is the final
classifier of that we created from all
these things all we do is we sum up all
the classifiers we chose and we multiply
them times their weight alpha so 1/2 Ln
4 times E whether or not even turns true
right plus 1/2 on 3 times B plus 1/2 Ln
17 7 times a is our final classifier
where e returns a plus 1 if it's a
vampire if he thinks is a vampire and a
minus 1 of a thinks it's not same for B
and a alright and
then we take the sign of this and I
don't mean sine and cosine I mean just
is it positive or negative okay so the
question is now on the exam is how many
of the ten data points we get right if
we use this let's give it a look see E
is so we have romantic answers greater
than two
we have --mo yes and we have evil yes so
oh my gosh logarithms they're sometimes
annoying do we have to actually add them
up I claim we don't here's a nice
special case of having three logarithms
on the board one of two things is true
either one of those three logarithms is
so large that it's bigger than the other
two combined in which case if that one
returns a positive or negative it's just
positive or negative because that one's
so big or it's not but one is not that
large and in which case any two can
dominate the other one and so just as an
equivalent to a majority vote so I can
never have to add them when there's only
three you guys see what I mean like
let's say one of them was one half log
of a billion and the others were one
half log of 3 and one half log of four
obviously whatever the one half log of a
billion says which is multiplied by one
half log of a billion is it's just going
to be that and the others will be
ignored
however if it's not the case that one of
them is larger than the other two
combined then it's a simple vote between
the three because any two can outvote
the other one if they work together and
in this case let's see seventeen
sevenths is not quite three however log
of four is certainly not better than log
of three plus log of seventeen 7s it's
not even log of four is equal to log of
2 plus log of two and these are both
bigger than log of two that's was of log
log of four equals log of two two
squared and you can take the two out so
these are not big enough that one of
them is bigger than the other two
combined so it was just gonna be a
simple vote so let's go through Dracula
okay he's got tons of his little vamper
he's not emo so that gets it he gets it
right he's not emailed so that gets it
wrong but he is evil that gets it right
two out of three vote that he's a
vampire
correct next angel okay well he was in a
long-running series he's got plenty of
romantic interests so that gets it right
um he is certainly email that gets it
right and even though he's not evil two
out of three says he's a vampire so
correct next Edward Cullen well Twilight
here we come let's see he only has one
romantic interest so that gets it wrong
okay
he's emo so that gets it right but he's
not evil so - wrong so Everage not a
vampire according to our final
classifier but he is so we got one of
the data points wrong you guys see that
because two out of three of our
classifiers here said that he was not a
vampire
all right let's see SIA well she has
more than two remaining interests and
she's emo
so even though she's not evil we got it
right okay let's see Lestat
he is also has many love interests is
emo and is not evil so you get it right
okay Bianca is evil with many love
interests even though she's not emo two
out of three you got it right all right
um
Carmilla miracle of Karn Stein is
basically exactly the same as Bianca
with the number of romantic interest
fixed the way it is so she will always
do the same thing that Bianca does it's
why six and seven always travel together
so we get it right
Sailor Moon is supposed to be not a
vampire
so her number of love interests say that
she's not a vampire because she only has
one the fact that she's not evil and not
emo says that actually she's perfectly
not a vampire they all agree and that's
correct
squall has only one love interest rinoa
and he is not evil both of which a is
not a vampire but he is email but 203
say he's not a vampire we get it correct
and seriously despite her many romantic
interest which says she's might be a
vampire is another evil nor emo and is
not a
vampire so we got everything right
except Edward Collins which perhaps says
more about Stephenie Meyer's writing
than about our boosting all right final
question Wesley wyndam-pryce a fellow
consultant has noticed a few
correlations between some of the pasta
fires you use he suggests using a new
set of weak classifiers consisting of a
pair of your original classifiers such
as loon
a pair of your classifiers that are
logically ended and Ord together for
instance two of the new classifiers
would be emo equals yes or evil equals
yes or sparkley equals no and transforms
equals yes so they know that would cut
out Sailor Moon from the transference
cloud he believes that you'll be able to
classify large vampire data sets larger
than this one anyway more quickly with
fewer rounds of boosting using his
system do you agree or disagree with
Wesley explaining your arguments so this
was you know the tough concept question
does anyone have it just an instinctual
thing other than like ho meant was that
he must be wrong more classifier but you
have to search through more classifiers
ah that is the rare full point answer
very few people realize that Wesley was
partially right they either said
something about him being completely
wrong which was wrong or said he was
completely right yes it will use fewer
rounds of boosting because of the fact
that you can essentially one of the
things boosting already does is sort of
gets things to vote together in an end
like fashion so it may it'll use
approximately half the number of rounds
of boosting by being able to combine
into two but there's a lot of ands and
ORS there's in fact n choose 2 where n
is the number of vampires and since
using half the number of rounds but
taking n choose two time for each round
is not last time so that's exactly
correct not that many people got full
credit on that one because sometimes
they were seduced by Wesley's idea or
they just like it's Wesley he's wrong or
some or just some other funny answer um
any questions about boosting question
how you know how do you know how many
rounds of boosting to do the answer is
so on the quiz it tells you that you
have three in real life you kind of you
might want to just kind of keep it
running until it converges that's one
possibility
keep it running until it converges to an
answer and it doesn't do anything
anymore Patrick has a little widget on
the 604 website I think that lets you
plunk down some data points and run
boosting on them and you can see that
eventually it converges the boosting
converges to an answer and it doesn't
change the basically the clasificado you
picked of course or the weight but what
converges is which ones of the years of
your data set you get correct because he
does his in two dimensional space rather
than like this and he shows you the
lines the boosting is drawing between
classification and colors things in
green and red or something like that and
eventually converges where the lines are
and which ones is getting right it
generally converges to getting
everything correct and when that happens
then you can stop but that's a good
question and it's not always that easy
in the real world you have to sometimes
just say this is enough for me I've
given it a number of rounds and it's
much more than a number of classifiers
so maybe it won't get anything better