Variable Selection in Linear Regression Using Partial F-Test in R (Tutorial 5.11) Video Lecture | Mastering R Programming: For Data Science and Analytics - Database Management

hi I'm Mike Marin and in this video
we'll discuss the use of the partial
left test as well as how to implement
the partial left test in our the partial
left test is used in model building and
variable selection to help decide if a
variable or a term can be removed from a
model without making the model
significantly worse conversely we can
think of the test as being used to help
us decide if adding a variable or a term
to a model makes the model significantly
better soon we will provide a clearer
definition of exactly what we mean by
better or worse we refer to the larger
model as the full model and the model
with one or more variables or terms
removed from it as the reduced model a
term that often gets used here with the
partial F test is that is used to
compare nested models let's discuss a
few examples to help explain what we
mean by the term nested models as well
as a full and a reduced model we will be
working with the lung capacity data that
was introduced earlier in the series of
videos I've already imported the data
into R and attached it now let's discuss
a few examples as a first example
suppose that we have a model to estimate
the mean lung capacity using age gender
smoke and height we'll call this model
here the full model now we may want to
consider something like removing height
from this model our thought here might
be that given that we know the age and
the gender of these children also
knowing the height may not be necessary
in our model we can test this hypothesis
here we'll compare this full model to a
reduced model that does not include the
height variable we would say that the
reduced model is nested within the full
model that is the terms used in the
reduced model are a subset of the terms
used in the full model we'd like to know
if removing height from this model
results in a statistically significant
increase in the sum of squared error or
residual sum of squares in other words
does removing height from this model
result in increased error in our model
and hence a decrease in predictive power
conversely we can think of this as
asking the same question does including
height in the
model significantly decrease the error
and significantly increase the
predictive power of the model that is
does including height improve the model
this can help us decide whether or not
we should include the height variable in
our model if height does not add
anything to the model we should exclude
it as the simpler or more parsimonious
model is always preferable now let's
consider one more example in example two
suppose that we're going to model the
relationship between age and lung
capacity and we think that that
relationship may be in lon linear and
we'd like to consider adding age squared
to our model to allow for fitting a
quadratic curve rather than a line in
this case our full model would be a
model that includes age and age squared
and the reduced model would exclude the
age squared term again we could say that
this reduced model is nested within the
full model as the reduced model contains
a subset of the terms used in the full
model we would like to know if the model
that includes age squared is
significantly better than the one that
does not or again conversely we'd like
to know if the model that excludes age
squared is not significantly worse than
a model that includes it the way we will
define if the full model is
significantly better is by determining
if the sum of squared error or the
residual sum of squares is significantly
lower in the full model as compared to
the reduced model so let's do a quick
review of what the sum of squared error
or residual sum of squares is the sum of
squared error measures how close the
models predicted Y values are to the
observed Y values in some sense it can
be thought of as summarizing how well
our model fits the observed data the
partial F test compares the sum of
squared error of the full model and the
reduced model to see if there has been a
significant change in the sum of squared
error and hence a significant change in
how well the model fits or predicts the
observed data let's consider that second
example where we'll model the
relationship between age and lung
capacity suppose the scatterplot and the
fit of the reduced model looked as
follows if we were to take all of these
residuals or these errors
square them all and sum them all up this
is what we call the sum of squared error
now imagine instead fitting what we're
going to call the full model a model
that also includes age squared and
allows for curvature for this model we
can take a look at all of those errors
squared and summed up in this fictitious
example we can see that the sum of
squared error of this full model looks
like it's significantly less than the
sum of squared error for the reduced
model presented earlier we can
subjectively say that this full model
appears to be better if on the other
hand the decrease in sum squared error
we're going to this full model was not
very much at all we would conclude that
adding age squared to the model does not
significantly improve it and is not
necessary
so partial F test makes comparison of
these two sum of squared errors but in a
more objective way to help us decide if
the decrease in sum squared error when
moving to the full model is
statistically significant the partial F
test this test has a null hypothesis
that there's no significant difference
in the sum squared error of the full and
reduced model or in other words we can
state it as the models do not differ
significantly it has an alternative
hypothesis that the full model has a
significantly lower sum of squared error
than the reduced model or stated in
another way that the full model is
significantly better than the reduced
model and of course the null and
alternative hypothesis can be expressed
in many different ways our test
statistic looks as follows we can see
that it takes the difference or the
change in sum squared error from the
reduced model to the full model and this
is divided by the change in the number
of parameters in the model the number of
terms dropped from the model divided by
the mean squared error for the full
model we can also see looking at this
test statistic the larger the value of
the test statistic the larger the change
in sum squared error or the larger the
difference between the two models let's
carry out this test using our first on
that example number two I've already
prepared a script to do all this stuff
so let's run through that first we're
going to fit that full model that
includes age and age squared then we can
go ahead and fit the
Joost model that includes only age these
are the two models that we're going to
want to compare and just for our own
sake let's take a look at the summary of
the full model we can see there that
this model has an r-squared of 0.67 to 7
and the residual standard error of 1.5
to 5 if we take a look at the summary
for the reduced model we can see the
r-squared is 0.67 1:9 and a residual
standard error of 1 point 5 to 6 just
looking at these subjectively we can see
that there hasn't been very much changed
in the r-squared or the residual
standard error when we look at the full
model and the reduced model
it seems that including age squared does
not improve the model but let's formally
carry out the partial F test you can
recall that this test has a null
hypothesis that there is no significant
difference in the models and an
alternative hypothesis that the model
with age squared is significantly better
to complete this test we can use the
ANOVA command and we will first enter
the reduced model and then the full
model in the test results we can see the
sum of squared error which R is labeling
as the residual sum of squares for each
of the models we can also see the F
statistic and the p-value for the test
based on the p-value of 0.172 will fail
to reject our null hypothesis and
conclude that we do not have evidence to
believe that the full model is
significantly better we've decided that
we don't think it's necessary to include
the age squared term in our model and
it's worth mentioning a reminder here
that failing to reject a null hypothesis
is not the same as accepting it we have
not proved the reduced model to be
better we fail to convince ourselves
that it's worse now let's go back to
that first example and test if we can
drop height from the model first we're
going to fit the full model and we'll
call it model 1 this is a model that
includes age gender smoke and height now
we'll fit a reduced model that includes
only age gender and smoke and has the
height variable removed and again a
reminder that our null hypothesis is
that there is no significant difference
in these models and the alternative
hypothesis is that the full model is
significantly better again we can use
the ANOVA command
first entering our reduced model and
then our full model and taking a look at
the results we can see the p-value is
extremely small here we will reject our
null hypothesis and conclude that the
model that includes height is
significantly better that is it has a
significantly lower summer squared error
or residual sum of squares and we can
see that large change in the sum of
squared error when we look at the output
I'm sure you can imagine all sorts of
applications for the partial F test the
main thing here is that the two models
being compared are nested it's also
worth mentioning that model building and
variable selection are very large topics
and they depend largely on the goals of
your model here we've simply presented
the partial F test as a way for
comparing nested models to see if there
has been a significant change in the sum
of squared error thanks for watching
this video make sure to subscribe to our
channel and check out our other
instructional videos