SPSS for newbies: Watch out for multicollinearity in multiple regression Video Lecture | SPSS: For Beginners - Data & Analytics

hi everyone Phil from Statistics
mentor.com
now when we were running a multiple
linear regression which is one where we
have more than one IV something I have
to watch for is possibility that there
is high correlation among the IVs or
that there is a strong linear
relationship among the IVs such a thing
needs to problem what of what is called
multi collinearity why is that a multi
color to a bad thing why don't you watch
for it well we have to watch for it
because it could make its appear as
though some one or more of the IVs may
be insignificant when they're actually
significant ie multicollinearity it may
appear that some of the IVs may seem to
be not to be predictors of the DV when
they actually are okay
of course then if you if it appears that
they are not significant you end up
dropping them and you and hence you you
get a poorer model because you exclude
something that should be in the model so
we want to know how to detect whether
there is a presence of multicollinearity
in the model that is of a level that is
important to deal with
so let's say again multicollinearity is
a problem that could arise in multiple
linear regression not simple linear
regression and it's bad because it could
lead us to thinking that certain
coefficients on IVs are not significant
when they actually are now water clarity
is not like a light switch on or off
it's basically what interesting it's
only an issue when the degree of it is
high so when we look at multi courante
we say it's an issue it's because the
degree of it is high or severe okay so
it's on it's on some kind of scale kind
of thing if you imagine it 4-0 up to
very very severe to perfect when we have
perfect multicollinearity we won't even
be able to estimate the model you will
find you'll see that there's an error
message although I think on this SPSS
pasw now you won't get perfect
multicollinearity because if you were
where you inadvertently put an IV that
create a perfect multicollinearity SPSS
will drop one of the variables okay now
honestly how to determine whether water
on T is an issue we need something
called the tolerance or the variance
inflation factor so we estimate the
model as usual regression linear in the
depend variable put in earnings for the
independent let's put in experience
school experience squared now the
background of this application is that
we're interested in the factors that
influence earnings level experience of
work experience level of schooling
number of years in education
possibly this experience square of
experience okay now to bring up the
statistics were interested in the
variance inflation factor
multicollinearity statistics we click on
statistics this pop up box appears and
we click on colon arity Diagnostics then
ok and then ok again now if you watched
my previous videos you'll kind of
recognize this output what is different
is in the coefficients box I've got this
these two extra columns and also a new
box called Co linearity Diagnostics but
this colinearity Diagnostics does not
interest us all we need is the this one
call energy statistics now these two
depending what you've been taught here
you they're both basically saying the
same thing the ISO is 1 over tolerance
so you can look at either they'll give
the same information so let's look at VI
F which stands for so let's look at V VI
F which stands for variance inflation
factor now the rule of thumb is this if
the VI f for any IV is around or exceeds
5 then we should start looking into
possibility of should be kind of worried
about this issue of multicollinearity so
5 upwards which is this is when it
becomes a concern maybe if it's around 5
it's its concern but if it's around 10
or exceeds 10 then it's a major concern
so what we have here is for experience
that the eye F is 2 1 9 that's big way
bigger than 10 experience squared that
is 206 17 ish so it tells us that we
have a problem of multicollinearity here
and if we look at the T statistics
experience look it's not significant p
value of 0.3 one experience squared is
not significant all right from education
Theory we would expect the work
experience
to be significant but we would expect
that the more that years working
experience will impact earnings back
onto this model it doesn't and that's
because multicollinearity is present so
what can we do about modularity
well I can do loads of things it's
really in introductory stats course you
might be told well just drop the just
drop each of them in turn and see which
one delivers a model with higher R
Square and take just that R squared and
take that as the model okay so in other
words what we do here with tropics
experience and refit it and then we'll
try dropping experience squared and
refitted idea is that by dropping one of
these ones which are more tick which are
involved which which for which of this
nautical energy problem the problem
should disappear okay
but we have to be careful in just
dropping variables because if theory
says that it supports evidence that the
variable should be included then we
shouldn't really just be dropping stuff
okay so in the current in by including
this experience squared a negative sign
on this coefficient means that there is
diminishing returns to scale of
experience on earnings so it kind of
makes sense to include that although
there's a problem of multicollinearity
so what else can you do what else you
can do is you can possibly increase the
sample size by increasing the sample
size you will reduce the effect of
multicollinearity but in practice you
know increasing the sample size is not
very practical because you could
increase sample size one you do earlier
yet another thing you can do is you can
use something called mean centering now
the mean centering does not always work
mean centering means that instead of
regressing the Y and the X is will
regress the Y the dependent on the DV on
the axes which happens from from which
we have subtracted the mean so instead
of regressing earnings on experience we
do earnings on experience minus its mean
and then experience squared would be the
experience minus the mean all squared
and then statistical methods we could
employ include Ridge regression or
somehow combining these two variables
which causing the motor clarity into
one's single index okay right so I've
pointed out the problems of
multicollinearity and we've looked at
how to detect it and discussed ways of
possibly handling it