SPSS for newbies What the residual plot in standard regression tells you

okay hi everyone this is Phil from
statistics mentor.com and today we're
going to be looking at the residual plot
now once you've run a regression
standard regression model so your DV is
continuous then we could use the
residual plot to see whether any of the
conditions of your classical regression
models are not satisfied for this I'm
going to be looking at education data
where my DV is earnings and my IV is
years of schooling let's recap on what
this residual plot is now see I've got a
scatter plot here of my IV let's call X
and my DV so in this case t v--'s
earnings X's years of schooling and I
might have some data points like this
loads of them okay and the idea of
fitting the model is to find the line of
best fit it might look something like
this I can draw it right now what the
residual plot is is a plot of the
residuals so what is the residuals in
layman's terms the residual is the
mistake that your line makes in
predicting the value of y example say
that I want the value of the predicted
value of y when x is this amount because
remember this is just a series of number
the stood numbers so I would read off
the graph like this and go up there
however so the Pittock value will be y
this value is here however the true
value for that value of x is not that
but this so we've made a mistake and
that difference between the dot and the
line is the residual okay
so if we have to write the down the
residual for a point for a particular
point is equal to the observed value of
that point minus the predicted value
from the from your model in notational
terms it's Y I I denote size observation
- predictive value we use hat to denote
different the predictive value okay so
that it's a difference between observe
and what your model thinks is so that's
um if it's bigger than zero if it's not
zero then that's that's a mistake what's
called right the residual then so
residual goes up on the vertical axis
and along this horizontal axis you can
see variously that some people use the
IV they expand to verbal or they might
use the let's use the word or predicted
value of y standardized or under
standardized now let us use for X so I
can kind of compare these two pictures
and show you how build-up the residual
plot this particular point here I can
see that the residual the error here is
positive and do a different color okay
that that is a positive residual why
because residual is observed value
that's observed value right here the dot
there - the predictive element
predictive value is lower so the
residual is positive so we go map this
below that's the same value of x and the
residuals positive so it might be
something there this residual this time
it's negative so we go to the same axis
that negative that one that's positive
one though it's a slightly smaller one
and so on and so you can see that I can
build up a whole picture over many many
dots I can build up a residual plot now
let's think about let's think about an
extreme picture of the residual plot
suppose I have a perfect fit
I mean by perfect fit I mean that I
managed to find a line that goes through
all the points we'll say I got three
points see easily you can see that the
line goes to all points what might would
my residual plot look like residual at
this point
well the observed matches are predicted
so that's zero zero scale of 0 that's
negative that's positive there that's
also zero there residuals also in other
words when we have a perfect fit when
the line goes for all the points you
residual plots kind of like this now
this is up this is the extreme so we're
not going to see that so what can we
hope for the most residual X or the IV's
we would hopefully see something looks
like this so let's make a note of what
we observe we observe that the mean of
the residuals hence of the errors is 0
you can see that the symmetric balance
symmetric above and below the below zero
well if that is the mean of zero then
the width of this band represents the
variability or the parent standard
deviation of the residual tones the
error terms I will see that the constant
as well so that's another condition we
want is that the variance of the error
terms is constant by the way I'm using
the word error and residual
interchangeably here although there is a
difference but what won't need to
explain in this video the variance of
the error term is constant and that
condition is something we want its
jargon it's called homoscedasticity
returns have different variants we say
that the error terms are heteroskedastic
finally we can use the residuals here to
see if there's any outlier circle some
observation way out here this is far
from the
of a lot it's a possible outlier so to
recap for a good model the residual the
error terms should mean of zero but this
condition will be satisfied
automatically if you conclude a
intercept I a constant term in your
model so that doesn't need to be checked
for it will just happen another thing is
that we want our errors to be
homoscedasticity runs another thing what
to look for is that we have no outliers
all right now a few other things but
these this one can spot with the
residuals apart from but I shan't go
into detail in this video is the that of
the dots here should have rank should be
randomly scattered right but that is
only the case if we are looking at
something like time series or the data
can be ordered a natural ordering such
as in time series or if we're dealing
with distances lastly to note that the
residuals here that we're going to be
looking SPSS are not just plain old
residuals they're going to be
standardized or studentized residuals
for the reason I will explain once we
start looking at the data right we're
fitting the model then regress earnings
on school analyze regression linear take
earnings to dependent box schooling to
the independent box go to plots now what
one horizontal axis we call x-axis is
the deep the predict X or the predicted
value of y that's denoted by is Z pred
so that's the standardized actually the
standardized version of Y hat and for
the vertical axis we're going to have
you can use if the 0 is 8 which stands
for the standardized version of the
residual not the plain old residual that
I talked about or we can use the
studentized s or is it s president ok
entirely up to you it doesn't matter
which when you use continue okay
right so here is the residual plot and
on the vertical axis we have
standardized residual and on the
horizontal axis we have standardized
predicted value so does ever mean of
zero is it balanced from zero we said if
we include the intercept it should be
automatically well can see in the
residual statistics also a residual mean
and residual is zero so that's why I
said we don't need to check for it
really does have constant variance I use
it balanced around is it now is it even
banned around zero well sometimes it's
hard to tell from the picture sometimes
it's easy here you might say that well
it seems to be fanning out as with
increasing predictive value for
increasing predicted value so that would
suggest this problem of
heteroscedasticity and how about this
value here that could be an outlier it's
far from the masses isn't it now in
saying that scale is important so notice
that standardized residual the minimum
value here is -2 and it goes up to 4
because when we standardize something
remember this is think about to the
z-score we make it so that the variable
has mean is normal distribute to mean of
0 and the variance of 1 so that's why
it's tightly bunched up on this band we
know that if it falls out Sun falls
outside plus or minus 2 if it's way
outside there then it's very untypical
and it could be an outline now this
value here takes around you or before so
it's way above everything else in
absolute terms it could be it so that
reason could be an outlier how do we
identify that case that person we double
click on the picture to bring up the
child eater then I click once you see
that yellow halo is appearing all the
dots then we click on that dot that we
want again now only that dots will have
a halo
and then we go to elements and we go
show data labels case number given so
it's case 15 what does it mean to say
case 15 means that we go back to the
data set if we go here to case 15 this
oops
this the person with earnings of $19,000
years of schooling 12 is the outlier
possible outlier and why it's important
to use standardized as a or studentized
residual as opposed to the normal
residuals is that the variance of the
residuals even when conditions of
classical assumptions are satisfied can
differ and this therefore makes it hard
to compare the residuals by
standardizing we force all the residuals
to have unit variance and hence the
easier to compare just to see what the
residual plot looks like we use the
normal residuals let's go to let's run
the regression again analyze residual
after linear regression this time we go
to save and we asked it to save
unstandardized residuals
s'okay
and now what we should have is a new
column visit rez
okay so let's build up the graph using
chart editor chart builder scatter plot
listen to their the raziel the
understand the riser is Ajay the royal
visit your goals there this time instead
of using y hats the predictive values
I'll just use the X values remember I
said it doesn't really matter which one
you use okay here it is you see the
patterns is the same look at the axis
though understand our residuals look the
way it goes from minus 40 to plus 60
supposed to what we've seen before minus
2 to plus 4
so just to say again why when you
standard dies or studentized residual is
that with the unstandardized residual
each of these residuals could have
different variants so we can't really
can't really compare answer on that
Mouse it's not respect might not happen
house is going running around so we
can't can't compare them if we use
unstandardized residuals each of these
residuals kind of different variants so
we can't really compare them even if it
looks like this is far from the other
ones by standardizing them I force them
to have unit variance so and now I can
make the comparison so where we got so
far I've shown you that we're a residual
plot we can check to see that some of
the assumptions of the regression
satisfied such as zero mean but we don't
have to check for that if we go and
intercept that we have the errors are
almost got a stick and that there are no
outliers some of you will be taught and
course is that the errors should be
independent as well now that whether you
can test for that really depends on the
nature of your data if you do something
like a questionnaire and that
independent should be built into your
data collection so you would have used a
probabilistic sampling method such as
simple random sampling or cluster
sampling or stratified sampling is
something like that and then your your
data your variables will be independent
if you do something like something where
your data is naturally ordered such as
the data comes over time that's called
time series or if is some ordered vice
distance then we can run checks for what
they call autocorrelation but that's
that's a beyond this video tutorial
right okay so residuals helpful to check
some conditions visually but they are
not formal tests so formal tests would
should be run as well
okay well that's to get you started then
hope that's been helpful