SPSS for newbies: the simple/bivariate linear regression model explained Video Lecture | SPSS: For Beginners - Data & Analytics

hi everyone Phil from Statistics
mentor.com so you know or you've been
told that you have to fit a a regression
model so just what can a regression
model do for you let's start first by
talking about what constitutes a
regression model new regression model we
have two types of variables a dependent
variable that is the one we wish to
excuse them all to explain and the
independent variable generic symbols Y
to denote the dependent variable and X
to denote the independent variable and
as the name suggests then DV the deep in
purple depends or is explained by other
variables
hence this X is independent variables
now there's some other jargon which
means the same thing so response
variable also means the dependent
variable instead of using the word
independent variables you'll see some
text using the word predictor or
explanatory variable or I've missed
another one out covariant covariant
sorry I don't have my pen with me so
covariates okay now once we have the
regression model we can use it to do two
things and it's important to bear this
in mind because it keeps you keeps your
head can focused on what you're trying
to use model for you can use the
regression model to make predictions
about the values of Y given the values
of X so in other words suppose you've
got so you're thinking about a scenario
so X takes a particular value what
predictive value doesn't model have for
y
okay now the word prediction here is not
to be confused with cause and effect we
can't say that X causes Y although a
regression model could be part of the
evidence for a cause-and-effect
relationship
prediction is slight is the word
prediction means something else it means
that given the value of x the model what
does a model say guess about the value
of y it doesn't imply that X causes Y
okay the other thing and use a
regression model for is to say things
like if x increases by one unit the
model predicts that Y will increase by
blah-blah-blah units for example in
marketing X could be amount spent on
advertising Y could be sales so we
couldn't make the statement with
regression such as if we spend an extra
1000 pounds on the marketing campaign
the model predicts that sales will which
is the Y will increase by maybe 8,000
pounds per week something like that okay
so bear in mind that once we've got the
regression we can use it for prediction
given values of X we can say we all can
predict the value of the actual value
for y or we can use it to measure the
sensitivity of the Y to X using a
sentence such as if x increases by one
unit the model predicts that Y will
increase or decrease by so many units
okay now in the first course in stats
the first regression model you will be
introduced to it's a simplest one of all
it's called the simple linear regression
or the bivariate regression because by
because there are two variables it's
just one dependent variable and the one
independent variable in any model that
is always when they're going to be one
deepened variable it can be more than it
can be one or more independent variables
in the model where is only one
independent variable we call that model
of a simple linear regression model
where we have more than one independent
variable we have what are called the
multiple linear regression model okay so
now let's turn our attention to examples
I'll show you how to get the output for
the simple linear regression model given
data and also have a brief look at the
output so I have two variables Ryne X
and moreover they are the y is scale it
is it is not categorical all right this
is important because in regression first
of all before you can run it you've got
to look at your Y is it continuous eyes
scale it is we can run the standard
regression model if it isn't you we can
still run a regression but then it's
going to be a non standard one and more
it's a slightly more technical alright
so the first regression model you ever
look at in a course will be one for
which the Y the deep in trouble will
have continuous continuous scale X
here's also on the continuous scale but
notice that there's no such restriction
X X could be qualitative or X could be
if you've got more than one type of
independent dependent model in your
model you could have a mix of continuous
and non continuous right then to fit
this simple linear regression model go
to analyze regression linear here the
box pops up okay it should be all be
empty and these options boxes dependent
so we transfer a deep end mobile into
that using of the arrow or you could
just type it in but let's choose that
notice the ruler here denotes a statists
Kayle bearable so it's fine
independent variable Riv just X so we
transfer that in
now their various options here but we're
just not going to look at those we're
gonna just look at the Bach standard
output so click OK right so let's just
have an overview of what tables were
given given one table two tables three
tables four tables right there's a heck
of a lot of output the first one doesn't
tell us anything just tells us what is
what X is in the model model summary now
the we're interested here in the R and R
square for simple linear regression R is
actually here and a simple linear
regression R is the correlation
coefficient so point we call a
correlation coefficient takes the value
from minus 1 to plus 1 close is to
either minus one of the plus one is a
stronger the negative or positive linear
relationship between the two
quantitative variables here it's 0.599
that is over 0.5 so you could say there
is and moreover it's a positive sign or
negative so we can say it's a positive
correlation between y and X and that it
could be moderately strong now one has
to be careful about talking about the
strengths of relationships because with
real data depending on your subject
field you're studying in some subjects
around point three could be moderate and
some subjects point three could be
considered quite strong
okay the R square not to confuse 'var to
ask wise otherwise known as the
coefficient of determination takes the
scale of zero to plus one and closer is
to 1 that means closer the line is 2 all
the dots in the case that a square is
equal to 1 means you've got a perfect
fit because the model will explain
precisely the explain precisely by the
relationship between y and X in other
words the rely on passes through all the
dots which in real life data will not be
probably not be the case
so here the r-square is 0.36 or 36% we
could say that 36% of the total
variability in the dependent variable is
explained by the model I'll say again we
can report if we have to write this down
a square equals not 0.359 means
approximately 36% of the total
variability in the DV is explained by
the IV or the model now in regression
there are two basic tests that are
presented in the output you have an F
test which is here under the ANOVA table
and we also have a t-test okay recall
that for any test since we're doing
applied stats if any test we just need
to know what is the null and what's the
alternative hypothesis for that test and
then we just interpret the corresponding
p-value denoted but in SPSS by sig all
right for the F test here the null
hypothesis always the ANOVA table in
regression the null hypothesis is that
the model has no explanatory power
another way of putting that is that IVs
in the model do not help to predict why
yet another way to say that is there is
no relationship between any of the IVs
to the DV okay pick whichever one of
those statements I made make sense to
you okay so the values five point six
what is the choripán but that value is
not of interest in itself but if you
were to report it you just write that
down we're actually interested in the
sig the p-value 0.039 that is less than
if we're going to use the
yeah standard yeah stick naught point
naught 5 it's less than that so means
that we reject the null in favor of the
alternative conclude that there is some
explanatory power in the model in other
words IV there's evidence that the IV
helps to predict the DV all right okay
now many students it's quite common that
students forget this rule about the
p-value but it's easy to remember just
remember if P is low no must go so what
do we mean by low we could say that's
like not quite not 5s of threshold so
lower than not quite not 5 means no must
go ie reject the null in favor of the
alternative next we come on to the
coefficients box probably the most
interesting of the boxes we're going to
look here at the we're going to look at
a T in a moment the first let's look at
the actual model here understand guys
coefficients that's the beaters down
here constant that is for the intercepts
X that is my IV and these are the
coefficients now if I were to write this
down as a formula because this is
ultimately what I'm interested in
because this these coefficients here
will help me to get my predicted values
of Y given X and also will help to
explain to me what happens if X
increased by one unit what's the impact
on Y so let's look and see all that if
we write this down as an equation ok so
we can write this as I'm not going to be
too pedantic here y equals 1 5 6 which
is the intercept right just looking down
this column here these are the
unstandardized coefficients that be plus
not point 3 3 8
times X is a model and and this is the
formula of a straight line where the
intercept the y intercept is 156 and the
slope is positive so it's an upward
sloping curve not 0.33 8x okay that's
what we can see from here now this point
we look at the significance I mean the
sorry the T statistic okay so it's a
test so what's the null hypothesis okay
for this default output the null
hypothesis is that the coefficient in
question that is a to two T's here one
for each row I one for each of the
parameters so the null hypothesis is
that the coefficient in question is
equal to zero versus the alternative
experimental hypothesis that the
coefficient is not 0 another way of
saying expressing this is the null is
that the coefficient is insignificant
and the alternative is that it is
significant again we just need to look
at the p values to determine whether we
go with the null all the alternative now
we're not usually not too bothered about
the constants we're always interested
sure in the coefficient on the IB
otherwise known as slope coefficients
okay so the T statistic for the X is two
point three six eight the associated
preview is not point naught 3 9 which is
less than 0.05 so using the rule that if
P is low no Moscow will reject the null
conclude therefore that the slope
coefficient here is not is not 0 it is
significant
okay significant what does that mean
that means basically there is a
relationship there's evidence of a
relationship between x and y
which means that we can go ahead and
start doing predictions or
interpretations subject to a massive
caveat which is which is that I'll tell
you in a moment it's not right to say at
this point okay so there we go so this
is our model we found like this second
is kind of different from zero so now we
can use it for the purposes that were
built it for which is to talk about the
relationship between y and X or use it
for prediction let's just give you a
demo of what I mean by prediction
prediction so prediction means that give
the value for the give me a situation
where I'm interested in value of x say X
is 2 what would does the mod predict the
value of y to be according to my model
actually yeah let's use this advertising
and sale no let's not use up to so let's
just do that so y equals 1 5 6 plus not
point 3 3 8 times 2 the model predicts
that the Y will be get your calculator
it will be 1 5 6 plus not point 3 3 8
times 2 equals 156 point 7 approximately
so that is the predicted value 156 point
7 units whatever that those units of
measurement are for y okay what's the
interpretation of this slope coefficient
which is the second which is another of
the usages of a regression model if x
increases by one unit the model predicts
that Y increases because it's a plus on
the here that it's plus on the
coefficient on the slope increases by
not point three three eight units place
here we replace the words units by
whatever the measurement of x and y are
so if we're using sales X is measuring
thousand pounds and we would say if X
increased by one thousand pounds the
model predicts that Y that's why say
sales is measured in a thousand pounds
as well then Y then the sales increases
by 338 pounds per period that were
interested in right so to finish off I
have just got a few things to say first
of all so we've said what the use of
regression is it is for making
predictions give valuable the
independent variables we can predict the
value of y we can use it to measure
sensitivity of changes in Y changes unit
changes in X but we can only do that
that analysis object the thing those
statements i've just made is only is
conditioned on this model being the
Wrights model all right now how do we
know that this model is an acceptable
model well the model is acceptable if
the if two things really the if a the
statistical assumptions for regression
are satisfied these statistical
assumptions called the classical
assumptions of regression also known as
the gauss-markov theorem
okay and this is where part of it is
where you if you want to check whether
those assumptions hold this is where we
start looking at the called a residual
analysis okay which is only which is a
big part of checking whether the
assumptions hold so residual analysis so
before we actually start making what
I've said statements about worship
between x and y using natural using the
model we should have checked whether
those assumptions hold so we should have
done a residual analysis at least done a
residual analysis okay but that for a
newbie like us that where we leave that
to another time right that's very
important that okay another thing is in
some courses they will show you they'll
talk about the standardized coefficients
because notice that they actually two
sets of corphish's
under standardized and the standardized
no there isn't one for the constant
there is one for the intercept
standardized coefficients basically just
alters the units of measurement so
whereas in the first example I gave you
about marketing about 1,000 pounds
increasing advertising for the X leads
to a 338 pound increase in sales per
week say so x and y here both measuring
thousands of pounds but sometimes x and
y could be measured in different units x
might be kilograms and y could be
something like could be like yes pounds
and pence or dollars or something like
that different units by standardizing it
but the units of measurement for x and y
becomes thumb deviations so if we were
to write down the models based on
standardized coefficients we would say
for one standard deviation increase in X
naught predicts that Y will increase by
point six standard deviations okay why
would they want to do that well if they
the same unis it might make it easier to
understand okay
finally I'm going to finish off just by
trying to link up some of these results
in the tables together to make you kind
of tie up some ice ideas here now so
this final part of video you might kind
of watch it another time okay I want to
point out two things let's go let's do
the hapless dismal point thing first F
Ches t-test right I want to point out
this stage that there are many F tests
and there many T tests by many I mean
that these F test and T tests do not
have the same null and the alternative
so this F test I said none of the table
Mesa and null is a null has no exponents
or power it doesn't mean that every
single F test you're gonna see in your
career and statistics will be testing
that that the model has no explanatory
power why these such testicle F test is
because test is based on looking up the
critical value in the F table table of F
distribution likewise the t-test is
called t-test because there many such
teachers all these T tests are called
t-test because you need to look up the
table of the T distribution to find
critical values all right so don't think
that you just memorize it once and then
it's always going to be the same however
for the group for the regression and
default the F will always mean the null
is always means that the mall has not
spent her power and T is testing for the
significance right I want to say that
the F if you think about it the null for
the F here is actually the same as the
null for the T in other words for the
simple regression model they're both
saying the same thing they're identical
because what does F say to think about
what I said before
one way to
set up one way to express the null of
the f is that IV does not help predict
why what let's say missing must be same
as saying the coefficient on the IV is
zero but that's what the t-test is
testing for down here and that's why
it's not a surprise if you look at the
significance level point zero three nine
look at significance for the F same is
no more over let's get out my calculator
look what happens if I take the T for
the slope parameter two point what is it
three six eight and I square it I square
it I get five point six oh look at that
it's the same in other words the
relationship between T and F is quite
clear T Square is equal to f you're
telling is the same they're doing the
same thing but this is only for simple
linear regression once you start doing
multiple don't I think that this won't
hold one saying okay so with that bit of
trivia might kind of seem to be a bit
magic to you right finally there is also
relationship and this could be an exam
type question if you're studying this
thing for you know college there is a
relationship between the correlation and
R square as well now let's take the
correlation which is point five nine
nine and we square it
go get 0.35 94 lined up look at the a
square point three five nine in other
words a correlation Square equals R
square and this bit of trivia this only
holds for a simple linear regression
model I want for which there is one high
V okay so the first remark I've made
ties up the F with the T so it ties up
the results of these two tables and the
model summary were tied up relation
between R and R square and these things
I mentioned
hold only for the simple linear
regression model so to recap use of the
regression model is to say something
about the relationship between x and y
which is stronger stronger kind of
statements than correlation can make but
these statements can all be made if the
model is adequate but adequate means
that we should check that the conditions
of the assumptions of regression hold
and part of this require using residual
analysis okay but that's for another
time and that's where it gets pretty
messy
that's why statistics ain't so simple
okay well get some data and have a go of
that and best way is to practice okay
good luck