SPSS for newbies Exploratory factor analysis (principal components) Video

hi everyone so how I've ended up making
this video is leading up to Christmas I
had a guest stay with me who turned out
to be an online dating expert well she'd
written thousands of pieces for the
media and she dropped names of a few
online dating sites that she thought
were good so I checked them out and what
I found is that some of these dating
sites they have personality
questionnaires that you fill in and then
they match you by your responses some of
people on these sites talked about
themselves they talked about themselves
as being easygoing funny intelligent
articulate romantic here and even kinky
but what did the question is say about
them so this is a case when we can use
exploratory factor analysis so the idea
what we can use it for is I've got
measurements comfort can pick from
questionnaires or may not be from
questionnaires but let's focus on
question is because a lot of newbies
will be using for questionnaires I think
so these questions are there to probe
aspects of people's personality and the
questions otherwise known as items will
be interrelated some of them okay so you
could have some of them can be
concerning intelligence about computers
concerning about the GUI Romanticism and
so on so you've got loads and loads of
questions what factor analysis can do
then is summarize those loads are loads
of questions in the - and a few factors
so a factor I've given you some examples
a factor from this questionnaire
personality question I could be like
intelligence or degree of articulate
nurse and stuff like that right so
that's what it can do so summarizing
lots and lots of questions by a few
factors which can represent the kind of
interrelatedness of correlation among
these items
once we've got the fact that these
factors we can create what called
the estimated factor scores for each of
the attributes like degree of
intelligence that grew being funny
romantic and so on so can classify
people by a you know degrees of these
kinds of traits so that's one use and I
think that's would be the major use for
newbies we can then take those scores
and just do a summary stats of those
like report the mean standard deviation
do plots or could also take them to do
another other other types of deep
analysis such as regression or ANOVA
that's not the only use of factor
analysis we can use it to validate a
scale we can use it to check the Uni
dimensionality of a scale which is
necessary for running Chrome back alpha
all right but I'm not really concerned
with those other things I've just
mentioned which is to me just lots and
lots of jargon really focusing on
finding these factors and then you can
then use these factors to do whatever
you wish when you're reading around
you're going to find there are two types
of factor analysis there's exploratory
and confirmatory we are going to focus
on exploratory I'm not going to just
don't need to spell out what these types
of things are I'm just telling you so
that when you're reading around forget
anything that's talking about
confirmatory factor analysis for the
purpose of this video which is on
exploratory factor analysis okay another
reason really why profit making this
video on EFA is because it's kind of a
lot of murky murky things about it a lot
of gray issues areas that are Vegas
sometimes it's a bit like playing with
tarot cards you get what you want
especially how some people teach it so
but being newbies we want
like we'd appreciate a like a recipe you
know steps simple steps that we can
follow to kind of steer as clear of all
these murky things so stay
is what I'm going to discuss next in the
finally I'm going to do an example of
the EFA on personality data ok the steps
is like my flow chart which I know some
of you appreciate so just like with all
other kinds of some other kinds of
analysis we have conditions and
assumptions that we need to kind of
check first okay if I don't tell you
about these assumptions or the more
important ones it's a bit like selling
you a dodgy car all right I just show
you the good bits I don't tell you about
the faulty bit so while we're together
the can might be fine but when you go
off and drive off you might find that it
doesn't start so I don't feel like I can
tell you about FA EFA without telling
you a bit about the assumptions
otherwise I'm a bad salesman aren't I so
the classical exposure of factor
analysis is based on if we still think
about the question is that your
variables they are normally they're
continuous and normally distributed or
could you say normally distributed
jointly normally distributed because
that implies that it must be continuous
obviously with questionnaire data it's
not going to be like that we're going to
have mixtures of could be continuous
could have ordinal because we could have
Likert scales could they have normal as
well now if you look around the
literature the statisticians will tell
you you cannot apply EFA as SPSS does it
with ordinal and nominal data but if you
read outside the stats literature like
psychologists how they use it they'll
just say this is where we pretend the
ordinal are continuous okay hmm
so being newbies let's accept that but
if you're going to do a more kind of
serious analysis just know that if
you're going to treat ordinal as if it
were continuous then you are likely to
over you're likely to come up with and
come up with more factors than actually
are present so if you bear that in mind
then even if you have ordinal you can
kind of just cut down on the factors or
kind of just
bit more cautious about choosing more
factors okay
because you know that if you use Likert
scales you could come up with more
likely to come up with more factors than
actually there and this has been shown
in simulations okay
nominal nominal so this is like where
the variables the level measurement they
they have no ordering if it's just like
yes or no answers what called binary or
dichotomous that's accepted okay but not
if it's like on more than two modern two
levels so for example religious belief
Christian Christian I don't know
Buddhism no it's a bad example and
another one okay so three then this
wouldn't work
there are ways around that but for a
newbie just know that hopefully that
you've set it up to you're not your own
got continuous and ordinal all right
maybe nominal if it's just two
categories alright so then the next
thing we do is we also have to know that
factor analysis is a large sample kind
of procedure so you've got to have this
is where you get different numbers
depending who's teaching it to you some
say minimum of fifty respondents or what
called cases okay so in other words
fifty people have responded to the
questionnaire some people say at least
you want 100 or right I guess it really
the number really depends on how many
factors you're going to try to get
outward thing but just say we want lots
and lots just know what we want our
sample size to be large the other thing
is that we got to have enough questions
as well to make up these factors so
again depending what you read around
some people say if you expect you should
have like five three to five variables
minimum four per factor so if you expect
that to be a factor measuring degree of
trusts that somebody has in a partner
you'd hopefully you'd have at least
three to five questions relating to
trust so it's really how you are
constructing your questionnaire here you
know you've got some hypotheses you've
got some these factors in your head so
you'll make sure you've got enough
questions concerning those factors okay
and then then there are a couple more
couple more conditions that we're just
going to come across when I look at SPSS
right the next step we can go safely go
to the next step is that determine the
number of factors all right so we want
really to have the whole point is we're
supposed to kind of reduce these number
of items to be represented by a few
number of factors so if we've got like
50 questions we're not after 50 factors
all right because that's not we have
introduced anything well I like more to
kind of have those 50 questions 50 items
reduce down to say like 4 or 5 or 3
something very small number of factors
okay which those factors should carry
some kind of meanings which relates to a
set of questions in your questionnaire
so once we've determined number of
factors the next step is to rotate right
this thing called rotate just jargon at
the moment whole point of rotating is to
get a sharper distinction between the
factors meaning that I want to be a
clear a distinction that this set of
questions is about degree of
intelligence this are the set of
questions about the degree of somebody's
sportiness or something like how they're
extra looking a degree about being
extrovert
okay then flowing down this chart what
happens next is we drop the pour factors
I pour that to meaning maybe it's got
poor interpretations and we have a clear
interpretation what the thing means or
there's not enough it's not related to
enough questions like you know Isis said
we need two three five questions per
factor if we had only one question for
that fact it's pointless keeping that
factor and also we tend to drop
variables that a lot and of associated
with modern one factor what they call
them cross what do they call it cross
cross loading or something
yep cross loading I am NOT an expert in
factor analysis by the way okay so a
good point about that is that I'm asking
simple kinds of simple questions that
hopefully you'd be asking okay you can
see that the this factor analysis is a
cyclical nature
so once we've dropped if we had to drop
anything
we kind of cycled back and we re
estimate with a fixed with the same
number of factors and then rotate again
and just see what we get okay once we're
happy with that that means we've got the
result and at this stage your your your
teachers might stop and you're thinking
why you just stopped you know what's the
point of then you're doing factor
analysis
so once you've actually gone through
there then you're going to next obvious
step to do for newbies is go ahead now
and calculate or estimate these
estimates of the factors course or
estimate of their degree of intelligence
estimate of the degree of
trustworthiness all this kind of stuff
if we are moving on to try to calculate
the comeback alpha will help we have
kind of verified the uni-dimensional
nosov the scale or we could have also we
could have validated the scale as in
meaning that we've kind of checked that
all the questions and measuring what
they're supposed to measure okay already
that sounds like long enough so let's
jump straight into the application so
I've got the personality data here okay
I did not go online and gather up all
this data from a dating website it's
just I've got a date personality
questionnaire it has it has it has 44
questions and I just delete this because
this is like something I did before I
started 44 questions and how many
respondents okay that's well over 100
that's 459 so cool I can run this thing
except for we know this that the
variables all these variables they are
on the cout scale I know they're all
ordinal I've got no nominal
got no continuous so basically this
means that if I'm going to run it on in
SPSS standard SPSS packages then I am
likely to overestimate the number of
factors all right which I might say that
in my report to kind of show my
supervisor that I've been reading around
I actually know something more than
books actually usually tell you so next
thing is determine the number of factors
right before I go into this let's just
assume this because there's so many
buttons that are options that you can
use in EFA we're just going to assume
that all the roads lead to the same
place okay if we just bear in mind that
at the end of the day what we want as
newbies is something factors that
interpretable then we not don't care
which of these kind of buttons were
pressing okay because depending who
you're taught by some professors have
some kind of liking for some methods
rather than others and I'll kind of
point them out as we're going along but
if you're on your own and you haven't
kind of been taught FA and using a big
dissertation just kind of bear in mind
that so long as we get some kind of you
get some result that is meaningful then
you don't need to spell out exactly how
you got there right they'll just be
happy with the factors that you end up
with at the end of the day okay because
if you don't bear that in mind it's like
you've got all you're going to get
anxious about pressing all these buttons
whether these buttons do you know what
do they mean and that's pointless you
don't need it okay so we'll go to
analyze and it's a dimensional reduction
method that's so that's what this is so
we'll go down to dimension reduction and
it's factor that we click okay I'm going
to reset this because I did it before
you want to take select the variables
and push it over into the variables box
now all these come from the same
questionnaire and putting them all in
okay then what
and assumptions will go up to
descriptives I just click on PMO and
Bartlett's test of sphericity okay so
they give you two things that we can
check for use use the practice okay to
get initial number of factors we need to
click on extraction so that's extract
the number the factors we've got choice
of method and now this is like where
it's like guard is down it's up to you
it's up to you statisticians if we click
on this prefer apparently the maximum
likelihood method ml and principal
components whereas say social scientists
they tend to talk about using weighted
using least squares or principal axis
factoring and sort a psychologist well
there are assumptions for these things
depending which one use so like for
maximum likelihood it kind of assumes
that kurisu's like joint normality
alright which for Likert scales is I
mean joint a lot of the factors which if
you do look at they've got liquid skills
you might say well maybe not okay or
maybe yes but if we bear in mind that so
long as we get a bouquet result we don't
care what we're going to use so let's
just use something alright so maybe
we're going to use a principal factor
not principal axis factoring analyze and
most people have done correlation matrix
they analyzed the correlation matrix ie
pick out the factors from the
correlation matrix so we're going to
stick with that so don't go for
covariance matrix if you go for
covariance matrix the method for finding
the number of determining the number of
factors as I'm going to show you is not
going to work so just leave it on
correlation matrix display and relative
factor solution leave that checked this
is thing called the scree plot mmm
most people check it so I'm going to
check it alright it's not great but
we're going to check it extract based on
eigen value we'll leave that at 1 and
continue
and for now that's what we need so let's
just click okay we start off with this
box that would we ask for the KML in the
Bartlett's test this kmo that is is a
number it's a number that measures the
proportion of variance in the variables
that might be explained by underlying
factors so it's a proportion that must
mean its value between 0 and 1 so closer
is to 1 the better and this is 0.8 for 1
and our some books a bigger than 0.6 is
fine for social science I'm say because
in 0.7 we are we are higher all right so
that's good that's basically saying that
a lot about 84% of variability in the
variables can be explained by some
factors underlying factors that is not a
test that is just a number
next thing Bartlett is a test okay being
a test it must have a null and an
alternative the null is basically that
there is no correlation among the
questions do we reject that go to p
value which is sick yet we want that to
be less than not less than naught point
naught 5 this is tiny so good means that
they structure their there are factors
that we can pull out they're
interrelated questions ok then I'm going
to go down to the total variance
explained box and there are two three
columns all with initial eigen values
the factor Rahman one initial email use
in another and extraction sum of squares
loading all we need to do because we
want to avoid any kind of we're not
doing a deep kind of analysis here want
to go straight into this second column
extraction because this tells you the
number of factors that have been pulled
out and you can show that the working is
like from this side okay so we can
assume so you can ignore that because so
we just want the results so if we look
down here we just go down to the bottom
number and go along 10 so we've got 10
factors from this 44 question 44 numbers
of a question yeah so if we go down to
the bottom table 44 that's like 44 right
44 questions has been reduced down to
ten things so those 10 things helped us
will summarize hoping most of the
variability in 44 questions now how did
I end up with 10 you know this is where
because we told SPSS to base it on
I can values the factors where eigen
values bigger than 1 so that's what it's
done so if you slide along here this is
where it's what 10 points is 1.04 4 it's
bigger than 1 that's why you've got
number there so it's taken across this
side whereas the next one do we need a
level factor well that I can value there
is 0.96 6 it is less than 1 hence it's
gone and that's it now if you look at
the screen plot it's basically a plot on
the x axis of the factors running from 1
to 44 plotted by the eigen values okay
so so long as you know that you already
know that if we're basing it on eigen
values bigger than 1 to say it's a
factor then you don't need the scree
plot right so scree plot is just an
alternative but some people like to look
at something visually so we can look at
that ok but just know that they're
slightly the screw pot just showing you
what you've already just seen but so if
you decided on the under one criteria
stick to that because screw plug is not
always clear what you do the way the
teacher is like they say suppose there
is a kink or like to use the word elbow
where that kink is then that's the
cutoff but if we determined that the
eigen value is 1 that was a cut-off
flare then would select whatever is that
number okay so I mean could just put
this in a report and just say you've
based it on line value bigger than 1
those two are not the only ways but
they're the most popular ways of
determining the number of factors okay
so let's say we stick with 10 so now we
want to kind of want to see if there's a
mean get a more interpret all meaning of
these 10 factors and usually this is
where we just rotate so if we go to
analyze again and we go to a dimension
reduction factor this time go to rotate
all right ok they're like how many
options are the 1 2 3 3 4 5 options
the most widely used ones are Eva berry
Max or or Pro max alright and the
distinction between the two is that the
very max one says that the factors our
ten factors are orthogonal to each other
whereas the pro max says that our
factors can be correlated which either
so orthogonal means like the independent
of each other nothing do each other
where the pull max says that our factors
for example in our case it could be I
don't know intelligence and being funny
could be like correlated right
if intelligence came out to be fact and
funny came out to be a factor so so
people deficit say go for pro max
because why do we assume our fog analogy
when when that's very strict thing
basically formality in maths is very
nice because it leads to kind of neat
results so let's just go for pro max but
again if you bear in mind that doesn't
matter what we do so long as we're going
to get something that's interpretable
then we're fine
okay and we say okay now just to make
the output clearer easy to interpret on
the on the on the factors so we go to
options and we go down here where it
says a coefficient display format and we
go sorted by size suppress small
coefficients suppress small coefficients
just mean delete or just do not show
those I don't use the word loading I'm
going to have their i loadings that are
below a certain value and what they tell
you textbooks are stuff is 0.3
right so we won't do anything below
point three so want something meaningful
in other words we don't want too small
and okay missing values we're just going
to if you have missing values in your
data set which in reality people do
we're just going to just leave it on
default exclude cases list wise okay and
then we're going to go okay
right cool
what you see here and this is where if
you look at my first what we've got here
is we've got a factor maker pattern
matrix structure matrix and a factor
correlation the factor the pattern
matrix shows us what are called the
loadings now these loadings I'll show
you what we're going to this loadings of
what people report the structure matrix
reports the correlation between the
variables and our ten factors okay then
you've got the factor correlation matrix
which shows you like the correlation
among our ten factors so let's go to the
pattern because that's what we're really
interested so we've got our questions
going down and then the first column and
then we've got these other things going
on
one for each factor and here's the thing
we're trying to interpret these factors
we've got ten factors this look at them
so let's go to the tenth one so I want
to see like now I want to see if I can
kind of chop anything out because we are
down here do I drop or factors variables
that are loading more than one factor
that kind of thing
let's go part way to doing this so for
the tenth let's just have a flick down
to three account of three items with
that okay only three right so that one's
only got three so we might say that we
might want to chop that out how about
the ninth fact how many questions are
related to that remember we're after
like minimum of three to five that's got
one two three four and eight one it's
only got three as well okay well knowing
that we're dealing Likert scales ordinal
data we're likely to have overestimated
so why did I just chop it go straight
down to okay seven has got so our
seventh fact has got one two three four
five cool so we could keep that why
don't we go straight to seven so in
other words I had dropped the poor
factors because the last three don't I'm
going to just say look they don't have
enough then are loaded onto enough items
so let's let's just do that we're going
to go down again to analyze factor and
now we're going to go to extraction and
we don't need a scree plot so so we
don't get cluttered with so much rubbish
on our screen let's get rid of that
let's get rid of that as well and rotate
it we don't because we're not interested
any of that stuff
what's a fixed number of factors now I
said we're going to drop eight nine ten
now so that means I only need seven
factors I'm telling seven extract only
seven and we'll go okay and we don't
touch anything else
okay so I'll still want it rotated and
sit okay and then we want to go down to
the pattern matrix again rightly see now
it's got seven columns factor seven
let's look at well my seventh one has
gone down is still three and my six has
got three four five questions so I'm
going to cut one more okay I'm going to
analyze damage your reduction factor you
can have over me by the way with what
I'm doing because I'm no expert this but
extraction six I'm going to come up with
something that's meaningful okay let
continue okay pattern matrix click
that's the first way to do it six let's
see six one two three four five okay I'm
not satisfied with that now so let's see
if we can do some interpretation let's
look at the first one first factor the
adjectives here disorganised it's got a
minus minus means that it's basically
the opposite of this so if this was a in
other words if it's - it means that nom
let me just rewind right this is what I
had a hard time finding you should know
this is that the pattern matrix show you
the loadings the loadings take values
between minus 1 to plus 1 because when
I'm saying small or big how the heck do
you know what is small what is big if
you don't know what the minimum value
and the maximum values are if there are
any
so I'm telling you that the minimum
values minus one and the maximum value
is plus one now this is minus 0.75 it is
not a correlation it's a loading okay
but this minus 0.75 is closer to minus 1
then it is to 0 isn't it so it must mean
it's quite negative related that's what
this sign means so when you look at each
of these what call the loadings here you
look at the sign which tells you is like
correlation is like which is it is it
kind of from positively or negatively
related and then the magnitude so I'm
just interested in the sign at the
moment the sign is negative so it kind
of means that this factors to do with
more organized person because the more
negative it is that means that actually
the more opposite is of this adjective
so it's not this person's organized ok
this factors to do organize there the
circle one is positive does a thorough
job ok so it means that it is a thorough
job efficient efficient ok that's a
positive one negative one careless oh it
must be the opposite of that means this
practice to do something that's been
more careful lazy that's a negative one
that means again opposite of this so
it's per person that is not lazy
this persevere is is positive it's quite
interesting this and reliable yep so
sticks the plan positive so all anything
else no so all together we've got to
kind of think of one adjective or one
phrase that summer you know what is this
factor measuring well is it measuring
his degree of well first of all it's
like positive quality isn't it somebody
you would use the word niceness to do
that isn't it so you want to find one
adjective that summarizes everything
that I've just said there well how about
just all run goodd goodness you can
probably find of something more kind of
succinct but does this the thing with
factor analysis is you've got to come up
with these phrases ok one thing to say
oh I've got pluses and negatives is yeah
it's because see some of these questions
is like AAPIs is how you phrased it so
if you wanted all to be positive instead
of positive and negatives especially
when you come to calculate
these factor scores you are going to
have to kind of recode it opposite code
this so that this is talking about being
degree of organizing this all right is
there been degree of degree of
disorganized and then you can see that
the next one I'm going to do some things
one more time so second factor it's
positive and it's quiet so it's positive
related quietness it's positively
related to the reserved Necedah we're
going to shyness is negative related to
talk of tipping us that means it must be
positively related to not talkativeness
all right if that's a word and
negatively correlated outgoing so
positive must mean negative must mean
it's the opposite the degree of not
outgoingness so this could all be like
as a character like a introvert nurse
isn't it basically like me
this one here Oh nervousness okay and so
on so if we go back to our flow chart
we've dropped the pour factors we
haven't opted any variables and I'm not
going to do it but I'm just going to
show you something we want to get rid of
things like nervous not because we don't
like the word it's because it's related
to more than one factor okay so we don't
want that factor two and three both
contain nervousness so that's called
cross loading we don't want that in this
case it's also the factors are not
uni-dimensional
okay so I have determined the number of
factors I've rotated them to kind of get
a sharper image of these
interpretability of these factors all I
don't do for you there but you can see
what I was doing
I've dropped the poor factors and I've
told you how to drop the poor variables
the data with more than one factor and
then we re estimated and yep and we've
got something of that so we're happy
with now of six factors now forty four
questions down can be rested by six
factors that happen each have a meaning
and now we can do the following we can
estimate the factor scores we can
estimate the factor scores now yeah I
know that newbies like to do this
because once you've got the factor
scores then you can compute the facts
like the degree of introvert nurse
okay and then report that or calculate a
mean standard deviation of that okay
that's good which is good
yep it's good so let's go over to to
this thing here there's nothing to do
with kinkiness here
okay well this question has nothing to
do with romantic nests or anything no
it's not not too it's not very
comprehensive it's not very
comprehensive a questionnaire on
personality is it so say we want to
create a score so we've got these six
factors well well
one thing you can do is I did it in
another video if you simply take these
questions will it like for the first
factor you want to measure one number
for each person that summarizes their
that measures their degree of se the
second one group of their degree of
introvert inist you will add up their
score of quietness reserving a shyness
the shyness now talkativeness you can
have to reverse code so not
talkativeness and then otherwise you get
all these signs are the same outgoing
and you've got to kind of reverse that
as well so it's talking about not going
out outgoingness alright so in other
words that they're all talking like in
the same direction then you can add them
up that's one way to do which I've shown
you in another video how to do another
way that that's gives you the unit
weighted what's called unit weighted the
scale of score
so what other way we could do is we can
use a weighted scale and and we can do
that in various ways so that we just go
down to dimension reduction factor and
this is where Z scores again I'm going
to bet again we've got more than one
option here I'm just going to bear in
mind that we just want to score all
right
this three methods regression Bartlett
Anderson Rubin just note that for all of
them the calculate scores so that on the
average score is zero so if somebody is
gets a score above zero it means they're
above average if it's negative it means
that they're below average all right if
you want the clear distinction between
them click on help button which will
tell you the
tell you more about the differences
between these three but the main thing
to notice that that there calculate the
mean values as zero okay let's continue
what's going to happen now is going to
should create it will create an
additional column for each of our
factors with the score alright here we
go
if you look down here yes look can you
see one two three four five six
aha so yeah I did six not seven right so
yeah so six view so remember the second
one was to do degree of introvert nice
wasn't it so degree of introversion so
if we look at factor to my don't eat we
even give it a proper name here so it's
revision intro the introvert extrovert
introvert okay just say introvert so
click on this again okay take us over
here our introvert so the first guy
first person first person got a score of
on the introvert scale of minus 0.15
seven that is negative that means it is
a less introvert like more positivism is
the more introvert so let's introvert
than average because zero is the average
the next person one point two three
that's above average now this is like
where the web dating sites like for each
person like for this person here second
person will have like I ignore a bar
chart yeah it's good way to show is in a
bar chart showing you like their score
for each one of these so that kind of
liked all of the bar for that particular
fact that means the more you are of that
thing easy for non mathematicians to
interpret so I think that's that that's
it really I try to make this really
clear I've given you a kind of like a
recipe like this something to follow
I've told you kind of emphasize this
thing about the conditions and
assumptions which is often kind of just
not mentioned I just skipped completely
but it's a it's actually very fun very
very important so just know that because
if we're doing questionnaires we have
ordinal probably ordinal then we are
going to
if we would use SPSS we were likely to
overestimate the number of factors SPSS
does have an add-on a module called what
is it called capture something like that
categorical pca that deals with
categorical I these two guys a the
dichotomous and ordinal questions but I
think you have to pay for it so hey I'm
not going to pay for that because other
software likes data M plus already have
that built in right so I mean something
like that should be so standard now or
you could use R as well which is free so
if you are kind of more serious doing
serious research and you kind of want to
kind of Lee you know you use like your
kind of use methods which kind of
recognize the deficiencies in in the in
the standard method so if you've got
categorical variables like you do in it
in a questionnaire you you want to be
using some package that kind of takes
account of the fact right rather than
just assuming that they're continuous as
we have done in this using this SPSS
package okay well you can write lot
loads of comments I don't know I'll be
interesting reading what your thoughts
are okay cheers guys and good luck with
your dissertation