Lec 3 | 18.06 Linear Algebra, Spring 2005 Video Lecture

I've been talking I've been multiplying
matrices already but certainly time for
me to discuss the rules for matrix
multiplication and the interesting part
is the many ways you can do it and they
all give the same answer so it's and
they're all important so matrix
multiplication and then come inverses so
we're we we mentioned the inverse of a
matrix
but there's that's a big deal lots to do
about inverses and how to find them okay
so I'll begin with how to multiply two
matrices first way okay so suppose I
have a matrix a multiplying a matrix B
and giving me a result well I could call
it C a times B okay so let me just
review the rule for for this entry
that's the entry in row I and column J
so that's the IJ entry right there is C
IJ we always write the row number and
then the column number so I might I
might maybe I take it see three four
just to make it specific so instead of
IJ let me use numbers see three four so
where does that come from the three four
entry it comes from Row three here Row 3
and column four as you know column four
and can I just write down or can we
write down the formula for it see three
four is if we look at the whole row and
the whole whole column the quick way for
me to say it is row
Oh three of a I could use a dot 4 dot
product I won't often use that actually
dot column four of B and but this gives
us a chance to just like use a little
matrix notation what are the entries
what's this first entry in Row three the
first the first that number that's
sitting right there is a so it's got two
indices and what are they three one so
there's an A three one there now what's
the first guy at the top of column four
so what what's sitting up there
b14 right so that this dot product
starts with a three 1 times B 1 4 and
then what's the nicks so this is like
I'm accumulating this sum then comes the
next guy a 3-2 second column times B 2 4
second row so it's be a 3 2 B 2 4 and so
on just practice with indices oh let me
even practice with a summation formula
so this is most of the course I use
whole vectors I very seldom get down to
the details of these particular entries
but here we better do it so I'm in some
kind of a some write of things in Row 3
column K shall I say times things in
wrote a column for you see that that's
what's that's what we're seeing here
this is K is 1 here K is 2 on the long
so up so the sum goes all the way along
the row and down the column say 1 to N
so that's what the a the C 3 for entry
looks like a sum of a 3 KB K for this
takes a little practice to do that ok
and oh well maybe I should say when are
we allowed to multiply these matrices
what are the shapes of these things the
shapes are if we allow them to be not
necessarily square matrices if there's
where they've got to be the same size if
they're rectangular they're not the same
size if they're rectangular this might
be well I always think of a as M by n M
rows and columns so that sum goes to n
now what's the point how many rows does
B have to have n n the number of rows in
B the number of guys that we meet coming
down has to match the number of ones
across so B will have to be n by
something whatever P so then the number
of columns here has to match the number
of rows there and then what's the result
what's the shape of the result what's
the shape of see the output well it's
got these same M rows or cut in rows and
how many columns P n by P okay so there
are n times P little numbers in there
entries and each one it looks like that
okay so that's the standard rule that's
the way people think of multiplying
matrices I do it too but that's that's I
want to talk about other ways to look at
that same calculation looking at whole
columns and whole rows okay
so can I do ABC again a B equaling C
again but now tell me tell me about yeah
let me I'll put it up here so here goes
a again times B producing C and again
this is M by n this is n by P and this
is n by P okay now I want to look at
whole columns I want to look at the
columns of in fact here's the second way
to multiply matrices because I'm going
to build on what I know already how do I
multiply a matrix by a column how do I
know how to multiply this matrix by that
column so I call that column 1 that
tells me column 1 of the answer the
matrix times the first column is that
first column because none of this stuff
entered that part of the answer the
matrix signs the second column is the
second column of the answer do you see
what I'm saying that I could think of
multiplying a matrix by a vector which I
already knew how to do and I can think
of the vector I can think of just P
columns sitting side by side just like
resting next to each other and I
multiply a times each one of those and I
get the P columns of the answer if you
see this is this is quite nice to be
able to think okay matrix multiplication
works so that I could just think of
having several columns multiplying by a
and getting the columns of the answer so
like here's column one I call that
here's it so I call that column one
and what's going in there is a Times
column one okay so that's the picture a
column at a time so what does that tell
me what does that tell me about these
columns these columns of C are
combinations as we've seen that before
of columns of hey every one of these
comes from a times this and a times a
vector is a combination of the columns
of a and right and it makes sense
because the columns of a have length M
and the columns of C have length s and
every column of C is some combination of
the columns of a and it's these numbers
in here that tell me what combination to
do
did you see that that that out that in
that answer C I'm seeing stuff that's
count that combinations of these columns
now suppose I look at it that's two ways
now the third way is look at it by rows
so now let me change to rope okay so now
I can think of a row of a a row of a
multiplying all these rows here and
producing a row of the front so this row
takes a combination of these rows and
that's the answer
so these rows of C are combinations
of what Oh tell me about ooh finish that
the rows of see when I when I have a
matrix B it's God's rose and I multiply
by a and what does that do it mixes the
Rose up it makes it creates combinations
of the rows of B Thanks
rows of B that's what I wanted to see
that this that this answer I can see
where the pieces are coming from the
rows in the answer are coming as
combinations of these rows the columns
in the answer are comedy coming as
combinations of their of those columns
and now that so that three ways now you
can say okay what's the fourth the
fourth way
so let's now we've got like the regular
way the column way the row way and
what's left the the one that I can I
want to tell you about what one way is
columns times rows what happens if I
multiply so this was Rho times column it
gave a numbers okay now I want to ask
you about column times row
what does if I multiply a column of a
times a row of B what shape of my ending
up with so if I take a column times a
row that's definitely different from
taking a row times a color so a column
of a was what's the shape of a column of
a n by one a column of a is is a column
got M entries and one cup and what's a
row of B it's gotten one row and teacup
so what's the shape what do I get if I
multiply a column by a row again a big
matrix I get a full-size matrix if I
multiply a column by a row I get shall
we just do one let me take them the
column two three four times the ro16
that is a that product there I'm just
following the rules of matrix
multiplication those rules
looking like kind of petite kind of
small because the the rows here are so
short the columns they're so short but
they're the same length one inch so
what's the answer
what's the answer if I do two three four
times 1/6 just for practice
well what's the first row of the answer
to 12 and the second row of the answer
is 318 and the third row the answer is
424 actually what am I mean that's a
very special matrix there very special
matrix what can you tell me about its
columns the columns of that matrix there
are multiples of this guy right there
are multiples of that one which follows
our rule we said that the columns of the
answer were combinations but there's
only to take a combination of one guy
it's just a multiple the rows of the
answer what can you tell me about those
three rows they're all multiples of this
row they're all multiples of 1 6 as we
expect but I'm getting a full sized
matrix and now just to complete this
thought if I have now let me write down
the fourth way a B is a sum of pawns of
a times rows and B so then for example
if my if my matrix was 2 3 4 and then
had another column say 7 8 9 and my
matrix here has say started with 1 6 and
then had another column like 0
then here's the fourth way okay I've had
two columns there I've got two rows
there so the beautiful rule is seeing
the whole thing by columns and rows is
that I can take the first column times
the first row and add the second column
times the second row so that's the
fourth way that that I can take columns
times rows first column times first row
second column times second row and add
actually what will I get what will the
answer be for that matrix multiplication
well this one that's just going to give
a zero so in fact I'm back to this
that's the answer for that matrix
multiplication I'm I'm sort of like
having to put up here these facts about
matrix multiplication because it gives
me a chance to write down special
matrices like this this is a special
matrix all those rows lie on the same
line all those rows lie on the line
through one six if I draw a picture of
all these low vectors they're all the
same direction if I draw a picture of
these two column vectors they're in the
same direction
later I would use this language not too
much later even I would say the row
space which is like all the combinations
of the rows is just a line for this
matrix the row space is the line through
the vector one six all the rows lie on
that line and the column space is also a
lot all the columns lie on the line
through the vector two three four
so this is like a really minimal matrix
and it's because of these one okay so
that's a survey now even yeah I can I
will you with your take this is I want
to say one more thing about matrix
multiplication while we're on the
subject and it's this you could also
multiply you can also cut the matrix
into blocks and do the multiplication by
blocks yeah that's actually so useful
that I want to mention it what multiple
so I could take my matrix a and I could
chop it up like maybe just for
simplicity let me chop in it into is a
four square block suppose it's squared
let's just take a nice cake and be
suppose it's square also same size
though these sizes don't have to be the
same what they have to do is match
properly here they certainly will match
so here's the rule for block
multiplication that if this has blocks
like a so maybe a 1 a 2 a 3 a 4 or the
blocks here and these blocks are B 1 B 2
B 3 and B 4 then
answer I can find block I can find that
block and if you tell me what's in that
block then I'm going to be quiet about
matrix multiplication for the rest of
the day what goes into that block you
see these might be this matrix might be
these matrices might be like 20 by 20
with blocks that are ten by ten to take
the easy case for all the blocks in the
same shape and the point is that I could
multiply those by blocks and what goes
in here what's that what's that what's
that block in the answer a1 b1 that's a
matrix type of matrix it's the right
size ten by ten and anymore
plus what's the what what what else goes
in there a2 b3 right it's just like
block rows times block columns I don't
know buddy I think not even Gauss could
see instantly that it works but somehow
if we check it through all five ways
we're doing the same mostly patients so
this this familiar multiplication is
what we're really doing when we do it by
columns by rows by columns times rows
and by blocks okay I just have to like
get the rules straight for matrix
multiplication okay
alright I'm ready for the second time
which is input okay
ready for it
and let me do it for square matrices
first okay so I've got a square matrix a
and and it may or may not have an
inverse right not all matrices have
inverses in fact like that's the most
important question you can ask about the
matrix is if it's if you know it's
square is it invertible or not if it is
invertible then then there is some other
matrix so I call it a inverse and what's
the what's if a inverse exists so this
is if there's a big if here if this
matrix exists and it'll be really
central to figure out when does it exist
and then if it does exist how would you
find it but what's the what's the
equation here that I haven't that I have
to finish this matrix if it exists
multiplies a and produces I thanks the
identity and actually there's a little
more to it because normally I mean that
right now that's like a left inverse
it's sitting on the left of it but a
real an inverse it for a square matrix
would be on the right as well so it so
this is true to that if if I have it
yeah in fact this is not this is
probably the this is something that's
not easy to prove but it works that a
left then for square matrices a left
inverse is also a right in
if I can find the matrix on the left
that gets the identity then also that
matrix on the right will produce it
again for rectangular matrices we'll see
a left inverse that isn't the right
interest in fact that the shapes
wouldn't 11 but for square matrices the
shapes allow it and it happens if it has
an inverse okay so give me some cases
let's see I hate to be negative here but
let's talk about the case with no
engines so so this is this is these
matrices are called invertible or
non-singular those are the good ones and
we want to be able to identify how if
we're given a matrix hasn't done an
inverse can I talk about the singular
case no English
all right best to start with an example
tell me an example lets us get an
example up here let's make it two by two
of a matrix that has not got an inverse
and let's see why let me let me write
one up know it let's let's see what let
me write up 1/3 2/6
why does that matrix have no engine you
can you can answer that various ways
give me one reason well you could if you
know about determinants which you're not
close to you could take this determinant
and you would get zero okay now let me
let me ask you other reasons for other
reasons that that matrix isn't
invertible yeah I could use use what I'm
saying here suppose suppose a times some
other matrix gave the identity why is
that not possible because oh yeah I'm
thinking about columns here if I
multiply this matrix a by some other
matrix then the results what can you
tell me about the column they're all
multiples of those colors right if I
multiply a by another matrix that the
product has columns that come from those
columns so can I get the identity matrix
no way the columns of the identity
matrix like one zero it's not a
combination of those columns because
those two columns lie on both lie on the
same line every combination is just
going to be on that line and I can't
one-zero so you see do you see that that
sort of column picture of the matrix not
being invertible in fact here's another
reason this is even a more important
truth well how can I say more important
although this is another way to see it a
matrix has no inverse yeah now this is
important a matrix has no a square
matrix won't have an inverse if there's
if no inverse because I can solve I can
find an X of a vector X with a times
this a times X giving zero that this is
the reason I like this that matrix won't
have an inverse can you well let me
change I to you so tell me a vector X
that solves ax equals zero I mean this
is like the key equation in mathematics
all the key equations have zero on the
right hand side so what's the X don't
tell me an X here so now I'm going to
put lip in the X that you tell me and
I'm going to get zero what X would do
that job 3 to negative 1 is that the one
you paste or yeah or another well if you
pick 0 and 0 I'm not so excited
because that would always work so it's
it's really the fact that this vector is
zero it's important it's a non zero
vector and three negative one would do
that just says three of this column
minus one of that column is the zero
color okay so now I know that a couldn't
be invertible what's the what's the
reasoning if ax is zero
suppose I multiplied by a inverse yeah
so here's the reason here this is why
this spells disaster for an inverse the
matrix can't have an inverse if some
combination of the columns gives up it
gives nothing because I could take ax
equals zero I can multiply by a inverse
and what would I discover suppose I take
that equation and I multiply by if a
inverse exists which of course I'm going
to come to the conclusion it can't
because if it existed if there wasn't a
inverse to this dopey matrix I would
multiply that equation by that inverse
and I would discover X is zero if I
multiply 8 by a inverse on the left I
get X if I multiply by a inverse on the
right I get zero so I would discover X
with zero but if X is not 2x this guy
wouldn't do there it is
3 minus 1 so conclusions only take us
some time to really work with that
conclusion our conclusion will be that
in that
non-invertible matrices singular
matrices some combinations of the
combination their columns gives a zero
problem they they take some vector X
into zero and there's no way a inverse
can recover right that's what this
equation says this equation says I take
this vector X and multiplying by a gives
zero but then when I multiply by a
inverse I can never escape from zero so
there couldn't be an a inverse we're
here okay now fix all right now let me
take a back to the positives let's take
a matrix that does head and why not
invert it okay so let me take on this
third board a matrix so I pick that up a
little
tell me a matrix that has done in well
let me say 1 3 2
what shall I put there well don't put 6
I guess is right join it any any
favorites here 1 8
I don't care seven seven okay seven is a
lucky all right okay okay so now what's
our idea we believe that this matrix is
invertible
those are like determinants have quickly
taken its determinants and found it
wasn't zero
those are like columns and probably that
that department is not totally popular
yet but those are like columns we'll
look at those two columns say hey they
point in different direction so I can
get anything now let me see what do I
mean how am I going to compute a inverse
so a inverse
here's a angel and I have to fight and
and what do I get when I do this
multiplication the idea
you know forgive me for taking two by
two but it's good to keep the
computations manageable and that the
ideas come out okay now what's the idea
I want I'm looking for this matrix
a-inverse how they're going to find
right now it's a I've got four numbers
to find I'm going to look at the first
column let me take this first column a B
what's up there what a crazy I tell me
this what equation does the first column
satisfy the first column satisfies a
times that column is 1 0 the first
column of the answer and the second
column C D satisfies a and the second
column is 0 1 you see that finding the
inverse is like solving two systems one
system when the right-hand side is 1 0
I'm just going to split it into two
pieces that I don't know I don't even
need to rewrite it I can consent pick a
times so let me put it here a Times
column J of a inverse is column J of the
identity I've got n equations ok I've
got what two in this and they have the
same matrix a but they're different
right-hand sides the right-hand sides
are just the columns of the identity
this guy in the sky and these are the
two solutions you see what I'm doing I'm
looking at that equation by columns I'm
looking at a times this column giving
that guy and a times I'd column giving
that guy so essentially so this is like
the Gauss we're back to guess we're back
to solving systems of equations but
we're solving we've got two right-hand
sides instead of one that's where Jordan
comes in so at the very beginning of the
lecture I mentioned Gauss Jordan let me
write it up again okay here's the Gauss
Jordan idea
now story is solved so two equations at
once okay let me show you how this how
the mechanics show how do I solve a
single equation so the two equations are
1 3 2 7 multiply a B give 1 0 and the
other equation is the same 1 3 2 7
multiplying C D give 0 1 ok that'll tell
me the two columns of the inverse I'll
have the inverse in other words if I can
solve with this matrix a if I can solve
with that right hand side and that right
hand side I'm inverter I've got okay and
the and Jordan sort of said two Gelles
solve them together look at the matrix
if we just solve this one I
look at one three two seven and how do I
deal with the right-hand side I stick it
on as an extra column right you remember
that's that's this augmented matrix
that's the matrix when I'm watching the
right-hand side at the same time doing
the same thing to the right side that I
do to the left so I just carry it along
as an extra color now I'm going to carry
along two extra columns and I'm going to
do whatever gelts wants right I'm going
to do elimination I'm going to get this
to be simple and this thing will turn
into the inverse this is what's coming
I'm going to do elimination steps to
make this into the identity and lo and
behold the inverse will show up here
let's do it okay so what are the
elimination steps so you see that here's
my matrix a and here's the identity like
stuck on augmented oh yeah did I Oh No
it works sorry Thanks okay thank you
very much and I've got them right okay
thanks
okay so let's do elimination all right
it's going to be simple right so I take
two of this row away from this one so
this row stays the same
and two of those come away from this
that needs to be with a zero and a one
and two of these away from this is that
what that is that what you're getting
after one elimination step let me sort
of separate the the left half from the
right half so two of that first row got
subtracted from the second row now now
this is an upper triangular form Gauss
would quit but the jordan says keiko use
elimination upwards subtract a multiple
of equation 2 from Equation 1 to get rid
of the 3 so let's go the whole way so
now I'm going to this guy is fine but
I'm doing it what do I do now what's my
final step that that produces the
inverse I multiply this by the right
number to get up to there to remove that
3 so I guess since this is a one there's
the pivot sitting there I multiply it by
3 and subtract from that so what do I
get
I'll have one zero what yeah that was my
whole point I'll multiply this by three
and subtract from that which will give
me seven and I multiply this by three
and subtract from that which gives me a
minus three
and what's my hope belief here here I
started with with a and the identity and
I ant ended up with the identity and who
that better be a inverse
that's the gauss-jordan idea start with
this long matrix double length a I
eliminate eliminate until this part is
down to I then this one will must be for
some reason and we got to find the
reason must be a ingress shall I just
check that it works let me just check
this can I multiply this matrix this
this part times a I'll carry a over here
and just do that multiplication you'll
see I'll do it the old-fashioned way
7 minus 6 is a 1 21 minus 21 is a 0
minus 2 plus 2 is a 0 minus 6 plus 7 is
a 1 chip so that is the English that's
the gauss-jordan idea
so you'll one of the homework problems
or more than once for Wednesday will ask
you to go through those steps I think
you just got to go through gauss-jordan
a couple of times but like yeah just to
see the mechanics but the important
thing is why is like what happened why
did why do we get a inverse day
let me ask you that
we got so we take week we do row
reduction we do elimination on this long
matrix AI until the first half is up
then the second half is a inverse well
how do I see that
let me put up here how I see them so
here's my here's my Gauss Jordan City
and I'm doing stuff to it so I well
whole lot of ease remember those are
those elimination mates those are the
those are the things that we figured out
last time yes that's what an elimination
step is is it's it's in matrix form I'm
multiplying by some e and the result
well so I'm multiplying by a whole bunch
of e so I get a can I call the overall
matrix e that's the elimination matrix
the product of all those little pieces
what do I mean by little pieces well
there was an elimination matrix that
subtracted two of that away from that
then with elimination matrix and
subtracted three of that away from that
I guess in this case that was all so
there were just two E's in this case one
that did this step and one that did this
step and together they give me an e that
does both steps and the net result was
to get an AA and you can tell me what
that has to be this is like the picture
of what happened if e multiplied a
whatever that is we never hear two
figures it up in five by in this way but
whatever that e times that
is e times a in what's a times a it's up
that e whatever the heck it was
multiplied a and produce I so II must be
ei equally I tells us ye namely it is at
the inverse of a great and therefore
when the second half when he multiplies
I is e what they say in verse you see
the picture looking that way e times a
is the identity it tells us what E has
to be it has to be the inverse and
therefore on the right hand side where e
where we just smartly suck on the
identity it's turning in step by step
it's turning into a there is the the
statement of Gauss Jordan elimination
that's how you find the entrance where
we look we can look at it as elimination
at solving an equation that
time and tacking on any problem solving
those equations up show the end of a
okay