Correlation Structure - Electronics and Communication Engineering Video Lecture - Electronics and Communication Engineering (ECE) | Best Video for Electronics and Communication Engineering (ECE)

So, yesterday we are discussing the Hermitian
matrices and then we discussed, what is
called auto covariance matrix auto correlation
matrix and all that. So, let us start from
there briefly to recall, what is a Hermitian
transposition? That is conjugate transposition.
For say a real matrices a real vector it is
simple transposition, but for complex value
matrices, it is conjugate transposition. And
then if a matrix a matrix you know vector
is a
special case of matrix is m cross n any one
cross m matrix is a vector. So, I may talk
in
terms of matrix, but that includes vectors
also.
Any matrix A if A H is A then A is Hermitian
A Hermitian. So, you understand for this
to be true matrix has to be square A Hermitian
means you are taking transpose and
original matrix; if they are same means dimensionally
they have to match, there will be
number of row and number of column of A have
to be same, which means the square
matrix that goes without saying. Then, we
discussed then look we said that look suppose
we consider a vector x random vector means
it consist of say p number of random
variables; any vector or scalar under indicated
by me by an underscore.
Normally, upper case letters with underscore
lower case letter with underscore for vector
upper case letters with underscore matrices.
Sometimes, you know by mistake I may skip
I might just you know you either tell me or
you assume at it is as it is. Because always
I
may not be able to stick to this underscoring
theory you know. Sometime, I may the just
skip unwittingly. Then, the mean of this is
there is a mean vector E of x E of x means
a
vector like this, where you have got E of
x 1 dot dot dot. Say, E of x p we can even
write
it as say mu 1 yesterday I call mu x 1 yesterday
I call mu x 1.
Let me, call it mu 1 because there is no other
say mu p. Then I said that is you subtract
if
you consider, these vector say x tilde as
x minus mu x vector it will give only the
deviation across the mean. Then, if you take
if you take this E of x tilde x tilde Hermitian
I called it auto correlation auto covariance
matrix; C you can even indicate c p cross
to
indicate the matrix. What does this matrix
gives today I will not expand yesterday I
expanded. Just for you consumption you remember
one thing suppose I give a vector
forget about all these.
Suppose, I give a vector say u as u 1 dot
dot dot u p and I give another vector same
length v v 1 dot dot dot v p. And if you multiply
u with v Hermitian; what you get
column vector u v Hermitian means it becomes
row vector with conjugate. If you take
this matrix and its general element i j th
element what will that be. See, how the matrix
is
formed u one into v 1 star first element u
1into v 2 star u 1 dot dot dot v p star that
is a
first row; i th row means u i u i into v 1
star u i into v 2 star. So, j th element means
u i
into v j star u i here i th element from here
j th element from here this is.
So, what is general element i j th element
of course, i and j remaining within 1 to p;
this
is simply i th element of here j th element
of a conjugate with expectation. So, that
mean
x what x tilde vector xi; I mean it is a component
xi minus mu i. There is a i th
component here of this vector and j th component
of that x tilde minus mu x tilde j minus
mu j star. No tilde; x j minus mu j x j minus
mu j star that is coming from the tilde vector
x j minus mu j star. So; obviously, this is
the covariance between covariance between
x i
and x j. If this is zero then x i and x j
are uncorrelated all those things you have
seen that
I am coming separately that these matrix we
have seen yesterday.
Hermitian that you can see directly I told
you yesterday, but I repeat again here you
can
see c p p i j is this. What is c? I am just
trying various ways because we will be doing
this
kind of exercise too often. So, I am just
making you familiar. What is the j i th element
E
of this will come first x j minus mu j and
so obviously, this is conjugate of this. So,
i j th
element and j i th they are same, but conjugate
of each other. So, it is conjugate
symmetry this why you can say this Hermitian.
Even, otherwise you say as I told
yesterday you do not have to do all these
by simply algebra if you take Hermitian of
this
matrix Hermitian operation you take Hermitian
of this.
You can push Hermitian operation inside expectation
operation because expectation
means what any term is multiplied by some
appropriate probability density integrated;
so, that will be replaced from i j th to j
i th it will go. So, you better if move it
fast and
then take expectation rather than taking expectation
first they move and also a
conjugation, but conjugation after multiplying
by the probability and integrating is same
as you better first conjugate the variable
and then integrate multiplied by p x because
the
probability is a real function. So, that way
these are the mathematical things you know
inside.
So, you can push Hermitian operation instead
of v operation and if you do that if you
take this polar x Hermitian x x tilde x tilde
Hermitian Hermitian of that; first second
term
will come first Hermitian Hermitian will cancels
you get back x tilde. First term, will go
second with Hermitian x tilde Hermitian, which
get back the same thing and that prove
that it is Hermitian. If mu x if you take
only this vector this much only x no subtraction
of mu no subtraction then it is called correlation
matrix. R matrix p cross p correlation
matrix this two become same this also Hermitian
you can easily see by the structures
here.
Expectation of something into something transfer
Hermitian and this becomes same as
this when your mu x is zero for zero mean
process is zero mean random variables
correlation covariant are same. As a special
case, now consider this that supposes there
is
a discrete time random process x n.
So, x n x n minus 1 x n minus 2 that is now
this is given. Stationary or not I am not
speaking so far. I am just taking some samples.
Please note all these things carefully
because later in the analysis of adaptive
filter; I will make use of all these results
frequently. So, I will not repeat rewrite
this. So, actually p plus 1 element have taken
in
n minus 0 n minus 1 up to n minus p. So, it
is not p p plus 1. Suppose, I form a vector
like this I call it x. I can if I take this
E x E x yes E x and x Hermitian; this will
be a
correlation matrix and if mean of this samples
is zero, then there will be covariance
matrix also.
What it will look like now I write it in expanded
form first term will be mod x l square x
n into x n then will be these are very important
matrix. So, that is why I am doing this
way x n into x star n minus one dot dot dot
E x n x star n minus p. Next, row will be
x
what you cannot see this is a conjugate of
this, you can otherwise you can take because
of the Hermitian property. This conjugate
will come here n dot dot dot. So, for there
is
something special this just a correlation
matrix you only in this earlier it was giving
general variables x 1 x 2; now the specific
terms x n x n minus one x n minus two so and
so forth.
So, in terms of those or other in terms of
the correspond correlation this matrix elements
will be obtained. That is all that is only,
but now suppose and this is always Hermitian.
Now, suppose I say this x n also is wide sense
stationary; that means, correlation is
dependent only on the gap. So, then it takes
a very very interesting structure. Firstly,
you
see even if not stationary this matrix if
I know one half diagonal and upper half lower
half can constructed because on the Hermitian
property. Conjugate transpose, one half is
enough for me upper triangular part with the
diagonal or trigonal for the diagonal. Now,
you see if there is stationary if stationary
if WSS is this leads to what and I write
separately here I define E of x n x star n
minus k correlation it is equal to r; I should
write r x x, but for the time being I am dropping
x x because I am handling only one
random process; so no point in writing too
much.
So, you can easily understand what I am referring
to so this is r k this is the definition
with this definition in mind. What is in matrix?
First term will be r 0, which is expected
instantaneous power r zero, then will be please
see this what is this r 1 then r 2 dot dot
dot r p and obviously, this will be r star
1 Hermitian property. Then, again r zero then
again I am not shown it here what will be
the x 1 r 1 then r 2 dot dot dot r p minus
1. You
will see along this diagonal r 0 will propagate
because of stationary T, where there is E
of mod x square E mod x n minus 1 square or
E of mod r 0.
Similarly, along this sub diagonal; r 1 will
propagate along this sub diagonal; r 2 will
propagate so on and so forth. On this side,
whatever be the element that also you
propagate, this kind of matrices are called
toeplitz matrix; this a special case you have
a
semi this Hermitian toeplitz. In the case
of toeplitz this may not be r star 1 it could
be
anybody. So, a a a this next one could be
b b b b that is the general toeplitz matrix,
but
this is totally toeplitz. This is why another
structure the Hermitian that this element
and
this element are conjugate of each other.
This guy and this guy are conjugate of each
other there is an extra thing, which is a
Hermitian property.
So, this is I am writing Hermitian; this matrix
place a very key role in all signal
processing application statistical signal
processing adaptive signal processing; linear
prediction that is filters, because we often
handle stationary processing, where model
the
model the process to be stationary. Generally,
we proceed further immediately the
correlation matrix we get that is a toeplitz
matrix. This matrix has beautiful property
that
in the sense and if you know one row or one
column entire matrix is known you can
construct; so that means, this matrix has
lot of redundancy, which means if you are
solving an equation a x equal to v and a is
matrix of this kind.
Very first algorithms can be developed, because
there is too many too much of
redundancy in a in the matrix elements. Those
algorithms exist very powerful
algorithms, which come in various context
of signal processing involving stationary
processes. This is a Hermitian toeplitz matrix,
this special case I have discussed. Again,
I
come back to general Hermitian matrices; general,
Hermitian matrices of which
correlation of covariance matrix are some
examples.
Suppose, not H suppose T Hermitian matrix
T you call p cross p. I am writing p cross
p,
but when I go to the adaptive filter part;
it will become p plus 1 into p plus 1 you
know
like it become last time. Simply, because
last time if I have taken if I had taken form
x n
to x n minus p if I wanted to make it length
p and not p plus 1; I would have to put n
minus p plus 1. So, again everywhere you know
n minus p plus 1 writing too much. So, I
better make it p plus 1 and go up to n minus
p just for my writing convenience writing
nothing else.
Suppose, it is the Hermitian matrix in generally
p cross p Hermitian matrix complex
value T in general; Hermitian means it implies
T Hermitian is T. I am skipping this
underscore often please excuse me. Then first
consider is Eigen value Eigen vector
problem, suppose there is you are solving
this equation. You know Eigen vector Eigen
vector product that firstly, I this vector
x must be non-zero. This is this equation
is valid
for any zero vector. So, you cannot find out
a lambda. Eigen vectors are by need
definition non- zero vectors and you are equation
like this. What is the meaning of Eigen
vector? It is such a vector that when you
multiplied by T.
You do not get any arbitrary vector; you can
make the same guy. Just scaled up a down
by of element called lambda, there is in a
geometrical sense it was pointing in rejection
after you operate it. There are many operation
of vector in a rotation and translation, but
in such an operation that when you operate
on that particular vector not for any vector
on
that particular vector. That is given an operation
it has got such specific vectors called
Eigen vectors. We are taking one Eigen vector,
if you operate that Eigen vector within
that operator. It will point it will dissolve
in a vector pointing in the same direction
its
magnitude will change.
It will change by a vector lambda could be
complex in generally, this called Eigen
vector, Eigen value problem and x cannot be
zero. Suppose, T is Hermitian now let us
derive some interesting property and very
simple properties very very simply derived.
Suppose, I take Hermitian of both side; I
can do that. So, if I take Hermitian of left
hand
side what will I get; x Hermitian T Hermitian
and T Hermitian is T because T is the
Hermitian matrix. So that means, x Hermitian
T and Hermitian of right hand side means
lambda star. You will take conjugate every
element is multiplied by lambda you will take
conjugate and transpose lambda will be replaced
by lambda star.
There again pull out lambda lambda star. So,
it will become lambda star x H. So, x
Hermitian is a row vector multiplied by a
matrix. So, real thing will be a row vector.
The
row vector if I now multiply here, this is
a row vector; if I now bring back x here then
row into column it will be a scalar. And lambda
star, but T x is lambda x; see this matrix
multiplication is commutative or distribute
associative it is. So, you are multiplying
this
with this and then result with this you can
also do this with this followed by this with
this. But T x is lambda x. So, bring that
lambda x here lambda is a scalar take it out.
So, you get back lambda x Hermitian x, on
this side lambda star x Hermitian x. So, now,
this result in this thing lambda minus lambda
stars equal to zero. I am not showing see
they are simple very silly very simple steps.
Now, x is a non-zero vector what is x
Hermitian x like you know I mean if x is of
this kind; x 1 dot dot dot x p. What is x?
Hermitian x summation mod x i square i from
1 to p; this is always real non- negative,
if
it is zero every term has to be zero because
every term is mod square term. If the entire
summation has to be zero every term has to
be zero. Every term has to be zero mean each
x i is zero, which means x is a zero vector,
but that is ruled out because x is an Eigen
vector x cannot be zero vector.
So that means, this is always greater than
0. So, in this equation then this is equal
to zero
this implied, because this cannot be zero.
This implies lambda equal to lambda star,
which means lambda is a real value. So, even
if matrices Hermitian and complex valued
Hermitian matrix Eigen vectors are in general
complex values. Eigen values are all real
of a Hermitian matrix. So, for any correlation
matrix if you find out Eigen value there
were real.
Another property, suppose I have got two distinct
Eigen values T x 1 as say lambda 1 x 1
T x 2 lambda 2 x 2 and lambda 1 and lambda
2 they are not same two distinct Eigen
values. Then, I can do one thing on this matrix
just a minute; you take the Hermitian of
both side of the first equation. So, you get
x 1 Hermitian T Hermitian, which is T and
on
this side lambda 1 star lambda 1 star x x
1 H, but I have proved lambda is real. So
that
means, this is equal to simply lambda 1 this
is this is reason, we have proven that. This
now on this side I multiplied by x 2 so result
are multiplied by x 2 here also.
Now, T x 2 is from the second equation lambda
2 x 2, so take out take that out. So, this
will lead to this lambda 2 minus lambda 1,
but I told you I am taking distinct Eigen
values lambda 1 and lambda 2 are not same.
So, this cannot be zero; that means, this
is
equal to zero. So, this is zero; that means,
we say there orthogonal; two vectors in our
definition if they satisfy this kind of theory
you know to vectors x and y. If you do the
x
Hermitian y or y Hermitian x x Hermitian y
then is zero that will mean y Hermitian x
also zero; it will be the Hermitian of both
side there, then x and y are called orthogonal.
Actually, you have done dot product of vectors
is a three dimensional wall. So, p vector
and q vector p dot q p suppose was p 1 p 2
p 3 q say q 1 q 2 q 3, what is p dot q p 1
plus
q 1 plus q 2 p 2 q 2 plus; that is p transpose
this p 1 p 2 p 3 transpose of this into q
1 q 2
q 3. That was the dot product, but if they
are complex value you can generalize. These
are generalized definition of a dot product,
where there are not only transpose there
conjugate transpose. Then, take these that
is the dot product between two vector p and
q;
in a generally algebraic sense not in the
three dimensional wall because there is nothing
complex there.
You can generalize from three dimensions to
any dimension. So, there is a given two
vectors x and y x Hermitian y is a dot product
and dot product, there is called inner
product. This is more general term than dot
product inner product; I will discuss all
this
later at length, when we come to vector space
theory is dot product and dot product zero
means orthogonal. Like p and q, if there are
dot product means that is orthogonal, but
there is a geometric sense that directly you
can see visible geometrics, which I would
say.
Geometry is there everywhere, whether is a
visual geometric as you can see the angle
is
ninety degree and all those things. So, x
1 Hermitian x 2 0; now supposed to start with
T
is such a matrix p cross q, I have matrix
how many Eigen values it can have at the most
p
Eigen value; p cross p matrix you solve the
characteristic u s and you can have at the
most p roots and for the time being assume
that all the p roots are distinct. When not
distinct what appears I will just tell few
lines I will let prove, but that is heart
of the
linear algebra thing you could just take down
my I mean take down what I say, but just
to start with assume all the p roots are distinct.
So, I have got lambda 1 up to lambda p all
real and I have got these equations T e k
as
lambda k e k; lambda k is the k th Eigen value
given, lambda one not equal to lambda
two distinct not equal to dot dot dot not
equal to lambda p. These are case I am handling
first so in that case can I write here directly.
Suppose, I make a I just put the Eigen vector
side by side e p, what will the this be T
into
e one will be the first column; T into e two
will be the second column so on and so forth.
And T e 1 is lambda 1 e 1 lambda 2 e 2 these
are all column vectors. These are all
column vectors now my claim is I can write
it like this. The same vector if I bring in
e 1
e 2 dot dot dot e p into lambda 1 lambda 2
dot dot dot lambda p a diagonal matrix. These
will give this. No, can you see that?
How do you do when you do matrix into matrix
multiplication how do you do? Suppose,
A and B are two matrices I come back to some
rough work done for you benefit; I am try
to give you some extra things regarding you
know basic matrix via operations.
Suppose, you are given one matrix A, say element
like a 1 1 a 1 2 a 2 1 a 2 2; another B I
am taking two by two from that you will see
what I trying to mean B 1 1 b 1 2 b 2 1 b
2 1
b 2 2 and you have to do A into B. How will
we do normally what you do? Tell me. You
do like this a 1 1 into b 1 plus a 1 into
b 2 1 term a 1 1 into like that. So, from
to today
class you should never do that, which is not
the way we should do that is done in the
school level and all that level not any one
now. My claim is if you call this column is
a
column vector a 1 first column a 2. Then,
this will be a matrix again two column; first
column will be this column vector times b
1 2 scalar plus second column times b 2 1.
First series, if you take this column b 1
1 times this b 1 1 into this b 1 1 into this
and take
this column b 2 1 into the this b 2 1 into
this add do you get back the same quantity.
Can
you going to see that, b 1 1 times this column
vector. So, first element a 1 1 b 1 1 stored,
what is this a 2 1 b 1 1 stored and then this
guy into second column, b 2 1 a 1 2 stored
on
top; b 2 1 a 2 2 stored and I you are adding.
So, you get back the this. That thing a 1
1
into b 1 1 plus c 1 into b 2 2 b 2 1 are you
convinced or not. Similarly, here you call
now
combined with this b 1 2 into again a 1 plus
b 2 2 a 2.
So, this you may forget about matrix if it
is a vector only for any column vector matrix
terms column vector. What you are doing? You
are taking the first element multiplying
first column with that. Second element multiplying
the second column with that likes
that and then adding; that will be the net
thing that is this matrix into that vector.
Again
these matrixes into this vector do accordingly;
so that means, you linearly combining this
column vectors by the elements of this column
here. Linear combinations of this
columns by the elements here that will give
you here take you here. Again take that next
fellow, linearly combine the columns b 1 2
times this column b this time this column
if
there are more columns this into third into
third fourth into fourth.
Then, add that will be the corresponding resulting
column vector. Are you convenience?
Now, come to this. I had please have a look,
I had lambda one scalar times e 1 lambda 2
scalar time e 2 dot dot dot and say either
can write like this. Very simply first this
by this
is a matrix first column second column p th
column this is a matrix. That times say first
column here that will gives rise to the first
column of the resulting product matrix that
should turn out to be this, but that is obvious
lambda 1 time e 1 other fellows are 0. So,
they are left out lambda one into e 1 comes.
See, how easily it comes if you know that
then here you have zeros zero into e 1 forget
it.
Other zeros forget only e two picked up by
lambda two that comes here so on and so
forth. All the Eigen values are real you give
a name to this matrix D, diagonal matrix D.
You give a name to this matrix say E and this
matrix E. So that means, I can write T E is
E D. What are these columns that the Eigen
vectors corresponding to distinct Eigen
values. Distinct Eigen values this is for
lambda 1; this is for lambda 2, that is how
these
vectors are obtained and you do T into this
you get this. So, there are mutually
orthogonal e 1 Hermitian e 2 0 e 1 Hermitian
e 3 0 only Hermitian with itself will not
be
zero. Because the vector is not 0, I told
you.
So, Hermitian with itself will be that summation
of mod square terms, which cannot be
zero because an every term has to be zero,
which is a contradiction. So, all cross terms
will be zero. So, if I say if you take E matrix
if you take E Hermitian into E if you taking
Hermitian, what will happen you have to take
conjugate transpose of this matrix? There
first column will become first row transposition
and then conjugate of that. So, here you
will have e 1 Hermitian this guy becoming
row if you take this first column that will
become row and every element conjugate it.
So, what it is that actually that row is nothing,
but original e 1 the Hermitian of that; then
e 2 Hermitian of that dot dot dot e p Hermitian
of that that is for E H. E is e 1e 2 as
before e p nephew to the product; row vector
column vector row vector column vector.
So, what do you see here? It will be diagonal
matrix; e 1 Hermitian with e 1 only nonzero
e 1 Hermitian with e 2 2 e p 0. Similarly,
e 2 Hermitian with e 1 0 with e 2 nonzero,
but others zero; so it would be a diagonal
matrix. Here, I forgot to do one thing. So,
I come back when I was solving this equations
you know T e equal to lambda e this kind
of equation.
You got lambda 1 upon lambda p and e 1 to
e p; whenever you get a vector say e k e k
Hermitian with e k; it is dot product with
itself. It is called actually now a term norm
square in our vector algebraic term, it is
a square of the length of that vector a norm
square. This always positive we have shown
because there is none of them is a zero
vector. So, if you call this quantity as norm
square is the scalar terms you give it a name
say theta theta k; then instead of e k if
I take e k by square root positive square
root of
theta k. If I take a vector like, this also
is an Eigen vector with Eigen value lambda
T of
this. There is a dividing I mean multiplying
both side by one by square root theta k.
Here also and that you can pull inside below
e here also below e. So, you can this you
can call a new vector say ek prime I am saying
T e k prime also lambda e k prime first
thing. You must know, if any ax equal to lambda
x x is an Eigen vector then two x three
x four all are Eigen vectors for the same
Eigen value scalar multiples. So, this, but
what
is the advantage of this e k prime is this
what is norm square of e k prime? e k prime
with
these are scalar and what kind of scalar positive
real. Do you cannot conjugate it? e k of
Hermitian means when you take the Hermitian
of it.
This will become row vector with conjugate.
So, this will get Hermitian transposed, but
this only remains as it is, because this real
fellow. So, that will become an from this
e k
prime from e k prime also another square root
theta k will come. So, it will become theta
k square root will go and then e k Hermitian
e k and e k Hermitian e k itself is theta
k; so
theta k by theta k which is equal to 1. So,
this call normalization you can always find
out
the norm square of that take this positive
square root of that divide the vector by that.
That vector also is an Eigen vector, but its
norm square is unity.
I will forget to mention that when I am dealing
with this e 1 to e p. I normalized each to
have norm equal to 1. So, e 1 Hermitian e
1 is 1 e 2 Hermitian e 2 is 1 so and so forth.
So, in that case this matrix becomes simply
identity matrix e 1 Hermitian e 1 all are
zero;
e 2 Hermitian e 2 second diagonal entry that
is 1 others 0. So, it becomes 1 zero zero
dot
dot zero zero one zero zero dot dot zero.
Any matrix and that has to be square matrix
because otherwise this product cannot take
place. Any matrix may be may not be
Hermitian because e is a is not a Hermitian
matrix.
Here, the E there is not a Hermitian matrix
any matrix says E; if it is satisfied property
that E Hermitian E is identity, then this
matrix is called unitary matrix. And E Hermitian
E is identity; that means, E Hermitian is
nothing, but equivalent to E inverse; if I
need a
square root two square matrix is A and B and
A B equal to y you know B is A inverse
and A is B inverse that we know. So, that
means E Hermitian to inverse such a matrix
you really do not have to do a enough computation
that in inverting matrix it is a big
problem. If a matrix is given to be unitary
you just take a Hermitian transposition
immediately get its inverse.
Therefore, this also implies that instead
of E H E if I make it E E H that also is identity,
where E H is inverse, so that means, T E is
E D and E is unitary T is unitary. Here, I
am
writing the separate part from this equation,
what I found TE is equal to ED and E is
unitary D is a diagonal matrix of real diagonal
values. So, if I now take this E square
matrix T square E square and bring another
matrix here E H here also E D E H, but E E
H cancels I have told you unitary as well
EH is the universe of p. So that means, T
is this
is the main result T can be written in this
kind of form E D E H, where E consist of the
Eigen vectors normalize to unity.
D consist of the corresponding Eigen values
and this is the Hermitian transposition of
that E consider Eigen vectors and E is a unitary
matrix here. E is a unitary matrix. Now,
you can ask me that I assumed this thing that
lambda 1 to lambda p are distinct and
therefore, e 1 e 2 e p; they are mutually
orthogonal, but if lambda 1 to lambda p are
not
distinct. Suppose, lambda 1 and lambda 2 are
same. So, e 1 e 2, where from you get and
are they orthogonal because the correspondence
same Eigen value lambda 1 equal to
lambda 2.
Then, at least between these two I cannot
have this kind of thing you k now that cross
terms canceling on inner product with itself
remaining one. So, diagonal matrix is
formed that kind of that property would not
hold good. They are you have to just here
me
out that in such cases for Hermitian matrix
T. The other way out mother matrix is
Hermitian matrix and finding an Eigen value
vectors and Eigen values of Hermitian
matrix only. So, for Hermitian matrix if suppose
an Eigen values repeated; suppose e 1
suppose lambda 1 equal to lambda 2 equal to
say some lambda. Then, given this lambda
you can find out two Eigen vectors if it is
repeated twice, you can find out two Eigen
vectors, which are mutually orthogonal.
That you put here, they will be orthogonal
with the rest, because there Eigen values
at
distinct from this lambda. They are mutually
orthogonal to this; I would able to prove
now with that will required lot of things
whether linearly lot of things. So, remember
given at Hermitian matrix T, whose examples
could be correlation or covariant matrix
not necessary stationary; if that is of stationary
and extra structure comes of top linear.
I am not taking any advantage of top of these
property. I am only seeking Hermitian
matrix, I even took a general Hermitian matrix.
That can be written as; this is what we have
proved. This kind of transpose, you know
transformation E D in fact, E inverse in general
here; because it is T Hermitian E here
Hermitian here are the Hermitian E is a unitary
matrix. So, E Hermitian is a universe of p
all those things nice properties are there.
But in general, if you have got structure
like
this that you take one matrix say A; then
some diagonal matrix D I am talking of a very
general case I am give an extra input to you.
A some matrix not unitary nothing just A
into a diagonal matrix D into A inverse A
is invertible that is only thing I am assuming
A
into D into A inverse.
You call the product as B, then B and A they
are solve they are similarly related. I mean
this is called similarity transformation on
B. In such cases that is suppose B is in general
A D A inverse, whose special case is this,
because A here is E; A inverse; obviously,
E
H; D is the diagonal matrix of Eigen and T
is its stands for B. In this case if you take
determinant of this matrix B what you get
you know determinant of a product is able
to
determinant of the individual matrices. So,
determinant of A into determinant of D into
determinant if A inverse.
Now, determinant A inverse that is one by
determinant of A do you know all these. So,
determinant of A and one by determinant of
A they will cancel. So, determinant of B will
be simply product I mean this determinant
of D and what is determinant of D is
diagonal? So, simply product of the diagonal
elements in this case Eigen values. So,
determinant of a Hermitian matrix is that
is a the product of the Eigen values. See,
if all
the Eigen values are non-zero; then determinant
is non-zero it is invertible because
determinant is non-zero. On the other hand
if any Eigen value happens to be zero. Then
that matrix B is not invertible. So, this
is a product of this is a product sign i equal
to one
two p.
Please understand these are the core of statistical
signal processing we need this often.
Every now and then I want you to be absolutely
conversant with this properties. So, there
is no scope of re thinking how this how that
all that. We find this extremely useful in
communication controls single processing this
three subjects, lambda i in this case
lambda is real and all that so we get a real
product pi. So, it will just product of the
Eigen
values that is one thing. Another thing, do
you know this that suppose A and B are two
square matrices two say m cross m matrices.
You know the trace of a square matrix, what
is it mean trace of a square matrix is
summation of other diagonal elements. Then
do you know this trace of A B and trace of
B A they are same. You may not know. So, we
just it is not difficult. What is trace of
AB
if I the find out the diagonal entries? What
are the diagonal entries? A B is a product
what is the i th diagonal entry i th diagonal
entry is A i i th row of this guy. So, a i
j and b
j i j as to be move from 1 to p; there is
the i th A B product I am talking of the AB
product. I further the product matrix what
is the I j th entry.
So that means, i th row of A j th column of
B i th row of A means a i 1 a i 2 a i 3 dot
dot
and they are multiplied by j th column. No,
i th diagonal i comma i comma I may wrong
statement. I am bothered about the diagonal
element only. So, the product matrix not any
arbitrary I j i comma i th element, i comma
i th element so that means, I start with i
th
row and i th column i th row of A; i th column
of B, so i comma 1 1 comma i i comma 2
2 comma i. So, that will be the i comma i
th element i th diagonal entry and now I have
to sum all the diagonal entries; so then final
summation over 1 to p.
This you get on the left hand side, but I
told you there are finite sum. So, you can
interchange the any DSP also you know the
next step is there is not DSP, but often
double summation means you interchange the
if you interchanged the two, the result
comes immediately b j i I write like this
a i j. So, j a th row of B and j th column
of A
they are multiplied and summed over all j.
So, this is what does it imply here? In general,
A D A in I again I am try to give in more
general case, A D A inverse instead of here
instead of here whatever is true for this
that is true for this do you agree is it.
So, I am
that is why taking of this. No, for us this
is important not that.
So, A into D is a square matrix. So, you break
it like this trace of this into this is simply
trace of A inverse into A D. Trace of this
part into this part is same as trace of these
goes
first and this go second, but you see A inverse
and A will cancel. So, simply trace of D
this is result you do not forget, I make use
of this result many times. So, Hermitian
matrix its trace will be same simply summation
of the Eigen values. So that means, trace
of T is summation of lambda i’s. Please
remember this, I will deal with trace of
correlation matrix auto correlation matrix,
so that time I simply summed Eigen values
do
not ask a question then get conversant with
all this. Now, I come to a special kind of
Hermitian matrices that is called positive
definite matrix. Have you heard of this terms
positive definite matrices? May be in way
I mean some communication course or some
course positive definite it may.
So, T positive it can be positive definite
or sometimes positive semi definite, positive
semi definite is also called non-negative
definite, positive definite or case is non-negative
definite. Non negative definite, which is
also called positive semi definite, there
are
various ways people write you know. So, this
case this second half this is more general
I
mean, I am considering on this and whatever
I say I will just modify a little, so that
will
apply here. In either case T first be a Hermitian
if not Hermitian then we cannot do
anything; if not Hermitian then we cannot
be anything. So, T first be Hermitian T must
be Hermitian. Secondly so Hermitian we have
already studied. So, we have there has to
be something more.
So, what is that for you know for this sign
for all? All x where x is a non-zero vectors
this is important. If you take this product
x Hermitian R x; R x is some vector y x
Hermitian that thing this must be real and
greater than zero, if positive definite and
greater than equal to zero if non-negative
definite, positive definite or non-negative
definite also called positive semi definite.
If this extra property holds, then it is called
our
correlation matrices or covariance matrices
they are not only Hermitian they are positive
definite; always non-negative definite, but
we model them as we can also take them as
positive definite unit under some assumption,
which is in general valid.
This is T, this is T this not R this T, because
I started with T; I always have in mind
correlation matrix are some pre are, x transposed
T x.
In such case before I wind up
within one or two minute just one property
you see. T is Hermitian right T is Hermitian
now again come to that Eigen value problem.
Suppose, x is an Eigen value Eigen vector
of Eigen value lambda for T; lambda is real
we have seen and x is a particular Eigen
vector non-zero Eigen vector, where x cannot
be zero. X is an Eigen vector x cannot be
zero say x is a nonzero vector. So, I told
for all x where x is non-zero.
So, now I pick up x itself multiply here,
then I can I have to multiply this also by
x H,
but x is a non-zero vector. So, then this
must be real which is of course, the case
because
this becomes lambda into x Hermitian x and
x Hermitian x is always real. But, if it is
positive definite suppose T is given to be
positive definite or non-negative definite
since
this Hermitian by its definition I took T
x equal to lambda x lambda real, then I
multiplied by x H here also x H x is a non-zero
vector, which is again vector on the right
side I get lambda into x Hermitian x, but
by since it is also positive definite.
This entire thing since x is non-zero vector
this product must be real really it is coming
in
any case, because this is real Eigen value
is real, but this must also be greater than
zero
or greater than equal to zero. Now, out of
this this is strictly greater than zero. This
is
norm square strictly greater than zero; that
means, lambda the simplest lambda greater
than zero or lambda greater than equal to
zero. So, if it is positive definite Eigen
values
are not only real they are all positive. If
it is positive semi definite or non-negative
definite Eigen values are not only real they
are non-negative, but non-negative means
zero can included, which means matrix T cannot
be may not be invertible, because that
determinant, which is a product of the Eigen
value.
Some Eigen value can still be zero. So, that
is why we will often make as an assumption
by which T will be your this thing positive
definite. So, I wind up today. So, this week
we are not meeting anymore because we have
taken three classes. So, next time we meet
that is on Monday three thirty.
Thank you very much.