Orthogonality, Gram-Schmidt Orthogonalization Process Video Lecture - Computer Science Engineering (CSE)

in this lecture we shall discuss about
orthogonality and a gram-schmidt
orthogonalization process here we shall
discuss about orthogonality of vectors
in an inner product space recall that in
the previous lecture we we have
generalized to the distance or length
concept of vectors that is we have
defined that length of a vector and
distance between two vectors so here in
this lecture we shall discuss about that
angle between vectors R that is
orthogonal T so this orthogonal itical
concept is a motivated by the law of
cosines in plane R 2 so let us see this
so here we shall discuss about this
orthogonality of vectors and this
grahame-smith orthogonalization process
orthogonalization process
so this orthogonalization concept has
been motivated by the law of cosines in
R 2 so the law of cosines in R 2 or in a
plain Euclidean plane is given by this u
minus B norm square is equal to norm of
U square but school square of norm of B
minus twice norm of U norm of P cos
theta
say this B equation 1 here the back
vectors U and V they are vectors in r2 U
and B are vectors in r2 and theta is the
angle between U and V so this norm we
have considered with respect to the
inner product in her two so the vectors
U and V are like this that if this is a
vector U and this is the vector V then
this u minus B will be this vector or in
other words and this angle theta is
angle between U and B or in other words
this is these are three sides of a
triangle and this cosine law that holds
for this triangle so this one if we
write the in terms of inner product then
this one gives that one gives that an
norm square of U minus B this vector
that is equal to inner product of u
minus B with itself and right hand side
is inner product of U with itself plus
inner product of V with itself minus
twice
norm of u norm of V cos theta so on
simplifying this
we get on simplifying on simplifying we
get minus two times inner product of U
and V that is equal to minus two times
norm of U norm of V cos theta or cos
theta is equal to inner product of U and
V divided by norm of u times norm of V
of course here U and V are nonzero
vectors that U and V are nonzero vectors
so this is that if u and V are
orthogonal that is if u and V are
orthogonal or U is perpendicular to V
then we get then value of this cos theta
that is cos PI by 2 and this is equal to
0 so this implies that this implies that
the right-hand side this inner product
of U V is equal to U and V is equal to 0
and conversely conversely if this inner
product of U and B is equal to 0 then
this value of cos theta will be equal to
0 or theta is equal to PI by 2 that is U
is orthogonal to V so this says that if
u and V are orthogonal
to each other orthogonal to each other
then this inner product is equal to zero
and the converse is also equal to zero
so it says that so on summarizing we can
write this that u inner product of U and
B that is equal to zero if and only if u
is orthogonal to V so motivated by this
concept that one defines this
orthogonality for vectors in an inner
product space so here we shall define
this orthogonality of vectors orthogonal
Atal of vectors in an inner product
space so let us take that V be let V be
an inner product space V an inner
product space then vectors vectors U and
V in V in this inner product space are
called orthogonal are called orthogonal
if this inner product of U and V is
equal to 0 we also define that
orthogonality of a set of vectors so we
say that a set s of vectors in V is
called an orthogonal set is called an
orthogonal set if every pair of vectors
every pair of distinct vectors where
pair of this
don't vectors in s are orthogonal
further we say that a orthogonal set is
orthonormal if it satisfy this condition
further an orthogonal set an orthogonal
set s is called orthonormal orthonormal
if norm of every vector is equal to 1 in
this set is if norm of V is equal to 1
for all vectors V in s so next we shall
discuss about an orthogonal basis so
earlier in case of a vector space we
have seen a basis so now we shall define
an orthogonal basis so this orthogonal
basis here we say again we consider an
inner product space so let V be an inner
product space
is it s of vectors in V is called an
orthogonal basis orthogonal basis of he
if the following conditions hold that
first condition is that s is an
orthogonal set s is an orthogonal set
and second condition is that s is a
basis for V that is instead of vectors
in an inner product product space is an
orthogonal basis if it is basis and
every pair of distinct vectors in s are
orthogonal so further we say that s is
an orthonormal basis if that Norm of
every vector in s is equal to one so
further an orthogonal basis and
orthogonal basis s is an orthonormal
orthonormal basis of V if norm of every
vector is equal to 1 so let's see few
examples of orthogonal basis so here
first example is that trivial one the
standard basis in RN this standard base
is in RN that is
this said that 1 0 0 0 1 0 0 and this 0
0 1 this is an orthonormal basis this is
an orthonormal basis for RN so second
example is like this
here we shall see that this set
consisting of vectors 1 1 minus 1 1 is
an orthogonal set is an orthogonal
orthogonal set in r2 with respect to the
standard inner product with respect to
the standard inner product so but we
shall see that if we tell this inner
product and take different one then this
vectors need not be orthogonal so it is
of course important to say that vectors
are orthogonal with respect to which
bases that with respect to which inner
product that mentioning that inner
product is also important take this
inner product of these two vectors 1 1
and minus 1 1 and this we get 1 into
minus 1 plus 1 into 1 and that is equal
to 0 so this implies that these two
vectors are orthogonal but this set is
not a North
normal set this one one minus one one is
not an orthonormal set because that norm
of this vector 1 1 that is equal to the
positive square root of inner product of
this vector with itself
and that is equal to square root of 2 so
next we can have another example that is
we consider and a different kind of
inner product in r2 so one checks that
this mapping checks that this checks
that for the vectors you say that is X 1
X 2 and V is vector y 1 y 2 in R 2 this
map mapping or this function that UV
defined like this X 1 Y 1 minus X 2 y 1
minus X 1 y 2 plus 4 X 2 y 2 is an inner
product is an inner product in R 2
so with respect to this inner product
the above set is not an orthogonal set
with respect to this inner product this
inner product this set consisting of
vectors 1 1 - 1 1 is not an orthogonal
set because here this inner product of
these two vectors is given by here we
get this 1 into minus 1 minus 1 into
minus 1 minus 1 into 1 plus 4 into 1
into 1 and here one can see that it's
value is equal to 3 that is not equal to
0 so therefore these two vectors 1 1 and
minus 1 1 is not an orthogonal set with
respect to this inner product here we
have defined so when we say that two
vectors are orthogonal with respect to
who its inner product we are saying that
is important in some inner product
vectors may be orthogonal but with
respect to some other inner product they
may not be orthogonal so next we shall
see another example
that fourth example here we consider the
inner product space consider the inner
product space that is consists of the
set of all real valued continuous
functions on this interval minus one one
so here this is the set of all real
valued continuous functions continuous
functions on this interval minus PI to
PI so V is an inner product space with
respect to the inner product to this
inner product inner product of F and G
is equal to 1 upon pi integral minus PI
to PI FX GX and DX so with respect to
this inner product we sell so that with
respect to this inner product the set of
all functions this set that is sine NX n
from 1 2 up to this infinity this is an
orthonormal set orthonormal
said so of to check this that this set
of functions signed n X n from 1 2 3 up
to infinity this is an orthonormal set
one should prove that for this one
checks that this integral value of this
integral 1 upon pi integral minus Phi 2
pi sine NX into sine M X DX this value
is equal to 1 if n is equal to M and
this is equal to 0 for n not equal to M
so the next we shall see another
important property of orthogonal vectors
or a set an orthogonal set of vectors a
set of orthogonal vectors is that they
are linearly independent so this is an
important result of this orthogonality
that every orthogonal set of every
orthogonal set of nonzero vectors
in an inner product space in an inner
product space is linearly independent it
is linearly independent so here this set
of vectors that consisting of orthogonal
vectors may be finite or infinite but in
any case this will be a linearly
independent set so let us consider let s
be a finite finite or infinite set of
non zero here we are considering
considering non zero orthogonal vectors
in the given space well here we are
considering non zero orthogonal vectors
one thing can be noticed that the vector
zero zero vector is orthogonal to every
vector and whenever this zero vector
belongs to si it that set cannot be
linearly independent therefore here we
are considering this non zero set of
vectors well to prove that this s is a
linearly independent set here we
consider a set of M vectors let V 1 V 2
to V M be a set of
M vectors in s of course this M is less
than or equal to cardinality of s and
the vectors V 1 V 2 to V M are all
distinct so let V be this linear
combination that alpha 1 V 1 plus alpha
2 V 2 plus alpha M V M where this alpha
eyes they belongs to the field for their
scalars so next for any K for any K K
lies in between 1 to M we can consider
this inner product of V with VK and this
inner product is from the property of
the inner product we can see that this
is equal to sum of this summation alpha
I VI VK inner product of V I V K I from
on to M and since this V 1 V 2 to V M
are orthogonal vectors so for I not
equal to K this inner product is equal
to 0 so therefore we get this is equal
to alpha K times this inner product of
VK with itself
this follows from orthogonality of the
vectors V 1 V 2 to V M so then since V K
is not equal to zero since this VK is a
non zero vector inner product of G K
with itself that is also not equal to
zero therefore we can have this value of
alpha K can be written as inner product
of V and V K divided by this inner
product of VK with itself or in other
words this is equal to inner product of
V and VK divided by norm square of this
vector VK so this is true for all K from
on to 2m so therefore if V is equal to 0
then alpha K is equal to 0 for all K
from 1 to 2 m hence this s is a linearly
independent set linearly independent set
so here we have some consequences some
obvious consequences that first
corollary that we have is if this set s
it is consists of say b1 b2 to VN is an
orthogonal basis
is an orthogonal basis for this inner
product space V then it is easy to
determine the coordinate of every vector
in this inner product space then for
every vector for every vector V belongs
to be the coordinates the K earth
coordinate of V is given by this inner
product of V with VK / PK norm square
that is the same thing we can write that
is if this V is equal to say alpha 1
alpha 2 to alpha n belongs to this inner
product space then this K earth
coordinate alpha K is given by inner
product of V and VK / VK norm square so
in an inner product space whenever we
have an orthogonal basis then the
coordinates can be determined by this
formula so the next we'll have another
consequence that is corollary 2 so this
says that
if we have a finite-dimensional inner
product space and if the dimension of
the inner product space be n then at the
most how many orthogonal vectors we can
have so the answer is obviously at the
most n so if V is an n-dimensional inner
product space n-dimensional inner
product space then we can have at the
most n number of orthogonal vectors we
can have at the most n number of
mutually orthogonal vectors so this is
obvious from the theorem that we have
more than n mutually orthogonal vectors
then they are they have to be linearly
independent and that contradicts to the
dimension of this vector space next we
will see another important result of
this inner product spaces is that given
any set of linearly independent vectors
we can construct orthogonal set of
vectors from those given linearly
independent vectors and that process is
called
grahame-smith orthogonalization process
so here this method is called
grahame-smith orthogonalization
process so this result we write is a
theorem so this theorem is like this we
consider an inner product space V let V
be an inner product space and V 1 V 2 to
V n be a set of linearly independent
vectors linearly independent vectors in
V then from this set of linearly
independent vectors we can construct a
set of orthogonal vectors like this then
one can construct then one can construct
a set u 1 u 2 up to this UN of
orthogonal vectors from this V 1 V 2 V n
such that search for any K
for any KK lies in between one and n
this set of vectors u 1 u 2 up to u K is
a basis for the span of that is a linear
span of the vectors V 1 V 2 of to VK or
in other words this vectors u 1 u 2 u K
they depends on the vectors V 1 V 2 2 VK
4 this is true for any K for 1 to 2 n so
here while proving this theorem we use
this gram-schmidt orthogonalization
process so we construct the vectors u1
we construct vectors u1 u2 to un in the
following way so that we write in the
step wise say first step is that we
consider this I is equal to say 1 and
second step we take this UI is equal to
say V I and third step we increase this
I is equal to I is equal to I plus 1 and
then we check that if I is greater than
n then Stav
otherwise we shall construct the vector
UI in this way this UI is equal to vi
minus summation this inner product of V
I with UJ divided by norm huges square
multiplied by this vector u J and J runs
from 1 to I minus 1 so we continue this
process that 5th step then go to third
step so continue this process that we
can construct the vectors u 1 u 2 2 u N
and this method of construction is
called grahame-smith orthogonalization
process this is our so called because
the vectors u are u 1 u 2 to u n will be
an orthogonal set so to prove this this
say T 1 u 2 2 even is an orthogonal set
first notice that this U is are not not
0
note that this UI is not equal to 0
because otherwise V I will be a linear
combination of epi is equal to 0 then VI
can be expressed as a linear combination
of vectors u 1 u 2 up to u I minus 1 and
this UI u 1 u 2 up to u I minus 1 are
expressed in terms of the vectors
be one we do to vi minus one so
therefore this if UI is equal to 0 then
V I will be a linear combination of a
linear combination of vectors V 1 V 2 up
to vi minus 1 and this is not true
because this set is a linearly
independent set V 1 V 2 2 GI is a
linearly independent set so this is not
true and hence each of this UI is a non
zero vector so next we shall check that
this u 1 u 2 next with check that this
set u 1 u 2 up to u n is an orthogonal
set so for this for example let us take
this inner product you to wait u 1 so
this u 2 is given by V 2 minus V 2 inner
product of V 2 and u 1 divided by Norm u
1 square multiplied by u 1 and u 1 so on
simplifying we get this V inner product
of V 2 with u 1 minus V 2 inner product
of V 2 you
one divided by norm a 1 square and inner
product product of u 1 with itself and
this is equal to 0 so this u 1 and u 2 u
1 and u 2 are orthogonal so in general
for any positive integer in general we
can verify this in general for any
positive integer M positive integer M
less than or equal to n and this are you
greater than or equal to 1 less than or
equal to M we have this inner product of
u m plus 1 you are so this can be
written is u n plus 1 can be expressed
as from this construction it is equal to
V M plus 1 minus summation inner product
of BN plus 1 with UJ divided by UJ norm
square times UJ j from on to m and inner
product with u R so here this
on simplifying we get that inner product
of V M plus one with you are minus
summation J equal to from J equal 2 from
1 to M V M plus 1 u J in operator P n
plus 1 he was here divided by u J norm
square is equal and this inner product
of u J with you are and this will be
equal to again from orthogonality that
this set of vectors this this evie this
we can get because it is true for all
these lower values we get this inner
product of V M plus 1 you are and - this
be M plus 1 you are because this week it
be M plus 1 you are and this is equal to
0 so this shows that the vector C 1 u 2
to u n is an orthogonal set
so this is how we construct that this
said u 1 u 2 UN is orthogonal then here
we have to prove the last part of this
theorem that for any cave they sit next
for any cave with K lies from 1 to 2 in
this u 1 u 2 to u n is a basis for this
span V 1 V 2 to V K because this e 1 u 2
to u K can be expressed as linear
combination of V 1 V 2 to V K and since
this is a linear that for any K here
this vectors u 1 u 2 to u K can be
expressed can be expressed from this
construction only construction of e 1 u
2 2 u K can be expressed as a linear
combination of can be expressed as a
linear combination of vectors V 1 V 2 to
V K and since this is linearly
independent set u 1 u 2 u K is linearly
independent linearly independent
we'll get that this set is a basis u1 u2
to UK is a basis for span of these
vectors V 1 V 2 to D K so here we can
have one remark that from a given set of
linearly independent vectors not only we
can construct orthogonal set of vectors
in fact we can construct an orthonormal
orthonormal set of vectors so here from
this set of from this set of vectors
from this set of vectors B 1 B 2 2 VK
and orthonormal set of vectors say W 1 W
2 W WK can be constructed as wi e can be
taken as UI / norm of UI so now this
norm of all these vectors u W 1 W 2 W K
will be equal to 1 and they are also
orthogonal so this will be an
orthonormal set so this suggests that
every finite dimensional inner product
spaces orthonormal basis so from here we
get this result that every finite
dimension
melina products fizz has an orthonormal
basis every finite dimensional inner
product space inner product space has an
orthonormal basis so this we can
construct from this gram-schmidt
orthogonalization process and then from
this remark we can get orthonormal
vectors so let us see one example that
applying gram-schmidt orthogonalization
process how to construct orthogonal
vectors from a given set of linearly
independent vectors so here we consider
r3 with standard with the standard basis
with the standard inner product with the
standard inner product and vectors be
like this let V 1 be the vector in r3 3
is 0 for V to be the vector minus 1 0 7
and V 3 V the vector to 911 so easily
one can see that V 1 V 2 V 3 is linearly
independent is linearly independent set
of vectors in r3 so here we shall
construct the corresponding orthogonal
set of vectors you
want you to to you three so applying
gram-schmidt orthogonalization process
we first take this you one B the vector
b1 so that is three zero four and you
two will be v2 minus inner product of V
two with u 1 divided by Norm u 1 square
multiplied by u 1 so this is equal to
minus one zero seven minus this inner
product can be calculated and it is
equal to 25 and norm of u 1 square is
also equal to 25 and he 1 is 3 0 4 on
simplifying we get this u 2 is equal to
minus 4 0 3 similarly this vector u 3
can be calculated from this grams with
orthogonalization process like this this
is equal to V 3 minus inner product of V
3 u 1 divided by norm e 1 square u 1
minus v3 u 2 this inner product divided
by norm u 2 square multiplied by u 2 and
this one can compute s 0 9 0 this vector
so now this on apply applying this
gram-schmidt orthogonalization process
we get this e 1 u 2
three that u1 u2 u3 that is the vectors
consist of 3 is 0 4 and the vector minus
4 0 3 and this vector 0 9 0 is a
orthogonal this is an orthogonal set of
vectors set of vectors in r3 with
respect to this standard inner product
in r3 so from here also one can get the
corresponding orthonormal set further
this said that you I divided by a normal
pie i from 1 2 up to 3 this set that is
norm of u 1 will be 5 3 0 4 norm of u 2
is also equal to 5 and that minus 4 0 3
and this better that 0 1 0 is the
orthonormal set of 10 from orthonormal
set of 10 from the vectors V 1 V 2 V 3
so this is how one can apply that
gram-schmidt orthogonalization process
and get a set of orthonormal
also further one can get a set of
orthonormal
vectors from the given set of linearly
independent vectors also this grams with
orthogonalization process one can apply
and test the linearly dependency or
independency of vectors suppose the
given set of vectors V 1 V 2 to V n are
given and they are not known whether
they are linearly independent then we
consider the non zero vectors V 1 V 2 to
V N and then apply this
orthogonalization process then at some
step we get the sum P Y will be equal to
0 then the set of vectors V 1 V 2 to V
and the difference set will be linearly
dependent otherwise we will able to
construct an orthogonal set of nonzero
vectors V 1 u 2 to even okay that's all
for the lecture thank you
you