Inner Product Spaces - Matrix Algebra, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET PDF Download

 Page 1


Inner Product Spaces
1. Preliminaries
An inner product space is a vector space V along with a function h,i called an inner product which
associates each pair of vectors u,v with a scalarhu,vi, and which satis?es:
(1) hu,ui= 0 with equality if and only if u =0
(2) hu,vi =hv,ui and
(3) hau+v,wi =ahu,wi+hv,wi
Combining (2) and (3), we also havehu,av+wi =ahu,vi+hu,wi. Condition (1) is called positive de?nite,
condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner
product is an example of a positive de?nite, symmetric bilinear function or form on the vector space V.
De?nition 1.0.1. Let V be an inner product space and u and v be vectors in V. We make the following
de?nitions:
(1) The length or norm of the vector u is:
||u|| =
p
hu,ui
(2) The distance between u and v is:
||u-v||
(3) The angle between u and v is:
? = cos
-1

hu,vi
||u||·||v||

(4) We say that u and v are orthogonal if
hu,vi = 0
(5) The orthogonal projection of u onto the space spanned by v is:
p =

hu,vi
hv,vi

v
Note that, a priori, we do not know that-1=
hu,vi
||u||·||v||
= 1 so ? may not be de?ned. Later we will show that
these bounds are valid and so our de?nition of ? is also valid. When referring to (5), we will usually say “the
projection of u onto v”.
Now we give some preliminary results:
Theorem 1.0.2 (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then
||u±v||
2
=||u||
2
+||v||
2
Page 2


Inner Product Spaces
1. Preliminaries
An inner product space is a vector space V along with a function h,i called an inner product which
associates each pair of vectors u,v with a scalarhu,vi, and which satis?es:
(1) hu,ui= 0 with equality if and only if u =0
(2) hu,vi =hv,ui and
(3) hau+v,wi =ahu,wi+hv,wi
Combining (2) and (3), we also havehu,av+wi =ahu,vi+hu,wi. Condition (1) is called positive de?nite,
condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner
product is an example of a positive de?nite, symmetric bilinear function or form on the vector space V.
De?nition 1.0.1. Let V be an inner product space and u and v be vectors in V. We make the following
de?nitions:
(1) The length or norm of the vector u is:
||u|| =
p
hu,ui
(2) The distance between u and v is:
||u-v||
(3) The angle between u and v is:
? = cos
-1

hu,vi
||u||·||v||

(4) We say that u and v are orthogonal if
hu,vi = 0
(5) The orthogonal projection of u onto the space spanned by v is:
p =

hu,vi
hv,vi

v
Note that, a priori, we do not know that-1=
hu,vi
||u||·||v||
= 1 so ? may not be de?ned. Later we will show that
these bounds are valid and so our de?nition of ? is also valid. When referring to (5), we will usually say “the
projection of u onto v”.
Now we give some preliminary results:
Theorem 1.0.2 (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then
||u±v||
2
=||u||
2
+||v||
2
Proof: We’ll just do the case ||u+v||
2
as the argument for the other case is similar. Note, as u and v are
orthogonal, hu,vi = 0. Now:
||u+v||
2
= hu+v,u+vi
= hu,ui+hu,vi+hv,ui+hv,vi
= hu,ui+hv,vi
= ||u||
2
+||v||
2

Theorem 1.0.3. Suppose that p is the orthogonal projection of u onto the space spanned by v. Then p and
u-p are orthogonal.
Proof: Recall that the projection of u onto v is given by:
p =
hu,vi
hv,vi
v
For notational convenience, we set ß =
hu,vi
hv,vi
. Then
hp,u-pi = hp,ui-hp,pi
= hßv,ui-hßv,ßvi
= ßhv,ui-ß
2
hv,vi
=
hu,vi
hv,vi
hv,ui-
hu,vi
2
hv,vi
2
hv,vi
=
hu,vi
2
hv,vi
-
hu,vi
2
hv,vi
= 0
Therefore they are orthogonal. 
Theorem 1.0.4 (Cauchy-Schwarz Inequality). Suppose that v and v are vectors in an inner product space.
Then
|hu,vi|=||u||·||v||
Proof: Let p be the orthogonal projection of u onto v. From the previous result, p and u-p are orthogonal.
We may then apply the Pythagorean Theorem to get
||p||
2
+||u-p||
2
= ||(p)+(u-p)||
2
= ||u||
2
Subtracting||u-p||
2
from both sides and noting 0=||u-p||
2
gives:
||p||
2
=||u||
2
-||u-p||
2
=||u||
2
In the proof of the previous theorem, we found that
||p||
2
=hp,pi =
hu,vi
2
hv,vi
=
hu,vi
2
||v||
2
Page 3


Inner Product Spaces
1. Preliminaries
An inner product space is a vector space V along with a function h,i called an inner product which
associates each pair of vectors u,v with a scalarhu,vi, and which satis?es:
(1) hu,ui= 0 with equality if and only if u =0
(2) hu,vi =hv,ui and
(3) hau+v,wi =ahu,wi+hv,wi
Combining (2) and (3), we also havehu,av+wi =ahu,vi+hu,wi. Condition (1) is called positive de?nite,
condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner
product is an example of a positive de?nite, symmetric bilinear function or form on the vector space V.
De?nition 1.0.1. Let V be an inner product space and u and v be vectors in V. We make the following
de?nitions:
(1) The length or norm of the vector u is:
||u|| =
p
hu,ui
(2) The distance between u and v is:
||u-v||
(3) The angle between u and v is:
? = cos
-1

hu,vi
||u||·||v||

(4) We say that u and v are orthogonal if
hu,vi = 0
(5) The orthogonal projection of u onto the space spanned by v is:
p =

hu,vi
hv,vi

v
Note that, a priori, we do not know that-1=
hu,vi
||u||·||v||
= 1 so ? may not be de?ned. Later we will show that
these bounds are valid and so our de?nition of ? is also valid. When referring to (5), we will usually say “the
projection of u onto v”.
Now we give some preliminary results:
Theorem 1.0.2 (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then
||u±v||
2
=||u||
2
+||v||
2
Proof: We’ll just do the case ||u+v||
2
as the argument for the other case is similar. Note, as u and v are
orthogonal, hu,vi = 0. Now:
||u+v||
2
= hu+v,u+vi
= hu,ui+hu,vi+hv,ui+hv,vi
= hu,ui+hv,vi
= ||u||
2
+||v||
2

Theorem 1.0.3. Suppose that p is the orthogonal projection of u onto the space spanned by v. Then p and
u-p are orthogonal.
Proof: Recall that the projection of u onto v is given by:
p =
hu,vi
hv,vi
v
For notational convenience, we set ß =
hu,vi
hv,vi
. Then
hp,u-pi = hp,ui-hp,pi
= hßv,ui-hßv,ßvi
= ßhv,ui-ß
2
hv,vi
=
hu,vi
hv,vi
hv,ui-
hu,vi
2
hv,vi
2
hv,vi
=
hu,vi
2
hv,vi
-
hu,vi
2
hv,vi
= 0
Therefore they are orthogonal. 
Theorem 1.0.4 (Cauchy-Schwarz Inequality). Suppose that v and v are vectors in an inner product space.
Then
|hu,vi|=||u||·||v||
Proof: Let p be the orthogonal projection of u onto v. From the previous result, p and u-p are orthogonal.
We may then apply the Pythagorean Theorem to get
||p||
2
+||u-p||
2
= ||(p)+(u-p)||
2
= ||u||
2
Subtracting||u-p||
2
from both sides and noting 0=||u-p||
2
gives:
||p||
2
=||u||
2
-||u-p||
2
=||u||
2
In the proof of the previous theorem, we found that
||p||
2
=hp,pi =
hu,vi
2
hv,vi
=
hu,vi
2
||v||
2
So we have
hu,vi
2
||v||
2
=||u||
2
hu,vi
2
=||u||
2
·||v||
2
which ?nally leads to
|hu,vi|=||u||·||v||

2. Examples of Inner Product Spaces
2.1. Example: R
n
. Just as R
n
is our template for a real vector space, it serves in the same way as the
archetypical inner product space. The usual inner product onR
n
is called the dot product or scalar product
onR
n
. It is de?ned by:
hx,yi =x
T
y
where the right-hand side is just matrix multiplication. In particular, if
x =
?
?
?
?
?
?
?
?
x
1
x
2
.
.
.
x
n
?
?
?
?
?
?
?
?
and y =
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
then
hx,yi =x
T
y = (x
1
,...,x
n
)
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
=x
1
y
1
+···+x
n
y
n
****PROOF OF DOT PRODUCT BEING INNER PRODUCT GOES HERE****
****GEOMETRIC PROOF OF ORTHOGONAL PROJECTIONS GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
p =

x
T
y
y
T
y

y
x
T
y =||x||·||y||cos(?)
An alternate inner product can be de?ned onR
n
by:
hx,yi =x
T
*A*y
where the right-hand side is just matrix multiplication. The n×n matrix A must be a type of matrix known as
a symmetric, positive de?nite matrix in order for this to satisfy the conditions of an inner product. If we choose
A to be a symmetric matrix in which all of its entries are non-negative and has only positive entries on the main
diagonal, then it will be such a matrix. (More generally a symmetric, positive de?nite matrix is a symmetric
matrix with only positive eigenvalues.)
Page 4


Inner Product Spaces
1. Preliminaries
An inner product space is a vector space V along with a function h,i called an inner product which
associates each pair of vectors u,v with a scalarhu,vi, and which satis?es:
(1) hu,ui= 0 with equality if and only if u =0
(2) hu,vi =hv,ui and
(3) hau+v,wi =ahu,wi+hv,wi
Combining (2) and (3), we also havehu,av+wi =ahu,vi+hu,wi. Condition (1) is called positive de?nite,
condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner
product is an example of a positive de?nite, symmetric bilinear function or form on the vector space V.
De?nition 1.0.1. Let V be an inner product space and u and v be vectors in V. We make the following
de?nitions:
(1) The length or norm of the vector u is:
||u|| =
p
hu,ui
(2) The distance between u and v is:
||u-v||
(3) The angle between u and v is:
? = cos
-1

hu,vi
||u||·||v||

(4) We say that u and v are orthogonal if
hu,vi = 0
(5) The orthogonal projection of u onto the space spanned by v is:
p =

hu,vi
hv,vi

v
Note that, a priori, we do not know that-1=
hu,vi
||u||·||v||
= 1 so ? may not be de?ned. Later we will show that
these bounds are valid and so our de?nition of ? is also valid. When referring to (5), we will usually say “the
projection of u onto v”.
Now we give some preliminary results:
Theorem 1.0.2 (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then
||u±v||
2
=||u||
2
+||v||
2
Proof: We’ll just do the case ||u+v||
2
as the argument for the other case is similar. Note, as u and v are
orthogonal, hu,vi = 0. Now:
||u+v||
2
= hu+v,u+vi
= hu,ui+hu,vi+hv,ui+hv,vi
= hu,ui+hv,vi
= ||u||
2
+||v||
2

Theorem 1.0.3. Suppose that p is the orthogonal projection of u onto the space spanned by v. Then p and
u-p are orthogonal.
Proof: Recall that the projection of u onto v is given by:
p =
hu,vi
hv,vi
v
For notational convenience, we set ß =
hu,vi
hv,vi
. Then
hp,u-pi = hp,ui-hp,pi
= hßv,ui-hßv,ßvi
= ßhv,ui-ß
2
hv,vi
=
hu,vi
hv,vi
hv,ui-
hu,vi
2
hv,vi
2
hv,vi
=
hu,vi
2
hv,vi
-
hu,vi
2
hv,vi
= 0
Therefore they are orthogonal. 
Theorem 1.0.4 (Cauchy-Schwarz Inequality). Suppose that v and v are vectors in an inner product space.
Then
|hu,vi|=||u||·||v||
Proof: Let p be the orthogonal projection of u onto v. From the previous result, p and u-p are orthogonal.
We may then apply the Pythagorean Theorem to get
||p||
2
+||u-p||
2
= ||(p)+(u-p)||
2
= ||u||
2
Subtracting||u-p||
2
from both sides and noting 0=||u-p||
2
gives:
||p||
2
=||u||
2
-||u-p||
2
=||u||
2
In the proof of the previous theorem, we found that
||p||
2
=hp,pi =
hu,vi
2
hv,vi
=
hu,vi
2
||v||
2
So we have
hu,vi
2
||v||
2
=||u||
2
hu,vi
2
=||u||
2
·||v||
2
which ?nally leads to
|hu,vi|=||u||·||v||

2. Examples of Inner Product Spaces
2.1. Example: R
n
. Just as R
n
is our template for a real vector space, it serves in the same way as the
archetypical inner product space. The usual inner product onR
n
is called the dot product or scalar product
onR
n
. It is de?ned by:
hx,yi =x
T
y
where the right-hand side is just matrix multiplication. In particular, if
x =
?
?
?
?
?
?
?
?
x
1
x
2
.
.
.
x
n
?
?
?
?
?
?
?
?
and y =
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
then
hx,yi =x
T
y = (x
1
,...,x
n
)
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
=x
1
y
1
+···+x
n
y
n
****PROOF OF DOT PRODUCT BEING INNER PRODUCT GOES HERE****
****GEOMETRIC PROOF OF ORTHOGONAL PROJECTIONS GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
p =

x
T
y
y
T
y

y
x
T
y =||x||·||y||cos(?)
An alternate inner product can be de?ned onR
n
by:
hx,yi =x
T
*A*y
where the right-hand side is just matrix multiplication. The n×n matrix A must be a type of matrix known as
a symmetric, positive de?nite matrix in order for this to satisfy the conditions of an inner product. If we choose
A to be a symmetric matrix in which all of its entries are non-negative and has only positive entries on the main
diagonal, then it will be such a matrix. (More generally a symmetric, positive de?nite matrix is a symmetric
matrix with only positive eigenvalues.)
2.2. Example: R
m×n
. Suppose A = (a
ij
) and B = (b
ij
) are matrices in R
m×n
. The usual inner product on
R
m×n
is given by:
hA,Bi =
m
X
i=1
n
X
j=1
a
ij
b
ij
Note that this is the sum of the point-wise products of the elements of A and B.
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
2.3. Example: P
n
. Here we will describe a type of inner product on P
n
which we will term a discrete inner
product on P
n
. Let {x
1
,...,x
n
} be distinct real numbers. If p(x) is a polynomial in P
n
, then it has degree at
most (n-1), and therefore has at most (n-1) roots. This means that p(x
i
)6= 0 for at least one i. We know
de?ne an inner product by:
hp(x),q(x)i =
n
X
i=1
p(x
i
)q(x
i
)
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
Since every polynomial is continuous at every real number, we can use the next example of an inner product
as an inner product onP
n
. Each of these are a continuous inner product onP
n
.
2.4. Example: C[a,b]. An inner product on C[a,b] is given by:
hf(x),g(x)i =
Z
b
a
f(x)g(x)w(x)dx
where w(x) is some continuous, positive real-valued function on [a,b]. The standard inner product on C[a,b]
is where w(x) = 1 in the above de?nition.
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
3. Subspaces, Orthogonal Complements and Projections
De?nition 3.0.1. Let U and V be subspaces of a vector space W such that U nV ={0}. The direct sum of
U and V is the set U?V ={u+v|u?U and v?V}.
De?nition 3.0.2. Let S be a subspace of the inner product space V. The the orthogonal complement of S
is the set S
?
={v?V |hv,si = 0 for all s?S}.
Theorem 3.0.3. (1) If U and V are subspaces of a vector space W with UnV ={0}, then U?V is also
a subspace of W.
(2) If S is a subspace of the inner product space V, then S
?
is also a subspace of V.
Proof: (1.) Notethat0+0 =0isinU?V. Nowsupposew
1
,w
2
?U?V, thenw
1
=u
1
+v
1
andw
2
=u
2
+v
2
with u
i
? U and v
i
? V and w
1
+w
2
= (u
1
+v
1
)+(u
2
+v
2
) = (u
1
+u
2
)+(v
1
+v
2
). Since U and V are
subspaces, itfollowsthatw
1
+w
2
?U?V. Supposenowthataisascalar, thenaw
1
=a(u
1
+v
1
) =au
1
+av
1
.
As above, it then follows that aw
1
?U?V. Thus U?V is a subspace for W.
Page 5


Inner Product Spaces
1. Preliminaries
An inner product space is a vector space V along with a function h,i called an inner product which
associates each pair of vectors u,v with a scalarhu,vi, and which satis?es:
(1) hu,ui= 0 with equality if and only if u =0
(2) hu,vi =hv,ui and
(3) hau+v,wi =ahu,wi+hv,wi
Combining (2) and (3), we also havehu,av+wi =ahu,vi+hu,wi. Condition (1) is called positive de?nite,
condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner
product is an example of a positive de?nite, symmetric bilinear function or form on the vector space V.
De?nition 1.0.1. Let V be an inner product space and u and v be vectors in V. We make the following
de?nitions:
(1) The length or norm of the vector u is:
||u|| =
p
hu,ui
(2) The distance between u and v is:
||u-v||
(3) The angle between u and v is:
? = cos
-1

hu,vi
||u||·||v||

(4) We say that u and v are orthogonal if
hu,vi = 0
(5) The orthogonal projection of u onto the space spanned by v is:
p =

hu,vi
hv,vi

v
Note that, a priori, we do not know that-1=
hu,vi
||u||·||v||
= 1 so ? may not be de?ned. Later we will show that
these bounds are valid and so our de?nition of ? is also valid. When referring to (5), we will usually say “the
projection of u onto v”.
Now we give some preliminary results:
Theorem 1.0.2 (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then
||u±v||
2
=||u||
2
+||v||
2
Proof: We’ll just do the case ||u+v||
2
as the argument for the other case is similar. Note, as u and v are
orthogonal, hu,vi = 0. Now:
||u+v||
2
= hu+v,u+vi
= hu,ui+hu,vi+hv,ui+hv,vi
= hu,ui+hv,vi
= ||u||
2
+||v||
2

Theorem 1.0.3. Suppose that p is the orthogonal projection of u onto the space spanned by v. Then p and
u-p are orthogonal.
Proof: Recall that the projection of u onto v is given by:
p =
hu,vi
hv,vi
v
For notational convenience, we set ß =
hu,vi
hv,vi
. Then
hp,u-pi = hp,ui-hp,pi
= hßv,ui-hßv,ßvi
= ßhv,ui-ß
2
hv,vi
=
hu,vi
hv,vi
hv,ui-
hu,vi
2
hv,vi
2
hv,vi
=
hu,vi
2
hv,vi
-
hu,vi
2
hv,vi
= 0
Therefore they are orthogonal. 
Theorem 1.0.4 (Cauchy-Schwarz Inequality). Suppose that v and v are vectors in an inner product space.
Then
|hu,vi|=||u||·||v||
Proof: Let p be the orthogonal projection of u onto v. From the previous result, p and u-p are orthogonal.
We may then apply the Pythagorean Theorem to get
||p||
2
+||u-p||
2
= ||(p)+(u-p)||
2
= ||u||
2
Subtracting||u-p||
2
from both sides and noting 0=||u-p||
2
gives:
||p||
2
=||u||
2
-||u-p||
2
=||u||
2
In the proof of the previous theorem, we found that
||p||
2
=hp,pi =
hu,vi
2
hv,vi
=
hu,vi
2
||v||
2
So we have
hu,vi
2
||v||
2
=||u||
2
hu,vi
2
=||u||
2
·||v||
2
which ?nally leads to
|hu,vi|=||u||·||v||

2. Examples of Inner Product Spaces
2.1. Example: R
n
. Just as R
n
is our template for a real vector space, it serves in the same way as the
archetypical inner product space. The usual inner product onR
n
is called the dot product or scalar product
onR
n
. It is de?ned by:
hx,yi =x
T
y
where the right-hand side is just matrix multiplication. In particular, if
x =
?
?
?
?
?
?
?
?
x
1
x
2
.
.
.
x
n
?
?
?
?
?
?
?
?
and y =
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
then
hx,yi =x
T
y = (x
1
,...,x
n
)
?
?
?
?
?
?
?
?
y
1
y
2
.
.
.
y
n
?
?
?
?
?
?
?
?
=x
1
y
1
+···+x
n
y
n
****PROOF OF DOT PRODUCT BEING INNER PRODUCT GOES HERE****
****GEOMETRIC PROOF OF ORTHOGONAL PROJECTIONS GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
p =

x
T
y
y
T
y

y
x
T
y =||x||·||y||cos(?)
An alternate inner product can be de?ned onR
n
by:
hx,yi =x
T
*A*y
where the right-hand side is just matrix multiplication. The n×n matrix A must be a type of matrix known as
a symmetric, positive de?nite matrix in order for this to satisfy the conditions of an inner product. If we choose
A to be a symmetric matrix in which all of its entries are non-negative and has only positive entries on the main
diagonal, then it will be such a matrix. (More generally a symmetric, positive de?nite matrix is a symmetric
matrix with only positive eigenvalues.)
2.2. Example: R
m×n
. Suppose A = (a
ij
) and B = (b
ij
) are matrices in R
m×n
. The usual inner product on
R
m×n
is given by:
hA,Bi =
m
X
i=1
n
X
j=1
a
ij
b
ij
Note that this is the sum of the point-wise products of the elements of A and B.
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
2.3. Example: P
n
. Here we will describe a type of inner product on P
n
which we will term a discrete inner
product on P
n
. Let {x
1
,...,x
n
} be distinct real numbers. If p(x) is a polynomial in P
n
, then it has degree at
most (n-1), and therefore has at most (n-1) roots. This means that p(x
i
)6= 0 for at least one i. We know
de?ne an inner product by:
hp(x),q(x)i =
n
X
i=1
p(x
i
)q(x
i
)
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
Since every polynomial is continuous at every real number, we can use the next example of an inner product
as an inner product onP
n
. Each of these are a continuous inner product onP
n
.
2.4. Example: C[a,b]. An inner product on C[a,b] is given by:
hf(x),g(x)i =
Z
b
a
f(x)g(x)w(x)dx
where w(x) is some continuous, positive real-valued function on [a,b]. The standard inner product on C[a,b]
is where w(x) = 1 in the above de?nition.
****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE****
****SPECIFIC EXAMPLE GOES HERE****
3. Subspaces, Orthogonal Complements and Projections
De?nition 3.0.1. Let U and V be subspaces of a vector space W such that U nV ={0}. The direct sum of
U and V is the set U?V ={u+v|u?U and v?V}.
De?nition 3.0.2. Let S be a subspace of the inner product space V. The the orthogonal complement of S
is the set S
?
={v?V |hv,si = 0 for all s?S}.
Theorem 3.0.3. (1) If U and V are subspaces of a vector space W with UnV ={0}, then U?V is also
a subspace of W.
(2) If S is a subspace of the inner product space V, then S
?
is also a subspace of V.
Proof: (1.) Notethat0+0 =0isinU?V. Nowsupposew
1
,w
2
?U?V, thenw
1
=u
1
+v
1
andw
2
=u
2
+v
2
with u
i
? U and v
i
? V and w
1
+w
2
= (u
1
+v
1
)+(u
2
+v
2
) = (u
1
+u
2
)+(v
1
+v
2
). Since U and V are
subspaces, itfollowsthatw
1
+w
2
?U?V. Supposenowthataisascalar, thenaw
1
=a(u
1
+v
1
) =au
1
+av
1
.
As above, it then follows that aw
1
?U?V. Thus U?V is a subspace for W.
For (2.), ?rst note that 0? S
?
. Now suppose that v
1
and v
2
? S
?
. Then hv
1
,si =hv
2
,si = 0 for all s? S.
So hv
1
+v
2
,si =hv
1
,si+hv
2
,si = 0+0 = 0 for all s? S. Thus v
1
+v
2
? S
?
. Similarly, if a is a scalar, then
hav
1
,si =ahv
1
,si =a·0 = 0 for all s?S. Thus S
?
is a subspace of V. 
Theorem 3.0.4. If U and V are subspaces of W with UnV ={0} and w?U?V, then w =u+v for unique
u?U and v?V.
Proof: Write w =u
1
+v
1
and w =u
2
+v
2
. Then u
1
+v
1
=u
2
+v
2
?u
1
-u
2
=v
2
-v
1
?u
1
-u
2
=0 =
v
2
-v
1
?u
1
=u
2
and v
2
=v
1
. 
Recall that one of the axioms of an inner product is that hx,xi = 0 with equality if and only if x = 0. An
immediate consequence of this is that SnS
?
= 0.
De?nition 3.0.5. Let S be a subspace of the inner product space V and let {x
1
,...,x
n
} be a basis for S such
that hx
i
,x
j
i = 0 if i6= j, then this basis is called an orthogonal basis. Furthermore, if hx
i
,x
i
i = 1 then this
basis is called an orthonormal basis.
De?nition 3.0.6. Let S be a ?nite dimensional subspace of the inner product space V and let {x
1
,...,x
n
} be
an orthogonal basis for S. If v is any vector in V then the orthogonal projection of v onto S is the vector:
p =
n
X
i=1
hv,x
i
i
hx
i
,x
i
i
x
i
Note that if{x
1
,...,x
n
} is an orthonormal basis, then we have the simpler expression:
p =
n
X
i=1
hv,x
i
ix
i
Also in the special case whereS is spanned be the single vectorx
1
, thenp is just the usual orthogonal projection
of v onto S, which is the line spanned by x
1
.
Now we can prove the main theorem of this section:
Theorem 3.0.7. Let S be a ?nite dimensional subspace of the inner product space V and v be some vector in
V. Moreover let{x
1
,...,x
n
} be an orthogonal basis for S and p be the orthogonal projection of v onto S. Then
(1) v-p?S
?
.
(2) V =S?S
?
.
(3) If y is any vector in S with y6=p, then ||v-p||<||v-y||
Note that part (3.) says that p is the vector in S which is closest to v. Moreover, an immediate consequence
of (2.) is that the orthogonal projection p of v onto S is independent of the choice of orthogonal basis for S.
Proof: (1.) We need to show that p and v-p are orthogonal. So consider hp,v-pi =hp,vi-hp,pi. Note
that hx
i
,x
j
i = 0 when i6=j, so that
hp,vi =
n
X
i=1
hc
i
x
i
,vi with c
i
=
hv,x
i
i
hx
i
,x
i
i
andhp,pi =
n
X
i=1
hc
i
x
i
,c
i
x
i
i?
hp,vi =
n
X
i=1
hv,x
i
i
hx
i
,x
i
i
hx
i
,vi andhp,pi =
n
X
i=1
hv,x
i
i
2
hx
i
,x
i
i
2
hx
i
,x
i
i