Orthonormal Basis - Matrix Algebra, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET PDF Download

Orthonormal Bases

The canonical/standard basis has many useful properties.

Each of the standard basis vectors has unit length:

The standard basis vectors are orthogonal (in other words, at right angles or perpendicular).

This is summarized by

where δ_ij is the Kronecker delta. Notice that the Kronecker delta gives the entries of the identity matrix.
Given column vectors v and w, we have seen that the dot product v • w is the same as the matrix multiplication v^T w. This is the inner product on Rn. We can also form the outer product vw^T , which gives a square matrix.

The outer product on the standard basis vectors is interesting. Set

In short, II_i is the diagonal square matrix with a 1 in the ith diagonal position and zeros everywhere else.¹

Notice that 

Moreover, for a diagonal matrix D with diagonal entries λ₁ ..... λ_n, we can write

Other bases that share these properties should behave in many of the same ways as the standard basis. As such, we will study:

Orthogonal bases {v₁,.....v_n}:

v_i v_j = 0 if i ≠ j

In other words, all vectors in the basis are perpendicular.

Orthonormal bases {u₁ ,..... u_n}:

u_i u_j = δ_ij :

In addition to being orthogonal, each vector has unit length.

Suppose T = {u_1,......, u_n} is an orthonormal basis for Rⁿ. Since T is a basis, we can write any vector v uniquely as a linear combination of the vectors in T :

v = c¹u₁ + .... cⁿu_n:

Since T is orthonormal, there is a very easy way to nd the coecients of this linear combination. By taking the dot product of v with any of the vectors in T , we get:

This proves the theorem:

Theorem. For an orthonormal basis {u₁,....u_n} any, any vector v can be expressed

Relating Orthonormal Bases

Suppose T = {u₁,.... u_n} and R = {w₁,.... w_n} are two orthonormal bases for Rⁿ. Then:

As such, the matrix for the change of basis from T to R is given by

Consider the product PP^T in this case.

In the equality (*) is explained below. So assuming (*) holds, we have shown that PP^T = In, which implies that

P^T = P ^-1.

The equality in the line (*) says that   To see this, we examine   for an arbitrary vector v. We can nd constants c^j such that  so that: 

since all terms with i ≠ j vanish

= v.

Then as a linear transformation,  xes every vector, and thus must be the identity In.

Definition A matrix P is orthogonal if P^-1 = P^T.

Then to summarize,

Theorem : A change of basis matrix P relating two orthonormal bases is an orthogonal matrix. i.e.

P^-1 = P^T.

Example : Consider R³ with the orthonormal basis

Let R be the standard basis {e₁, e₂, e₃}. Since we are changing from the standard basis to a new basis, then the columns of the change of basis matrix are exactly the images of the standard basis vectors. Then the change of basis matrix from R to S is given by:

From our theorem, we observe that:

We can check that P ^T P = I_n by a lengthy computation, or more simply, notice that

We are using orthonormality of the u_i for the matrix multiplication above.

Orthonormal Change of Basis and Diagonal Matrices. Suppose D is a diagonal matrix, and we use an orthogonal matrix P to change to a new basis. Then the matrix M of D in the new basis is:

M = P DP^-1 = P DP^T :

Now we calculate the transpose of M .

So we see the matrix P DP^T is symmetric!