Mathematics Exam  >  Mathematics Notes  >  Mathematics for IIT JAM, GATE, CSIR NET, UGC NET  >  Distribution of quadratic forms, CSIR-NET Mathematical Sciences

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET PDF Download

2.1 Quadratic Forms

For a k × k symmetric matrix A = {aij} the quadratic function of k variables x = (x1,...,xn)' defined by
Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

is called the quadratic form with matrix A. If A is not symmetric, we can have an equivalent expression/quadratic form replacing A by (A + A')/2.

Definition 1. Q(x) and the matrix A are called positive definite if 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

and positive semi-definite if

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

For negative definite and negative semi-definite, replace the > and ≥ in the above definitions by < and ≤, respectively

Theorem 1. A symmetric matrix A is positive definite if and only if it has a Cholesky decomposition A = R'R with strictly positive diagonal elements in R, so that R−1 exists. (In practice this means that none of the diagonal elements of R are very close to zero.)

Proof. The “if” part is proven by construction. The Cholesky decomposition, R, is constructed a row at a time and the diagonal elements are evaluated as the square roots of expressions calculated from the current row of A and previous rows of R. If the expression whose square root is to be calculated is not positive then you can determine a non-zero x ∈Rk for which x'Ax ≤ 0.

Suppose that A = R'R with R invertible. Then 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

with equality only if Rx = 0. But if R−1 exists then x = R−10 must also be zero.


Transformation of Quadratic Forms:

Theorem 2. Suppose that B is a k × k nonsingular matrix. Then the quadratic form Q∗(y) = y'B'ABy is positive definite if and only if Q(x) = x'Ax is positive definite. Similar results hold for positive semi-definite, negative definite and negative semi-definite.
Proof.
Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

where x = By ≠ 0 because y  ≠ 0 and B is nonsingular.

Theorem 3. For any k×k symmetric matrix A the quadratic form defined by A can be written using its spectral decomposition as

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

where the eigendecomposition of of A is Q'ΛQ with Λ diagonal with diagonal elements λi, i = 1,...,k, Q is the orthogonal matrix with the eigenvectors, qi, i = 1,...,k as its columns. (Be careful to distinguish the bold face Q, which is a matrix, from the unbolded Q(x), which is the quadratic form.) Proof. For any Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET Then

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

This proof uses a common “trick” of expressing the scalar Q(x) as the trace of a 1×1 matrix so we can reverse the order of some matrix multiplications.
 

Corollary 1. A symmetric matrix A is positive definite if and only if its eigenvalues are all positive, negative definite if and only if its eignevalues are all negative, and positive semi-definite if all its eigenvalues are non-negative.

Corollary 2. rank(A) = rank(Λ) hence rank(A) equals the number of non-zero eigenvalues of A


2.2 Idempotent Matrices

Definition 2 (Idempotent). The k×k matrix A, is idempotent if A2 — AA — A.

Definition 3 (Projection matrices). A symmetric, idempotent matrix A is a projection matrix. The effect of the mapping x → Ax is orthogonal projection of x onto col(A).

Theorem 4. All the eigenvalues of an idempotent matrix are either zero or one.

Proof. Suppose that λ is an eigenvalue of the idempotent matrix A. Then there exists a non-zero x such that Ax — λx. But Ax — AAx because A is idempotent. Thus

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

and

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

for some non-zero x, which implies that λ — 0 or λ — 1. 

Corollary 3. The k×k symmetric matrix A is idempotent of rank(A) — r iff A has r eigenvalues equal to 1 and k−r eigenvalues equal to 0

Proof. A matrix A with r eigenvalues of 1 and k−r eigenvalues of zero has r non-zero eigenvalues and hence rank(A) — r. Because A is symmetric its eigendecomposition is A — QΛQ' for an orthogonal Q and a diagonal Λ. Because the eigenvalues of Λ are the same as those of A, they must be all zeros or ones. That is all the diagonal elements of Λ are zero or one. Hence Λ is idempotent, ΛΛ — Λ, and 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

is also idempotent.

Corollary 4. For a symmetric idempotent matrix A, we have tr(A) — rank(A), which is the dimension of col(A), the space into which A projects.

 

2.3 Expected Values and Covariance Matrices of Random Vectors

An k-dimensional vector-valued random variable (or, more simply, a random vector),X, is a k-vector composed of k scalar random variables

X = (X1,...,Xk)'

If the expected values of the component random variables are µ= E(Xi), i = 1,...,k then 

E(X) = µX = (µ1,...,µk)'

Suppose that Y = (Y1,...,Ym)' is an m-dimensional random vector, then the covariance of X and Y, written Cov(X,Y) is

ΣXY = Cov(X,Y) = E[(X −µX)(Y−µY)']

The variance-covariance matrix of X is

Var(X) = ΣXX = E[(X −µX)(X −µ§)

Suppose that c is a constant m-vector, A is a constant m×k matrix and Z = ZX + c is a linear transformation of X. Then E(Z) = AE(X) + c

and

Var(Z) = AVar(X)A'

If we let W = BY + d be a linear transformation of Y for suitably sized B and d then Cov(Z,W) = ACov(X,Y)B' 

Theorem 5. The variance-covariance matrix ΣX,X of X is a symmetric and positive semi-definite matrix

Proof. The result follows from the property that the variance of a scalar random variable is nonnegative. Suppose that b is any nonzero, constant k-vector. Then

0 ≤ Var(b'X) = b'ΣXXb

which is the positive, semi-definite condition.
 

2.4 Mean and Variance of Quadratic Forms 

Theorem 6. Let X be a k-dimensional random vector and A be a constant k×k symmetric matrix. If E(X) = µ and Var(X) = Σ, then

E(X'AX) = tr(AΣ) + µ'Aµ

Proof. 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

2.5 Distribution of Quadratic Forms in Normal Random Variables

Definition 4 (Non-Central χ2). If X is a (scalar) normal random variable with E(X) = µ and Var(X) = 1, then the random variableV = X2 is distributed as Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET which is called the noncentral χdistribution with 1 degree of freedom and non-centrality parameter λ2 — µ2. The mean and variance of V are

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

As described in the previous chapter, we are particularly interested in random n-vectors, Y , that have a spherical normal distribution. 

Theorem 7.Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NETbe an n-vector with a spherical normal distribution and A be an n × n symmetric matrix. Then the ratio Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NETdistribution with λ2 = µ'Aµ/σ2 if and only if A is idempotent with rank(A) = r

Proof. Suppose that A is idempotent (which, in combination with being symmetric, means that it is a projection matrix) and has rank(A) = r. Its eigendecomposition, A = V ΛV ', is such that V is orthogonal and Λ is n×n diagonal with exactly r = rank(A) ones and n−r zeros on the diagonal. Without loss of generality we can (and do) arrange the eigenvalues in decreasing order so that λ= 1, j = 1,...,r and λj = 0, j = r + 1,...,n Let X = V 'Y 

  Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

(Notice that the last sum is to j = r, not j = n.) However, Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NETDistribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET  Therefore

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

Corollary 5. For A a projection of rank r, (Y'AY)/σ2 has a central χdistribution if and only if Aµ = 0

Proof. The χ2 r distribution will be central if and only if 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET 

Corollary 6. In the full-rank Gaussian linear model,Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET  the residual sum of squares,  Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET has a central  Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET distribution.

Proof. In the full rank model with the QR decomposition of X given by

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

and R invertible, the fitted values are Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET and the residuals are Q2Q2y so the residual sum of squares is the quadratic form Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET. The matrix defining the quadratic form, Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET, is a projection matrix. It is obviously symmetric and it is idempotent because Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET As 

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

the ratio

Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

and the RSS has a central  Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET  distribution. 

The document Distribution of quadratic forms, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET is a part of the Mathematics Course Mathematics for IIT JAM, GATE, CSIR NET, UGC NET.
All you need of Mathematics at this link: Mathematics
556 videos|198 docs

FAQs on Distribution of quadratic forms, CSIR-NET Mathematical Sciences - Mathematics for IIT JAM, GATE, CSIR NET, UGC NET

1. What is the definition of a quadratic form?
Ans. A quadratic form is a function that takes a set of variables and returns a quadratic expression in those variables. It can be written as a sum of terms, where each term consists of the product of a coefficient and the square of a variable.
2. How is the distribution of quadratic forms related to CSIR-NET Mathematical Sciences exam?
Ans. The distribution of quadratic forms is a topic that is often tested in the CSIR-NET Mathematical Sciences exam. Candidates are expected to have a good understanding of the properties and characteristics of quadratic forms, as well as the techniques used to analyze their distributions.
3. What are some important properties of quadratic forms?
Ans. Some important properties of quadratic forms include symmetry, positive definiteness, and eigenvalues. Symmetry refers to the fact that the quadratic form remains the same when the variables are interchanged. Positive definiteness implies that the quadratic form is always positive, except when all the variables are zero. The eigenvalues of a quadratic form provide valuable information about its behavior.
4. How can the distribution of quadratic forms be analyzed?
Ans. The distribution of quadratic forms can be analyzed using various techniques, such as matrix algebra, characteristic roots, and orthogonal transformations. These techniques help in determining the mean, variance, and other statistical properties of the quadratic form.
5. Can you provide an example of a quadratic form and its distribution?
Ans. Sure! Let's consider the quadratic form Q(x,y) = 2x^2 + 3xy + 4y^2. By analyzing the properties of this quadratic form, we can determine its distribution, including its mean and variance. This analysis will help in understanding the behavior and characteristics of the quadratic form in various scenarios.
Explore Courses for Mathematics exam
Signup for Free!
Signup to see your scores go up within 7 days! Learn & Practice with 1000+ FREE Notes, Videos & Tests.
10M+ students study on EduRev
Related Searches

practice quizzes

,

ppt

,

CSIR NET

,

Distribution of quadratic forms

,

study material

,

Important questions

,

shortcuts and tricks

,

Exam

,

UGC NET

,

pdf

,

Distribution of quadratic forms

,

Free

,

video lectures

,

CSIR-NET Mathematical Sciences | Mathematics for IIT JAM

,

MCQs

,

Distribution of quadratic forms

,

Sample Paper

,

CSIR NET

,

Summary

,

GATE

,

UGC NET

,

CSIR-NET Mathematical Sciences | Mathematics for IIT JAM

,

CSIR-NET Mathematical Sciences | Mathematics for IIT JAM

,

GATE

,

UGC NET

,

Extra Questions

,

Viva Questions

,

Objective type Questions

,

GATE

,

Semester Notes

,

mock tests for examination

,

Previous Year Questions with Solutions

,

past year papers

,

CSIR NET

;